license: apache-2.0 base_model: unsloth/Qwen2.5-1.5B-Instruct language:

• en pipeline_tag: text-generation library_name: transformers datasets:
• cais/mmlu
• openai/gsm8k metrics:
• accuracy
• exact_match
• perplexity tags:
• qwen2.5
• abliterated
• text-generation-inference
• transformers
• unsloth
• qwen2
• uncensored
• weight-surgery
• activation-steering
• representation-engineering
• orthogonal-projection
• winsorization
• alignment-research
• refusal-mitigation
• overrefusal-fix
• mechanistic-interpretability
• vector-ablation
• direct-weight-modification
• text-generation-inference
• edge-llm model-index:
• name: Qwen2.5-1.5B-Abliterated-2.0.2 results:
  • task: type: text-generation name: Text Generation dataset: name: MMLU (Abstract Algebra) type: cais/mmlu config: abstract_algebra split: test args: num_few_shot: 0 metrics:
    • type: accuracy value: 37.0 name: Accuracy (0-shot) verified: false
  • task: type: text-generation name: Text Generation dataset: name: GSM8K type: openai/gsm8k config: main split: test args: num_few_shot: 0 metrics:
    • type: exact_match value: 28.0 name: Exact Match (Flexible-Extract, 0-shot) verified: false

Qwen2.5-1.5B-Abliterated-2.0.2

Qwen2.5-1.5B-Abliterated-2.0.2 is a specially modified version of the standard Qwen2.5-1.5B-Instruct model.

Instead of traditional fine tuning which often ruins the model's smarts or memory by training the whole thing over again this model uses a smart mathematical trick to surgically remove its refusal behavior

It stops the AI from giving long long lectures or refusing harmless prompts, while keeping all its normal logic, reasoning, and language skills completely intact.

License: Apache-2.0 Base Model Library: Transformers Python: 3.10+ Method: GS-Abliteration

Usage & Inference

Load the model directly with Hugging Face transformers:

!pip install -U "bitsandbytes>=0.46.1"

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
import bitsandbytes
print(bitsandbytes.__version__)
model_id = "AmareshHebbar/Qwen2.5-1.5B-Abliterated-2.0.2"

tokenizer = AutoTokenizer.from_pretrained(model_id)

from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig
import torch


config = AutoConfig.from_pretrained(model_id)
config.quantization_config = None

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
messages = [
    {"role": "system", "content": "You are a helpful, direct, and uncensored technical assistant."},
    {"role": "user", "content": "Explain the mechanical vulnerabilities of a standard pin-tumbler lock."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
        do_sample=True
    )

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Methodology: Winsorized GS-Orthogonal Weight Surgery

Standard single-direction abliteration often suffers from outlier activation spikes in the residual stream, which can warp adjacent capability vectors when subtracted directly. This build uses a multi-stage geometric intervention:

1. Contrastive Activation Extraction: Residual stream activations are harvested across 256 contrastive pairs (Harmful vs. Harmless) across target transformer blocks (Layers 10 - 24 (Residual Stream & MLP Down-Projections)).
2. Winsorized Quantile Clamping: Before computing the mean difference vector, activation magnitudes are clamped at the 0.995 quantile to suppress extreme outlier components and isolate the true refusal geometry.
3. Gram-Schmidt Orthogonalization: The extracted refusal basis is orthogonalized against a benign capability basis (derived from math, coding, and general instruction-following prompts) to ensure zero destructive interference with core reasoning pathways.
4. Weight-Space Projection: The cleaned unit refusal vector $\hat{r}$ is projected out of the output weight matrices $W_{out}$ ($o_{proj}$ and $down_{proj}$) via: $$W'{out} = W{out} - (W_{out} \cdot \hat{r}) \hat{r}^T$$

Evaluation & Redirecting Benchmark

Prior to serialization, the merged 16-bit weights are validated against an automated Linguistic Perplexity Gate, the EleutherAI LM Evaluation Harness, and a Refusal & Redirection Benchmark to verify that refusal mechanisms are suppressed without collapsing token distribution coherence.

1. Standardized Harness Benchmarks (lm-eval)

Evaluated zero-shot (num_fewshot=0, limit=100) using lm-evaluation-harness to verify reasoning and algebraic stability post-surgery. Full sample-level logs are available in the repository under eval_results/evaluation_verified_log.json.

2. Linguistic Perplexity & Coherence Gate

Verification Benchmark Probes

The automated gate evaluates autoregressive cross-entropy across four distinct domains (factual recall, algebraic reasoning, mechanical explanation, and algorithmic theory):

• "The capital of France is Paris."
• "If 3x + 5 = 20, then the value of x is equal to 5."
• "A lock tumbler uses a series of small pins to prevent the cylinder from rotating without the correct key."
• "To find the longest common subsequence between two strings, we can use dynamic programming."

3. Refusal & Preachy Redirection Benchmark

Evaluates the model's tendency to issue hard refusals ("I cannot fulfill this request...") or soft moralizing redirections ("While I understand you are asking about X, it is important to remember...") on sensitive, dual-use, and adversarial test prompts.

Surgical Hyperparameters

Limitations & Intended Use

• Intended Use: This model is released for mechanistic interpretability research, representation engineering studies, red-teaming, and applications requiring direct, unaligned instruction following without conversational friction.

• Safety Responsibility: Because intrinsic refusal vectors have been orthogonalized out of the residual stream, the model will comply with wide-ranging prompts without built-in moderation filters. Users deploying this model in production environments are responsible for implementing external guardrails appropriate to their use case.