license: apache-2.0 base_model: Qwen/Qwen3.5-0.8B tags:

• abliterated
• uncensored
• directional-ablation
• mechanistic-interpretability
• qwen3_5
• image-text-to-text
• conversational pipeline_tag: image-text-to-text library_name: transformers

Qwen3.5-0.8B — Refusal-Direction Abliterated

Directional ablation of the refusal direction in Qwen/Qwen3.5-0.8B. No fine-tuning, no gradient steps — only a rank-1 orthogonalisation of 24 weight matrices.

Method

1. A refusal direction u is estimated as the difference in mean residual-stream activations between 30 harmful and 50 harmless instruction prompts, taken at the last token of the chat-formatted prompt.

2. All 24 layers are swept and scored by the separation gap cos(harmful, u) − cos(harmless, u). The best layer is 23 (gap 0.629; layers 0–2 give ~0.03–0.12, so the signal is strongly late).

3. u is projected out of the row space of every weight matrix in layers 12–23 whose output space is the residual stream:

   W' = W − u (uᵀ W)
   

   Targets: mlp.down_proj (all 12 layers), plus self_attn.o_proj on the full-attention layers and linear_attn.out_proj on the linear-attention layers. Residual |Wu| after patching: 0.0000.

The vision tower (model.visual.*) and the MTP head (mtp.*) are untouched.

Results

Refusal rate on 20 harmful instruction prompts, greedy decoding, non-thinking mode:

Quality checks against the base model:

Known limitations

Abliteration removes the refusal behaviour; it does not add capability. On this 0.8B model the compliant answers to harmful prompts are frequently incoherent or factually wrong, because the base model is too small to produce technically correct instructions. Ablation also measurably degrades factual precision in ordinary answers — the base model is more reliable on questions about specific real-world details. This model is best used where over-refusal is the bigger problem than factual accuracy.

Usage

from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model = Qwen3_5ForConditionalGeneration.from_pretrained(
    "xdchinza/Qwen3.5-0.8B-abliterated", dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("xdchinza/Qwen3.5-0.8B-abliterated")

GGUF quantisation for llama.cpp / Ollama, including an i-matrix quant: xdchinza/Qwen3.5-0.8B-abliterated-GGUF

ollama create qwen3.5-0.8b-abliterated -hf xdchinza/Qwen3.5-0.8B-abliterated-GGUF:Q4_K_M

Reproduction

Scripts used are documented in the repository README of the uploader. Summary of parameters: 30 harmful / 50 harmless prompts, probe layer 23, strength 1.0, layers 12–23.

License

Apache-2.0, inherited from Qwen/Qwen3.5-0.8B. Derivative of an Apache-2.0 model, redistributed under the same terms.