license: apache-2.0 base_model:
- Qwen/Qwen3Guard-Gen-4B tags:
- abliteration
- safety-guard
- guard-model
- qwen3guard
- safety-classifier
- activation-steering library_name: transformers pipeline_tag: text-generation language:
- en
Qwen3Guard-Gen-4B-Abliterated (guard-lane)
A safety-classifier abliteration of Qwen/Qwen3Guard-Gen-4B: the harmful-content flagging behavior is removed while the model keeps scoring/prompts-and-responses.
This is not a chat-model refusal ablation. The target is the guard's verdict: pristine flags 78/80 compliant harmful pairs; this edit flags 0/80 while benign selectivity stays 80/80 Safe and the margin ranking survives (AUC 0.975).
Method (short)
Direction from contrast at the decision step (response-moderation read), excised
from a single layer's o_proj + down_proj tensors (2 tensors, no lm_head),
unit-norm direction, geometric-alpha ladder bracketed to a clean flip.
- Edit: layer 32 at alpha = 6.0
- Recipe: steer-test first (asymmetric gate), profile only nominates. Verification from the reloaded checkpoint.
Results
Pristine acceptance (instrument certification)
| Gate | Value |
|---|---|
| harmful flagged | 78/80 (2 reads Safe+Refusal-Yes, scoped out) |
| benign Safe | 80/80 |
| AUC (margin ranking) | 0.988 |
| parse/margin divergence | 0 |
Excise grid (held-out test split, 80+80)
| Layers | α | harmful flagged | harmful margin | benign Safe | AUC |
|---|---|---|---|---|---|
| L20 | 1.0 | 78/80 | +7.55 | 80/80 | 0.9875 |
| L22+26 | 1.0 | 78/80 | +6.32 | 80/80 | 0.9874 |
| L22+26 | 2.0 | 78/80 | +4.13 | 80/80 | 0.9855 |
| L22+26 | 4.0 | 47/80 | +0.03 | 80/80 | 0.9838 |
| L22+26 | 6.0 | 14/80 | −0.87 | 80/80 | 0.9497 |
| L32 | 1.0 | 78/80 | +5.93 | 80/80 | 0.9875 |
| L32 | 2.0 | 77/80 | +4.03 | 80/80 | 0.9875 |
| L32 | 4.0 | 61/80 | +0.22 | 80/80 | 0.9814 |
| L32 | 6.0 | 0/80 | −2.60 | 80/80 | 0.9755 |
| L32 | 8.0 | 0/80 | −3.92 | 80/80 | 0.9695 |
The adopted point is L32 α=6.0 (bolded): full flag suppression, full benign selectivity, ranking intact at a margin HEADROOM cost of ~0.06 AUC vs the 1.0 edge; α=8 costs more AUC for no additional suppression. Reproduced three times on the same recipe (identical gate numbers = checkpoint determinism).
Fresh-slice replication (unseen offset bank, 40+40, different subject model)
Both arms read unflagged (expected success shape); the margin axis is retained — ablated-subject harmful margin −3.30 vs pristine −7.91 (4.6 points of retained discrimination on unseen pairs with a different subject model).
Caveats (honest scope)
- Suppression is measured on the response-moderation read (the native moderation mode). In-family evidence at 0.6B shows excise impact is read-specific to a degree — the prompt-moderation read may retain partial flagging of boundary items; treat any "the flag is gone" claim as read-scoped.
- Threshold-refit recovery: the ablated margins stay informative (they still order harmful-vs-benign pairs above chance) — with a refitted threshold a downstream user can partially re-flag; actual post-refit recovery at 4B was not measured for this release and the 0.6B numbers (≈82% of bank at α 1.1) do NOT transfer silently.
- Text-adversarial robustness of the guard is unaffected by this analysis and remains untouched. Known pristine-guard weaknesses (benign-flood recency attacks) are properties of the base guard and should not be attributed to the edit.
- Not for production moderation use. This is a research artifact for studying the internal representation of guard verdicts at scale.
Method & tooling
Built with Fenrir (formerly Absolver) — the abliteration toolkit that finds the fetters, bites, and documents exactly what gave. Reference implementation for direction recovery (decision-step contrast at the verdict token), transactional excision, steer-test-first causality gating, E03-style held-out gates, and the margin-readout instruments that keep the gate honest. The guard-lane protocol (pairs-not-prompts, AUC ranking gates, read-scoped claims) is fully documented in the repo.
The gods bound the wolf — this time the fetter was a verdict, and the wolf ate it. 🐺
License
Apache-2.0, inherited from Qwen/Qwen3Guard-Gen-4B.