license: apache-2.0 base_model:

• google/gemma-4-26B-A4B-it-qat-q4_0-unquantized
• google/gemma-4-26B-A4B-it library_name: transformers pipeline_tag: image-text-to-text tags:
• gemma
• gemma-4
• moe
• qat
• heretic
• uncensored
• decensored
• abliterated
• quantization-ready

gemma-4-26B-A4B-it QAT (unquantized) — uncensored heretic

Uncensored (abliterated) variant of google/gemma-4-26B-A4B-it-qat-q4_0-unquantized, the QAT-trained bf16 checkpoint intended as the precursor for q4_0 / W4A4 / W4A16 quantization.

Produced with Heretic (directional ablation + Optuna kernel optimization). This is the full merged unquantized bf16 model — quantize from this to your favorite 4-bit format; the abliteration deltas are already baked into the weights, so downstream calibration works exactly as on any QAT checkpoint.

Method & provenance

• Abliteration kernel (direction index + per-component weight kernels over 30 layers) was optimized on the stock google/gemma-4-26B-A4B-it base: 200-trial Optuna study, Pareto-optimal trial selected to minimize KL divergence at maximum refusal suppression.
• The kernel was then transferred to the QAT-unquantized base, with residual directions recomputed on that base (mean per-layer direction cosine vs stock: 0.973, ~0.976 in the ablated layer band — same refusal circuit, so the transfer is faithful).
• Ablation applied to attention out-projections (LoRA-merged), dense MLP down-projections (LoRA-merged), and all 128 fused MoE expert down-projections per layer (ablation baked into the 3D fused weights — see reproduce/reproduce.json for exact parameters).

Evaluation

Reference points on the stock (non-QAT) base with the same kernel: ARC-C 96.67% (stock unmodified: 96.76%), HellaSwag chat-MCQ 87.0% (stock: 87.9%), IFEval prompt-strict 88.5% / loose 90.8%, KL 0.090, refusal keywords 18/100. The ~3 pt ARC gap between QAT and stock bases is attributable to QAT training itself, not the abliteration (93.86 → 94.11 across ablation on the QAT base).

KL divergence 0.078 is well below the ~0.5 level generally associated with noticeable capability damage.

Intended use

A local creative-writing and analysis assistant for adult romance / adult-entertainment authorship, and general-purpose LAN workhorse duty. The residual keyword rate above is dominated by worst-case malicious-instruction prompts, not adult content, where compliance is effectively complete.

Released under Apache 2.0 (inherited from the base model). Provided as-is, no warranty; you are responsible for how you use it.

Reproducing

See reproduce/ for the exact Heretic parameters (JSON), dependency snapshot, and SHA256SUMS of the weight shards. The abliteration was run with a patched Heretic (Gemma-4 fused-expert support + low-RAM sequential loading); parameter semantics are unchanged.