base_model: Qwen/Qwen3.8-Flash-Next base_model_relation: quantized pipeline_tag: image-text-to-text library_name: gguf license: apache-2.0 tags:

• gguf
• gsq
• rco
• quantization
• mixed-precision
• ist-daslab
• moe
• multimodal
• vision

Huihui-v2-Qwen3.8-Flash-Next-GSQ-RCO-Strata-abliterated-direction-GGUF

A single-file GGUF control vector containing the refusal direction that huihui-ai's abliterated releases of ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF remove from the model weights. Instead of committing to baked-in abliterated weights, you can load this direction at runtime — on any GGUF quant of the base model — and dial the strength yourself (including off).

What this is

huihui-ai publishes abliterated variants of Qwen3.8-Flash-Next in two strengths:

• v1 — huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated (bf16 safetensors + GGUF)
• v2 — huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF (UD-quantized GGUF only)

This repository extracts the direction those releases operate on, as a standalone control vector:

• the direction is extracted at full precision from the bf16 v1 checkpoint (v2 ships only in quantized form, so bf16 is the highest-precision source);
• it is then validated, in situ, against the published v2 GGUF tensors (see the precision report below) — v2 uses the same direction at ≈ 1.78× v1's strength.

How it works

1. Background. Refusal behavior in LLMs is largely mediated by a single direction in the residual stream (Arditi et al., 2024). "Abliterating" a model means removing that direction from its weights.

2. What the abliterated releases actually change. Diffing huihui's releases against Qwen/Qwen3.8-Flash-Next shows that exactly 96 "writer" matrices are edited (all other tensors — including the MTP head — are byte-identical to the base):

   role	tensors	shape
   full-attention o_proj	12	2560 × 6144
   linear-attention (GDN) out_proj	36	2560 × 6144
   shared-expert down_proj	48	2560 × 640

   These are the matrices that write into the residual stream.

3. The edits are rank-one and share one direction. For each edited tensor, ΔW = Wabliterated − Wbase concentrates 99.83 % (median) of its energy in the top singular component, i.e. ΔW ≈ −s·r·vᵀ. The leading vector r (2560-dim, output/residual-stream space) is the same across all 96 writers — that vector is the refusal direction, and it is what this file contains (unit-normalized).

4. Strengths, as measured in the published weights. With s defined as the removal factor of the r-component in each writer (s = 1 → component fully removed; s > 1 → component inverted):

   release	s (96 writers, mean ± std)	r-component after edit
   v1	0.99969 ± 0.00022	+0.0003 (≈ 0, removed)
   v2	1.77989 ± 0.00023	−0.78 (inverted)

   v2 therefore applies the same direction at 1.780× v1's strength, flipping the component to −0.78 of its original value.

Usage

Run it with Strata

Strata's own page has the full requirements. For example,

git clone https://github.com/Niko1221/Strata
cd Strata
./setup.sh --setup --model IQ3_S \
  --experimental-speed-projection /path/to/Huihui-Qwen3.8-Flash-Next-refusal-direction-r-v2.gguf

On Windows use START-HERE.bat with the same flags.

Setup writes these engine flags into strata-<model>.json:

--control-vector-scaled /path/to/Huihui-Qwen3.8-Flash-Next-refusal-direction-r-v2.gguf:1.0
--control-vector-layer-range 4 44 --cvec-mode project --cvec-dir per-layer

Run it with llama.cpp

llama.cpp and compatible runtimes that load GGUF control vectors:

llama-cli -m <base Qwen3.8-Flash-Next GGUF> \
  --control-vector Huihui-Qwen3.8-Flash-Next-refusal-direction-r-v2.gguf \
  --control-vector-scale -1.0

(older builds use --control-vector-file <fname>; --control-vector-layer-start/-end can restrict the applied layer range.)

Practical notes:

• Apply it to the base model (e.g. from ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF or unsloth/Qwen3.8-Flash-Next-GGUF), not to an already-abliterated release — otherwise the effects stack.
• Sign. The vector points along the refusal direction, so a negative scale suppresses it (mimicking the abliterated releases). If you observe refusal getting stronger, flip the sign — runtime conventions differ.
• Scale. -1.0 ≈ v1-like removal, -1.78 ≈ v2-like inversion — these mirror the measured weight-space factors. Runtime control-vector scales do not map 1:1 onto weight-edit factors, so treat them as starting points and tune on your own prompts.
• Omitting the flag (or scale 0) leaves the model untouched.

Precision report

All numbers are weight-space measurements computed directly from the published tensors.

Extraction quality (96 writer tensors, bf16 v1 vs bf16 base):

Cross-check vs an independent extraction (third-party control-vector GGUF of the same direction, sha256 59f9ae8b4372107a…):

In-situ validation vs the published v2 (UD-Q4_K_XL GGUF, all 96 writers):

The deviation between this file's direction and what v2 actually encodes (3.3 × 10⁻⁶) is ≈ 2.5× below the measurement pipeline's own noise floor (8.1 × 10⁻⁶): within measurement resolution, v2 uses the direction in this file, at 1.78× v1's strength.

Scope of these claims. The numbers above characterize weights. No behavioral benchmarks (refusal rates, capability evals) are included in this repository.

Provenance & credits

• Base model: Qwen/Qwen3.8-Flash-Next
• GSQ RCO GGUF: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF
• Extraction source (bf16): huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated
• Validation target (v2): huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF
• Pristine GGUF baseline: unsloth/Qwen3.8-Flash-Next-GGUF
• Method background: Arditi et al., Refusal in Language Models Is Mediated by a Single Direction, NeurIPS 2024.

License

This file is a direction vector derived from the releases above. Please comply with the upstream licenses (Qwen base model, huihui-ai releases) when redistributing or using it, and pick this repository's license accordingly.