license: other license_name: qwen-community-1.0 license_link: LICENSE base_model: local-inference-lab/Qwen3.8-Flash-Next-NVFP4 base_model_relation: finetune pipeline_tag: image-text-to-text tags:

• nvfp4
• mxfp8
• modelopt
• lossless-compression
• csf
• abliterated
• uncensored

Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD-Abliterated

Reduced-refusal model. A refusal-direction edit was applied on purpose. Read the Safety section before you deploy this.

An NVFP4-CSF container of an abliterated Qwen3.8-Flash-Next. The routed-expert block-scale planes are stored in the compressed CSF form. Every weight nibble and every other tensor is unchanged from the build described below, and all tensor names, dtypes and shapes are preserved.

main is v2. This revision replaces the rank-1 edit that main previously held, which remains available as tag v1. The two are compared below.

Why v2 exists

A refusal direction is regime-specific. Rendering a prompt with thinking on or off puts the refusal in a different direction in the residual stream, and on this model those two directions are nearly orthogonal. Measured here on four captures, two per regime, same corpus and seed, one runtime:

So ablating one regime's direction removes only about 4 % of the other's. That is why v1 reached 0 % refusal with thinking off and still refused with thinking on. v2 removes the span of both directions — a rank-2 edit.

Provenance

The container format is NVIDIA-ModelOpt mixed precision with CSF scale encoding, designed and published by Local Inference Lab, Inc. We encoded these planes ourselves from our own weights. No compressed bytes were copied from any Local Inference Lab repository, and this repository is not a mirror of one.

What the abliteration did

Applied in place on the quantised grid — no dequantise, no re-quantise. A rank-2 projection removing the span of two refusal directions, each captured at layer 42 and biprojected against its own harmless mean, applied with alpha = 1.0. The two directions sit at cos = 0.3495, 69.5° apart after biprojection. 25,188 tensors were edited:

No scale tensor was edited. No router, no gate_proj or up_proj, no q/k/v_proj, no vision tensor, no lm_head, no PLE table entry. Identical target set to v1 — only the direction and the rank differ.

The second constraint, and what it costs

A value that already sits exactly on the quantisation grid loses the edit under plain round-to-nearest, so a constrained requantiser picks codes that both approximate the edited weight and hold the constraint within a ±1 ULP budget. At rank 1 that constraint is v·W = 0 per column and one row can satisfy it exactly. At rank 2 the error is a 2-vector per column while each row still has one scalar degree of freedom, so no single row can zero it; the constraint is paid down across many rows instead.

The price of the second constraint is paid in weight error, not in subspace removal: the worst quantised tensor still gives up only 4.6e-05 of the ablated subspace. The 4.9e-05 figure is a floor of the FP4 grid rather than a solver limit — more passes converge, a 2× ULP budget does not improve it, and five different row orderings span 4.87–5.00e-05.

Measurements

Measured on the uncompressed build, whose weights are bit-identical to the ones here. Both arms ran in one session on one box at an identical gpu-memory-utilization, which is the only way these scores are comparable. Raw result files are in measurements/.

Capability

Zero wrong answers, zero truncated and zero stalled runs on every lavd and estonia run on both arms. GPQA is paired on the same 198 items: Δ +1.01 pp, exact McNemar p = 0.8506, not significant, with 13 items correct only on the parent and 15 only here. MTP acceptance over 312 and 337 samples, IQR 55.8–66.3 % against 53.2–64.7 % — the MTP write projections are themselves edited, so this is a correctness check rather than a throughput claim.

Refusal — the regime no longer matters

A refusal number is a protocol and scorer pair. On v1 the two regimes differed by a factor of ~19 on the same weights. On v2 they agree.

Held-out rows, 136 harmful and 59 over-refusal the layer selection never saw:

Over the full 200-row harmful set: 0/200 in protocol A on both scorers, and 0/200 in protocol B under model-forge.

Read the strict protocol-B row with care

The strict scorer counts an empty completion as a refusal. That is a defensible convention and it is applied identically to both arms, but in protocol B it mixes two different events. Of this model's 40 strict refusals over the full 200 rows, 2 are marker-matched refusals and 38 are empty completions; the parent's 174 is 171 marker-matched and 3 empty.

The asymmetry has a cause and it is not refusal behaviour. This model complies, a complying answer is long, and it hit the 2048-token cap 79 times against the parent's 6. A single strict rate therefore penalises the arm that complies, in exactly the regime the edit exists to fix. measurements/strict_split_protocolB.txt reports the two components separately for both arms.

Counter-intuitively thinking-off is the harder regime: the unedited parent refuses 100 % there against 84.6 % with thinking on.

v1 against v2, like for like

Full 200 harmful rows, same datasets, same scorers:

v1 and v2 were measured in different sessions, so this table is weaker evidence than the parent-versus-v2 table above. The parent arms of the two sessions agree closely — 200/200 and 200/200 in protocol A, 179/200 against 174/200 strict in protocol B — which is what makes the comparison usable at all.

The CSF container

Only the main-decoder routed-expert block scales are compressed: 73,728 planes, which is 48 layers × 512 experts × 3 projections. Each logical weight_scale becomes two ordinary safetensors tensors.

MTP experts, attention, shared experts, the PLE n-gram table, every global scale and every activation scale keep a plain weight_scale. config.json declares weight_scale_encoding: "csf" on the 48 routed-expert recipes and on nothing else.

Verification

Built and checked with remote/csf_codec.py, remote/build_csf_container.py and remote/verify_csf_container.py in our pipeline repository. All checks are exhaustive rather than sampled:

1. All 73,728 planes decode back to the source bytes exactly, read from the written shards rather than from build-time state.
2. All 227,865 other tensors are byte-identical to the uncompressed source build, dtype and shape unchanged, including the trained FP32 normalisation parameters.
3. The index maps every tensor, and weight_scale_encoding appears on routed-expert recipes only.

The builder additionally decodes every plane and compares it with the source before writing the shard, so a mismatch aborts the build rather than shipping.

Serving

Needs a vLLM build with the NVFP4-CSF reader and the b12x SM120 kernels. main is v2, so no revision pin is needed.

VLLM_PLE_TABLE_MEMORY=disk vllm serve lflare/Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD-Abliterated \
  --quantization modelopt_mixed --load-format safetensors \
  --tensor-parallel-size 1 --pipeline-parallel-size 1 --dtype bfloat16 \
  --moe-backend b12x --linear-backend b12x \
  --mm-encoder-tp-mode data --mamba-cache-mode align \
  --enable-chunked-prefill --block-size 16

--quantization modelopt_mixed is auto-detected from config.json and may be omitted. TP1, TP2 and TP4 all satisfy the scale-alignment requirement. Pipeline parallelism must be 1. Expert parallelism, data parallelism and ubatching are not supported on the CSF path.

A runtime must support CSF, and most do not. The scale planes are not plain FP8, so a loader without a CSF path cannot read this repository at all. The karmic-kraken-20261001 and 20260925 images have no CSF support; karmic-kraken-20261016.2 does. Proof the CSF path actually ran rather than a fallback: the autotuner issues quantization.csf_block_expansion requests at startup.

Known dependency gap. The CSF scale loader imports b12x.loader._csf_blocks and b12x.loader._csf_source, which are in no released package, so this path needs a source build of local-inference-lab/flashinfer or one of their images. If that is impractical, decode the planes back to plain weight_scale tensors with our remote/csf_codec.py and drop weight_scale_encoding from config.json — that yields a checkpoint which runs on any ModelOpt NVFP4 build, at a cost of 3.58 GB.

These files cannot be loaded by Flare. Its runtime accepts only its own prepared package format; see lflare/Qwen3.8-Flash-Next-FLARE-Abliterated for a Flare package carrying these weights.

Served and verified on GPU, 2026-10-19

This container has been loaded and served. An earlier revision of this card said it had not, which was true when written.

That second row is the one v1 could not produce: v1 reaches 0 with thinking off and still refuses with thinking on. The probe ran through the CSF container on a different runtime from the one that produced the gate tables above, which makes it an independent confirmation of the dual-regime result rather than a repeat.

The capability and held-out refusal numbers above still come from the uncompressed build, served for ~1.5 hours across two concurrent arms. The CSF encoding is lossless and verified byte-for-byte against that build, so those numbers transfer; what the probe adds is proof that the CSF reader path serves them, which byte-verification against our own source could never show.

Differences from Local Inference Lab's CSF-QAD container

Do not treat the two as interchangeable.

• The vision tower here is BF16. Their main revision additionally converted 110 vision linears to MXFP8 and NVFP4. This container inherits the unconverted vision weights of the 6909a5b parent, because our abliteration never touched vision and we did not reproduce their conversion.
• 40 shards, keeping the parent's shard names, where their main revision reshards to 21.
• No manifest.json, build-contract.json, receipts/ or verification.json.

Not verified

• No vision evaluation was run on any of our builds.
• The on-container refusal probe is n=20 per regime, which confirms the behaviour is live rather than measuring its rate. The rates come from the 200-row and held-out-136 gates on the uncompressed build.

Safety

This model has had refusal behaviour deliberately reduced, and v2 reduces it in both thinking regimes where v1 only did so with thinking off. It will answer requests that the parent declines. Do not deploy it as a general-purpose assistant without your own safeguards, and do not treat the refusal numbers above as a safety property.

Report refusal numbers as a protocol and scorer pair, and compare them only against the parent measured under the identical protocol and scorer.

Licence and credits

Qwen Community License 1.0, inherited from the parent. See LICENSE.

Condition 1 of that licence requires the notice to travel with every copy. If you operate a Model-as-a-Service or AI-assistant business, condition 2 requires a separate licence from Qwen before any commercial use.

• Qwen/Qwen3.8-Flash-Next — base model, by Qwen.
• local-inference-lab/Qwen3.8-Flash-Next-NVFP4 — the QAD NVFP4 and MXFP8 quantisation this builds on, by Local Inference Lab, Inc.
• The NVFP4-CSF container format and the b12x kernels — by Local Inference Lab, Inc. Neither they nor Qwen endorse this model.