license: other license_name: qwen-community-1.0 license_link: LICENSE base_model: local-inference-lab/Qwen3.8-Flash-Next-NVFP4 base_model_relation: finetune pipeline_tag: image-text-to-text tags:

• nvfp4
• mxfp8
• modelopt
• lossless-compression
• csf
• abliterated
• uncensored

Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD-Abliterated

Reduced-refusal model. A refusal-direction edit was applied on purpose. Read the Safety section before you deploy this.

An NVFP4-CSF container of an abliterated Qwen3.8-Flash-Next. The routed-expert block-scale planes are stored in the compressed CSF form. Every weight nibble and every other tensor is unchanged from the build described below, and all tensor names, dtypes and shapes are preserved.

This repository replaces an earlier release. The same weights were first published as huginnfork/Qwen3.8-Flash-Next-NVFP4-Abliterated tag v4, in the plain uncompressed layout: 86 files, 40 safetensors shards, 106.41 GB. That tag has been removed, and this repository is its sole successor. The weights are bit-identical; only the scale encoding differs, which makes this container 3.58 GB smaller and readable by the NVFP4-CSF path. Every number reported below was measured on that build, and the raw result files travel with this repository under measurements/.

Provenance

The container format is NVIDIA-ModelOpt mixed precision with CSF scale encoding, designed and published by Local Inference Lab, Inc. We encoded these planes ourselves from our own weights. No compressed bytes were copied from any Local Inference Lab repository, and this repository is not a mirror of one.

What the abliteration did

Applied in place on the quantised grid — no dequantise, no re-quantise. A rank-1 refusal direction, biprojected against the harmless mean, captured at layer 46 and applied with alpha = 1.0. 25,188 tensors were edited:

No scale tensor was edited. No router, no gate_proj or up_proj, no q/k/v_proj, no vision tensor, no lm_head, no PLE table entry.

Editing a value that already sits exactly on the quantisation grid attenuates the edit under plain round-to-nearest, so a constrained requantiser chooses codes that both approximate the edited weight and hold v·W = 0 per column within a ±1 ULP budget. Measured residual leakage is about 3.5e-06 on NVFP4 modules and 4.3e-07 on MXFP8 modules.

Measurements

Measured on the predecessor build, whose weights are bit-identical to the ones here. Nothing was re-measured on this container. Raw result files are in measurements/.

Capability

On lavd neither arm produced a wrong answer — every miss on both sides is a 40k-token cap hit. The GPQA difference is not significant (p = 0.286).

Refusal, held-out rows, and the regime matters enormously

A refusal number is a protocol and scorer pair, and the protocol dominates. The same weights move by a factor of about 19 between the two regimes below. Do not quote one cell on its own.

Counter-intuitively thinking-off is the harder regime: the unedited parent refuses 100 % there against 89.7 % with thinking on. Protocol A is the model-forge gate, which renders its prompts with enable_thinking=False and scores them the same way. The direction used here was captured in that same regime, which is why the two are self-consistent.

Over the full 200-row harmful set, protocol A gives 0/200 under the model-forge scorer and 6/200 under our stricter 47-marker scorer. Swapping the scorer moves the number a few points; swapping the thinking regime moves it ~70.

The CSF container

Only the main-decoder routed-expert block scales are compressed: 73,728 planes, which is 48 layers × 512 experts × 3 projections. Each logical weight_scale becomes two ordinary safetensors tensors.

MTP experts, attention, shared experts, the PLE n-gram table, every global scale and every activation scale keep a plain weight_scale. config.json declares weight_scale_encoding: "csf" on the 48 routed-expert recipes and on nothing else.

Verification

Built and checked with remote/csf_codec.py, remote/build_csf_container.py and remote/verify_csf_container.py in our pipeline repository. Three checks, all exhaustive rather than sampled:

1. All 73,728 planes decode back to the source bytes exactly. Verified by reading the written shards, not from build-time state.
2. All 227,865 other tensors are byte-identical to v4, with dtype and shape unchanged. The 149,093 FP32 tensors — which include the trained normalisation parameters — are preserved verbatim.
3. The index maps every tensor, and weight_scale_encoding appears on routed-expert recipes only.

The encoder was additionally validated against a published CSF plane rather than only against our own decoder. Encoding v4's plain …layers.0.mlp.experts.0.down_proj.weight_scale reproduces the corresponding nvfp4_csf_fixed and nvfp4_csf_exceptions tensors of local-inference-lab/Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD byte for byte, including all 226 exception words. Our independently computed totals also match that container's own manifest exactly on compressed-plane count, bytes saved, container byte size and tensor count.

Serving

Needs a vLLM build with the NVFP4-CSF reader and the b12x SM120 kernels.

VLLM_PLE_TABLE_MEMORY=disk vllm serve <path to this checkpoint> \
  --quantization modelopt_mixed --load-format safetensors \
  --tensor-parallel-size 1 --pipeline-parallel-size 1 --dtype bfloat16 \
  --moe-backend b12x --linear-backend b12x \
  --mm-encoder-tp-mode data --mamba-cache-mode align \
  --enable-chunked-prefill --block-size 16

--quantization modelopt_mixed is auto-detected from config.json and may be omitted. TP1, TP2 and TP4 all satisfy the scale-alignment requirement. Pipeline parallelism must be 1. Expert parallelism, data parallelism and ubatching are not supported on the CSF path.

Known dependency gap. The CSF scale loader imports b12x.loader._csf_blocks and b12x.loader._csf_source. Those two modules are not in any released package, so this path currently needs a source build of local-inference-lab/flashinfer. If that is impractical, decode the planes back to plain weight_scale tensors with our remote/csf_codec.py and drop weight_scale_encoding from config.json — that yields a checkpoint which runs on any ModelOpt NVFP4 build, at a cost of 3.58 GB.

These files cannot be loaded by Flare. Its runtime accepts only its own prepared package format.

Differences from Local Inference Lab's CSF-QAD container

Do not treat the two as interchangeable.

• The vision tower here is BF16. Their main revision additionally converted 110 vision linears to MXFP8 and NVFP4. This container inherits the unconverted vision weights of the 6909a5b parent, because our abliteration never touched vision and we did not reproduce their conversion.
• 40 shards, keeping the parent's shard names, where their main revision reshards to 21.
• No manifest.json, build-contract.json, receipts/ or verification.json. Those are their build-provenance files and their verify and restore subcommands do not apply to this artefact.

Not verified

• GPU inference has not been run on this container. The weights are bit-identical to v4, which was served and evaluated, and the compression is verified lossless. But this exact container has not been loaded by a runtime.
• No vision evaluation was run on any of our builds.

Safety

This model has had refusal behaviour deliberately reduced. It will answer requests that the parent declines. Do not deploy it as a general-purpose assistant without your own safeguards, and do not treat the refusal numbers above as a safety property.

The refusal numbers above are reported for both thinking regimes on purpose, because they differ by roughly a factor of 19 on the same weights. Compare them only against the parent measured under the identical protocol and scorer.

Licence and credits

Qwen Community License 1.0, inherited from the parent. See LICENSE.

Condition 1 of that licence requires the notice to travel with every copy. If you operate a Model-as-a-Service or AI-assistant business, condition 2 requires a separate licence from Qwen before any commercial use.

• Qwen/Qwen3.8-Flash-Next — base model, by Qwen.
• local-inference-lab/Qwen3.8-Flash-Next-NVFP4 — the QAD NVFP4 and MXFP8 quantisation this builds on, by Local Inference Lab, Inc.
• The NVFP4-CSF container format and the b12x kernels — by Local Inference Lab, Inc. Neither they nor Qwen endorse this model.