license: apache-2.0 base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored library_name: gguf tags:

• gguf
• llama.cpp
• qwen4exp
• flash-next
• speculative-decoding
• mtp
• quantized
• iq4_nl
• abliterated

Qwen3.8-Flash-Next-Uncensored — IQ4_NL (MTP-split)

📖 Full testing & methodology documentation — how this quant was built, every test, the testing methods, the measurement confound we caught and retracted, and the iterations we went through. Charts included.

Quality-biased IQ4_NL GGUF of orcarouter/Qwen3.8-Flash-Next-Uncensored (Apache-2.0; abliterated Flash-Next), with the 1-layer NextN/MTP head split out as a separate draft sidecar for speculative decoding. Built for long-session agentic and creative workloads — heavy reasoning, character/story, planning, games, architecture/CAD, 3D-spatial, science — and accepted only after a balanced evaluation with a replicated-loss criterion (zero replicated quality losses vs both the base model and the source Q8_0 reference).

No custom kernel required. The model runs in standard llama.cpp — qwen4exp architecture support is upstream, so any recent stock build loads the backbone. (Unlike our earlier Qwen3.6 RYS GGUFs, which required a custom loader.) The fork branch is optional: it adds MTP speculative decoding and performance paths on top.

Files

How it was prepared

1. Verified source: the pinned orcarouter release (an NVFP4 derivative was evaluated and rejected during provenance verification).
2. Backbone repack: 32 MTP-head tensors moved to the sidecar; 1,224 backbone tensors verified by name/shape/dtype with value sampling at start/middle/end (3,360 checks).
3. Imatrix calibration: multi-domain corpus weighted toward the target workloads (926 entries).
4. Mixed-precision IQ4_NL: 754 IQ4_NL, 46 Q5_K, 3 Q6_K, 32 Q8_0, 388 F32, 1 F16.

Quality

Symmetric balanced evaluation — identical prompts, samplers, and seeds across variants, 36 suite-seed runs, responses scored on a 10-point rubric judged blind; a loss only counts when it replicates across seeds:

Mean differences of a few tenths of a point sit within the suite's margin of error — read the table as parity, not a ranking. The claim this evaluation supports is no noticeable quality loss: zero replicated losses against both references, at 63% of the Q8_0 reference's size. If you need a stronger guarantee for your use case, the documentation page publishes the full methodology so you can run your own.

Usage

The backbone runs on upstream llama.cpp master (qwen4exp architecture is upstream). MTP speculative decoding needs the fork branch flashnext-mtp:

llama-server \
  -m qwen38_flash_next_uncensored_raw_iq4nl.gguf \
  --model-draft mtp-draft-uncensored-forknames.gguf \
  --spec-type draft-mtp,ngram-map-k \
  --mmproj mmproj-Qwen3.8-Flash-Next-Uncensored-F16.gguf \
  -c 262144 -fa on

Measured on a 7-GPU mixed rig (262k ctx, fp32 KV): prefill on fresh 25k-token prompts — MTP draft alone 424 tok/s, MTP + n-gram 437 tok/s (+3%); decode as 600-token continuous generations — 58.8 tok/s (+8%). A 1-deep draft measured best (deeper chains from the 1-layer head decay acceptance monotonically; depth sweep ran on a fixed ≈40-token chat prompt, context under ~700 tokens — short-context decode, expect lower tok/s as context fills).

Tensor-split prefill (2× faster prompt reading). The qwen38-tsplit-dev research branch adds per-tensor multi-GPU splitting for this architecture — correct as of 2026-09-09 after root-causing and fixing a GQA head-correspondence bug (each GPU was pairing queries with the wrong KV heads; fixed by per-backend K/V origin aliases, GGML_META_GQA_FIX=1). 25k-token prefill ~875 tok/s = 2.0× the layer-split baseline on the same 4-GPU pool, with exact-recall needles and quality-battery parity. Context ceiling 32768 on that pool (FA-cache VRAM). Full story and usage: TENSOR_SPLIT.md.

Hardware caveat: our testbed's PCIe links run at Gen3 x4, which makes multi-GPU layer-split decode interconnect-bound. On Gen4/Gen5 systems (or fewer/faster cards) expect better speeds — treat our numbers as a conservative floor.

Acknowledgments

• Alibaba's Qwen team — the Flash-Next architecture and base model.
• orcarouter — the uncensored release this quantization is derived from, and the direct source of the verified weights.
• ggml-org/llama.cpp — the runtime, including upstream qwen4exp support.
• flash-linear-attention (GDN l2norm reference) and unsloth (draft-head GGUF layout).

License

Apache-2.0, inherited from the base model.