license: other license_name: qwen-community-1.0 license_link: LICENSE base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored base_model_relation: quantized pipeline_tag: image-text-to-text tags:

• nvfp4
• fp8
• modelopt
• sglang
• blackwell
• uncensored
• abliterated

Qwen3.8-Flash-Next-Uncensored-NVFP4 (beginning-ai)

A mixed-precision quantization of orcarouter/Qwen3.8-Flash-Next-Uncensored (revision e096800036ec20da7e2442dcd4044a004d4e99fa), made to run on one 96 GB Blackwell GPU with SGLang. That model is an abliterated (refusal-removed) BF16 build of Qwen/Qwen3.8-Flash-Next.

This checkpoint has the same structure, formats and file layout as our censored quant, beginning-ai/Qwen3.8-Flash-Next-NVFP4. The two can be swapped by changing only the model path. Its activation scales were measured again on this model.

• 120 GiB on disk. The BF16 source is 335 GiB.
• About 78 GB of weights on the GPU. The 47.7 GiB n-gram PLE table stays in host RAM or on NVMe.
• On one RTX PRO 6000, single-stream decode runs at about 265 tok/s at 2K context and about 240 tok/s at 200K, with a 786K-token KV cache and 524K context.
• Nothing is removed: the vision tower and the multi-token-prediction (MTP) draft layer are kept.

Warning. The source model has had its refusal behavior removed. It will follow harmful, unethical or illegal requests that the original model refuses, and it has no built-in guardrails. Use it only where you add your own safeguards, and follow the license and the laws that apply to you. We did not change or measure its refusal behavior; see the source model card.

How this differs from the source and the base

From Qwen's base to orcarouter's source. We compared every tensor with the official base. orcarouter changed 149 of the 1,658 tensors, all of them weights that write into the residual stream:

• the output projections of all 36 linear-attention layers and 12 sparse-attention layers;
• the routed-expert and shared-expert down projections in all 48 layers;
• the token embeddings and one PLE projection;
• 3 tensors in the MTP layer.

The router, the norms, the vision tower, the output head and the 47.7 GiB n-gram table are byte-identical to the base.

From orcarouter's source to this checkpoint:

The full per-layer list is in hf_quant_config.json (quant_algo: MIXED_PRECISION). Byte counts and the calibration report are in MANIFEST.json.

How it was made

• Quantizer. NVIDIA ModelOpt 0.47.0 tensor quantizers (NVFP4QTensor, FP8QTensor), applied to orcarouter's BF16 weights.
• Shared scale for gate and up. SGLang runs each expert's gate_proj and up_proj as one fused GEMM with one global scale. So both halves are quantized against one shared weight_scale_2, as ModelOpt does for fused layers. A build check confirms that all 24,624 fused pairs, plus the 512 MTP experts, have equal scales.
• Activation scales, measured on this model. We served a first build of this model in SGLang with a forward hook on every routed-MoE layer. The hook recorded the MoE inputs and routing for 16 real coding-agent prompts (332,244 tokens) and saved 100,123 rows per layer. From that capture:
  • gate_proj/up_proj: one scale per layer, from the largest MoE input over every token.
  • down_proj: one scale per expert, from that expert's own activations (silu(gate) * up), computed with the BF16 weights on the tokens routed to it. 24,184 of the 24,576 experts had at least 32 routed tokens; the median is about 1,350. The rest take the larger of their own maximum and the layer's 95th percentile.
  • MTP experts: one scale per layer, from the same capture.
  • No scales were taken from another release.
• No quantization of the output head. It stays BF16, as in NVIDIA's and RadixArk's releases of the base. A 4-bit head changed the top token at about 20% of positions in our tests.

Results

Test machine

Correctness checks

Coding agent: DeepSWE smoke test

DeepSWE v1.1 with the mini-swe-agent harness, reasoning effort medium, 524K context. We ran 4 tasks that our censored quant solves, one attempt each, to check that the model works as an agent. This is a smoke test, not a benchmark score.

Speed

All numbers are for one stream on the test machine above. The prompts are real coding-agent prompts. Each run generates 1,024 tokens with temperature 1.0, top_p 0.95 and top_k 20.

Decode is the median of 4 seeds at 2K and 64K. At 200K it is the total over 4 prompts × 4 seeds, with a warm page cache.

Serving

The results above were measured with this exact SGLang build:

• Commit 93540334d2 (tag qwen38-fn-nvfp4-2026-10-10 in beginning-ai/sglang).
• It is upstream SGLang f3fe753475 plus 9 commits (compare). The censored model card lists what each commit adds.

pip install -e "git+https://github.com/beginning-ai/sglang@93540334d2ce687f0677bebb42dbd252b218e92f#egg=sglang&subdirectory=python"

We have not tested this checkpoint on unpatched SGLang.

python3 -m sglang.launch_server \
  --model-path beginning-ai/Qwen3.8-Flash-Next-Uncensored-NVFP4 \
  --trust-remote-code \
  --quantization modelopt_fp4 \
  --dtype bfloat16 \
  --kv-cache-dtype fp8_e4m3 \
  --page-size 64 \
  --context-length 262144 \
  --linear-attn-decode-backend flashinfer \
  --linear-attn-prefill-backend flashinfer \
  --mamba-radix-cache-strategy extra_buffer \
  --speculative-algorithm NEXTN \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Notes:

• PLE table on NVMe. If the 47.7 GiB PLE table does not fit in host RAM, as on our 64 GB machine, add --ple-offload-backend nvme (patched SGLang).
• 524K context. Add --context-length 524288 and set SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1. Also pass --json-model-override-args with a YaRN rope_parameters block: rope_type: "yarn", factor: 2.0, original_max_position_embeddings: 262144, and the base config's mrope_* / rope_theta / partial_rotary_factor values.
• Decode kernels. SGLANG_FP8_ROW_WEIGHTS=1 (FP8 output head and FP8 draft projections, converted at load time) and SGLANG_NVFP4_DECODE_MOE=1 turn on the patched decode kernels.

Files

License and credits

• License: the Qwen Community License 1.0, the same as the base model (see LICENSE).
• Source model: orcarouter/Qwen3.8-Flash-Next-Uncensored by OrcaRouter.
• Base model: Qwen/Qwen3.8-Flash-Next by the Qwen team. The PLE table is from Qwen/Qwen3.8-Flash-Next-FP8.
• Quantization: NVIDIA TensorRT Model Optimizer (ModelOpt).