license: other license_name: qwen-community-1.0 license_link: https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/main/LICENSE base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored base_model_relation: quantized tags:
- exl3
- exllamav3
- qwen3.8
- flash-next
- abliterated
- uncensored
- mtp
- moe
- quantized language:
- en
- fr
- zh pipeline_tag: image-text-to-text
EXL3 quantization of orcarouter/Qwen3.8-Flash-Next-Uncensored, an abliterated (refusal-removed) build of Qwen/Qwen3.8-Flash-Next. Built with ExLlamaV3 1.4.6.
| Size on disk | 102 GB |
| Language model | 4.05 bpw |
lm_head | 6 bpw |
| MTP layers | 8 bpw |
| n-gram (PLE) tables | 6 bpw |
| Vision tower | unquantized (16-bit) |
| VRAM when loaded | ~66 GB (FP16 cache) / ~71 GB (8-bit cache, 1.4.8) |
Config
model:
model_name: Qwen3.8-Flash-Next-Uncensored-exl3-4bpw
cache_size: 16384
ngram_ram: true
tensor_parallel: false # required
draft_model:
draft_mode: mtp
tensor_parallel must be false. Otherwise:
NotImplementedError: Tensor-parallel is not currently implemented for Qwen4ExpForConditionalGeneration. Autosplit works. Confirmed by turboderp
as not supported yet (2026-09-06).
Cache quantization needs ExLlamaV3 ≥ 1.4.8. Below that, cache_mode
other than FP16 is rejected with QSA attention currently supports only the fp16 cache layer. On 1.4.8 "8,8" loads fine — but it buys nothing here:
the KV cache is ~2 kB per token per attention layer, so a few hundred MB at
16k context. VRAM went up (66 → 71 GB) with the new expandable-segment
allocator, and throughput moved <5%. Use FP16 unless you have a reason not
to.
ngram_ram: true loads the n-gram table into system RAM instead of
streaming it from disk per forward.
MTP is preserved; TabbyAPI logs Using main model MTP component for drafting on load.
Conversion
python convert.py \
-i Qwen3.8-Flash-Next-Uncensored \
-o Qwen3.8-Flash-Next-Uncensored-exl3-4bpw \
-w /tmp/exl3-work \
-b 4.05 -hb 6 -ngb 6 -mb 8 -vb 16 \
-d 0,1,2,3 -v
Default calibration corpus, 250 rows x 2048 columns.
Measurements
4x RTX 4000 Ada (20 GiB, sm89), PCIe, no NVLink, 2x Xeon Gold 5318Y.
ExLlamaV3 1.4.6 + TabbyAPI, autosplit, MTP on, FP16 cache. AIPerf,
streaming, ignore_eos, seed 42, with warmup.
ISL 1000 / OSL 500 / concurrency 4 / 20 requests:
| Metric | value |
|---|---|
| Request latency | 25,508 ms |
| Inter-token latency | 46.6 ms |
| Time to first token | 2,484 ms |
| Output throughput | 76 tok/s |
Same run on 1.4.8 with an 8-bit cache: 26,129 ms / 47.1 ms / 2,620 ms / 73 tok/s.
ISL 8000 / OSL 200 / concurrency 2 / 12 requests:
| Metric | value |
|---|---|
| Time to first token | 12,724 ms |
| Active prefill | 760 tok/s |
| Inter-token latency | 37.3 ms |
| Request latency | 19,957 ms |
Single stream via chat.py: 34.5 tok/s with MTP (70% acceptance on code),
20.8 tok/s without.
Same box and tooling, my EXL3 6.0bpw of the dense Qwen3.8-27B-Uncensored gives 20,816 ms / 35.0 ms ITL / 91 tok/s on the first run.
20 requests per run, so gaps under ~5% are noise.
Quality
Not benchmarked at 4.05 bpw. Abliteration figures are orcarouter's, measured on the BF16 source, not on this quant.
Safety
Safety alignment has been substantially removed via abliteration. It will comply with requests the original Qwen3.8-Flash-Next refuses, and has no meaningful built-in guardrails. Add your own moderation layer before any deployment.
License
Qwen Community License 1.0, inherited from Qwen/Qwen3.8-Flash-Next.