license: other license_name: qwen-community-license-1.0 library_name: basalt pipeline_tag: text-generation tags:

• moe
• qwen
• basalt
• quantized
• uncensored
• speculative-decoding

Qwen3.8-FN-Uncensored-Q4_K_M — Basalt pack

A ready-to-run .basalt pack for the Basalt inference engine (a fork of Strata, MIT). This is an uncensored Q4_K_M build of Qwen3.8-Flash-Next (125B-class mixture-of-experts, qwen4exp architecture), converted from GGUF and rebuilt with tuned speculative-decoding assets.

File

What is inside

• Experts: Q4_K_M, 48 layers, 512 routed experts per layer (24,576 expert slots in the routing profile).
• Output head: Q6_K (output.weight, 2,560 × 248,320).
• MTP / speculative decoding: mtp/dense marked tuning = self_distilled, mtp/draft_vocab = true, draft expert_format = q2_0. The draft head and draft vocabulary are the tuned ones, not the published defaults.
• Vision: vision/mmproj (907,543,296 bytes) plus vision/text_vocab, so image input works when the engine runs with --vision.
• Tokenizer: a full Qwen tokenizer directory and chat template are embedded in the pack; no external files are needed.
• Sections: tokenizer/*, dense, dense.index, experts, experts.layout, mtp/{dense,index,experts,draft_vocab}, profile, vision/*, gguf, meta.

Provenance

Converted with Basalt's tools/convert/convert.py (git 2ef470c) from:

Each source's fingerprint (sha256 of the first and last 64 MiB plus the file size) is recorded verbatim in the pack's meta section. The pack also carries the Qwen Community License 1.0 text and the model's own sampling defaults: temperature 1.0, top_p 0.95, top_k 20.

Running it

Requirements: an NVIDIA Blackwell GPU (compute capability 12.0) and CUDA 13, plus enough system RAM for the experts that do not fit in VRAM. Upstream targets Linux; a native Windows port exists in this fork's sibling work. Use an engine build that has the Q4_K MMQ tiles (upstream 0e1f926 or later).

Flags this pack was measured with:

--max-context 524288 --kv int8 \
--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 262144 \
--spec 7 --spec-min-p 0.5 \
--expert-cache auto --prefill auto --resident-experts --vram-reserve-mib 700 \
--ple-io ram --kv-resident 32768 --vision dynamic

--mtp-max-t should stay unset: a nonzero value disables the suffix drafter.

Development A/B on one RTX 5090 (575 W power limit): ~220 tok/s decode at a 512k context window, ~5,900 tok/s prefill. Numbers are labelled with their power limit and are development measurements, not headline figures.

License and attribution

The pack records the Qwen Community License 1.0 as its license; the license text is embedded in the .basalt file itself. The engine is Basalt, a fork of Strata (MIT). This repository is an independent conversion of third-party uncensored weights — check the upstream model's terms before redistributing or deploying it commercially.

This repository is currently private and under test.