license: apache-2.0 base_model:
- huihui-ai/Huihui-Qwen3.8-27B-abliterated base_model_relation: quantized tags:
- ninfer
- qwen3.8
- abliterated
- uncensored
- rtx-3090
- image-text-to-text
Huihui-Qwen3.8-27B-abliterated, NInfer v3 artifact
A NInfer v3 artifact of huihui-ai/Huihui-Qwen3.8-27B-abliterated, the abliterated Qwen3.8-27B with the lowest reported divergence from the base checkpoint among the widely used releases (the first fifteen layers are untouched; independent KL divergence measurements sit around 0.008) and a working MTP head.
Built for the RTX 3090 line of NInfer
(iamwavecut/ninfer-3090, branch franken/v0.11,
which is ashalliants' v0.11.0 plus that fork's production fixes).
What is inside
| component | representation |
|---|---|
| text layers (64) | official qwen3_8_27b recipe: Q4/Q5 grouped projections, Q8 vocabulary embedding and output head |
| vision tower | as the recipe assigns it (the checkpoint's own model.visual.* weights) |
| MTP head | the checkpoint's own mtp.* weights, Q8 |
| DFlash2 adapter | z-lab/Qwen3.8-27B-DFlash2, trained on the vanilla model; applied unchanged (the target verifies every draft, so it only affects acceptance) |
| proposal head | the indexed 131,072-row draft head for --lm-head-draft |
| chat template | NInfer's pinned qwen3_8.jinja |
Conversion command (from the fork's tree, CUDA device):
python3 -m tools.convert \
--model Huihui-Qwen3.8-27B-abliterated \
--recipe qwen3_8_27b \
--source dflash2=Qwen3.8-27B-DFlash2 \
--components text,vision,mtp,dflash2 \
--resource chat_template.jinja=tools/chat_templates/qwen3_8.jinja \
--proposal \
--name qwen3.8-27b-abliterated \
--out Huihui-Qwen3.8-27B-abliterated-ninfer-v3.ninfer
SHA256SUMS lists the checksum of every file.
Running
Any NInfer build that reads v3 artifacts (v0.11.0 or later). A single RTX 3090 serves the full 198,400-token window with vision and MTP:
ninfer-serve Huihui-Qwen3.8-27B-abliterated-ninfer-v3.ninfer --model-id qwen3.8-27b \
--max-context 198400 --kv-capacity 198400 --kv-dtype rk8v4 --gdn-state-fp16 \
--spec mtp --draft-tokens 3 --lm-head-draft \
--vision --vision-residency overlay --vision-max-merged 12288
The artifact also carries the DFlash2 adapter for --spec dflash2; only the MTP path was
measured below.
Measured on an RTX 3090
NInfer franken/v0.11 Release build (sm_86, CUDA 13.1) with the command line above plus
--max-concurrency 4 --host-kv-mib 12288 --host-state-slots 8, greedy decoding, thinking off. The
reference is WaveCut/Qwen3.8-27B-MXFP8-CRACK-NInfer
served by the previous NInfer line of the same fork with the same flags and the same client.
Memory with one active request: 16.7 GiB of weights and 5.54 GiB of runtime state at the
198,400-token rk8v4 window, which leaves 1.12 GiB free on a 24 GiB card.
| workload | decode tok/s | prefill tok/s | reference decode / prefill |
|---|---|---|---|
| short prompts | 131 | 126 | |
| 8k context | 111 | 1,433 | 108 / 1,222 |
| 32k context | 106 | 1,324 | 97 / 1,121 |
| 64k context | 98 | 1,116 | 86 / 965 |
| agent loop, 20k context with prefix reuse | 124 | 1,003 | 119 / 933 |
Decode speed is the median per request with MTP (three draft tokens, about 3.3 accepted per round on prose).
Quality over 1,179 items (GSM8K 200, MMLU-Pro 280, HumanEval 164, MGSM-ru 250, Global-MMLU-ru 285), the same items and client for every column:
| suite | this artifact | CRACK, same engine | CRACK, previous engine |
|---|---|---|---|
| GSM8K | 95.0 | 95.5 | 95.0 |
| MMLU-Pro | 80.0 | 79.6 | 78.6 |
| HumanEval | 94.5 | 92.1 | 92.1 |
| MGSM-ru | 91.2 | 92.0 | 91.2 |
| Global-MMLU-ru | 82.8 | 81.8 | 82.5 |
| all items | 87.6 | 87.2 | 86.9 |
Against the previous engine's CRACK run this artifact wins 46 items and loses 37 (exact McNemar p = 0.38); no suite differs significantly.
Perplexity on the quick corpus of ninfer-ppl-1m-v1 (261,167 scored tokens, 4,096-token window,
2,048-token stride, rk8v4 KV): 4.3750 for this artifact against 4.4685 for CRACK on the same
engine.
Caveats
This is an abliterated model: refusals are removed at the checkpoint level, and the usual
hesitation before destructive actions goes with them. Put the guardrails in the tool layer, not in
the model. rk8v4 is a lossy KV format (about +0.08 % perplexity against bf16 on the vanilla
checkpoint) chosen for the context window; --kv-dtype int8 or bf16 trade window for fidelity.