library_name: ninfer pipeline_tag: image-text-to-text inference: false license: apache-2.0 base_model:

• d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
• ukisai/Swift-Qwen3.8-27b
• z-lab/Qwen3.8-27B-DFlash2
• huihui-ai/Huihui-Qwen3.8-27B-abliterated base_model_relation: quantized tags:
• ninfer
• ninfer-v3
• qwen3.8
• qwen3_5
• nvfp4
• w4a4
• blackwell
• multimodal
• conversational
• cuda
• rtx-5090
• dflash2
• speculative-decoding
• abliterated
• uncensored

Qwen3.8-27B Swift-abliterated nvfp4full + DFlash2 for NInfer — v3 container

An uncensored (abliterated), all-NVFP4 Qwen3.8-27B artifact for the NInfer engine, built end-to-end on a single RTX 5090: d0xin's abliterated BF16 checkpoint → our own NVFP4 quantization pipeline → DFlash2 speculative drafter + indexed proposal head, packaged as a native v3 .ninfer container.

It is the uncensored counterpart to kaushikvira/Qwen3.8-27B-nvfp4full-dflash2-NInfer-v3 and replaced it as our production profile after a same-session A/B (below).

Credits & provenance (full chain)

This artifact exists only because of the following models and their authors — all credit to them:

The abliteration itself is d0xin's work, inherited byte-for-byte — no re-abliteration, no behavioural editing was performed here. Per ABLITERATION.json: OrcaRouter-compatible rank-1 residual-writer directional ablation, W' = W − r(rᵀ W), 131 modified tensors, direction layer reference 38, rank-1, fp32 projection math, max BF16 residual leakage 1.49e-3. d0xin's own paired evaluation (298 samples) measured MMLU-Pro +6.1 pts vs base (not significant, CI crosses 0) and math-500 unchanged.

What we did (the technique)

1. All-NVFP4 quantization of the BF16 checkpoint with llm-compressor (512 Ultrachat calibration samples, seq 2048, sequential pipeline): NVFP4 W4A4 group-16 on every text projection — MLP gate/up/down (all 64 layers), full-attention Q/K/V/O, GDN in_proj_qkv/z/out_proj — with the 9-tensor BF16 exception pattern of the registered Qwen3.6-27B profile; token embedding + output head in W8G32 groupwise (matching the cometkim nvfp4full profile); vision tower on the official q6/q8 allocation; MTP and DFlash2 drafter carried in BF16.
2. Global-divisor normalization (our own tool, the key trick). llm-compressor emits a per-module weight_global_scale; the NInfer engine's native A4 input route requires each layer's fused attention parent (GDN qkvz = 16,384 rows, full-attn qkgv = 14,336 rows) to be one contiguous region with a single divisor. We unify each packing group's divisor to D = min(dᵢ) and rescale every member's E4M3 block scales by D/dᵢ (RNE, shrink-only → overflow-free). Verified dequant-invariance: worst drift 2.7% on the smallest-magnitude elements, 0.0000% median — weights unchanged up to E4M3 re-encode noise. With uniform divisors the converter coalesces each group into one fused parent and the engine accepts the artifact unmodified.
3. Packing & conversion with the NInfer v3 converter (tools/convert @ fork 3090a5b1, 2026-09-24 rebase of upstream), --components text,vision,mtp,dflash2 --proposal.

Artifact

Serving (RTX 5090 32 GB, single GPU)

ninfer-serve qwen3_8_27b_swift_abliterated_nvfp4full-dflash2.ninfer \
  --host 0.0.0.0 --port 8080 --model-id Qwen3.8-27B \
  --max-context 262144 --kv-capacity auto --kv-dtype k8v4 \
  --max-concurrency 4 --default-max-tokens 32768 \
  --pending-timeout-ms 900000 --prefill-chunk 4096 \
  --temperature 0.9 --min-p 0.05 \
  --spec dflash2 --draft-tokens 7 --lm-head-draft \
  --host-kv-mib 49152 --host-state-slots 16 \
  --max-private-continuations 16 --max-shared-prefixes 16 \
  --vision --default-thinking-budget 16384 --preserve-thinking \
  --image-token-budget 1280

Live capacity on one 5090: weights 18.0 GiB, device KV pool 308,736 tokens (auto), k8v4, 48 GiB pinned host-KV arena, 262,144-token max context at concurrency 4. Full context is verified working (below).

Benchmarks — same-session A/B vs the prior production artifact

Both sides served by the same engine build, same port, fresh generations, temp 0, no cache reuse, RTX 5090 @ 450 W / SM clock pinned 2280 MHz.

Long-context recall method: needle-in-a-haystack, haystack 1,000,000 chars (~250k tokens), secret at three depths, exact-match verdicts, zero corrupted / one transient empty response at 75k (passed on re-run).

Why it is smaller and faster

The prior profile stored attention in heavier formats and carried a 2.23 GB W8G32-encoded drafter. This build quantizes the entire text backbone (attention included) to NVFP4 behind fused parents — NVFP4 tensor-core GEMMs being the fastest path on Blackwell — and the freed VRAM is handed to the device KV pool by --kv-capacity auto (+29,824 tokens ≈ 0.7 GiB). Abliteration itself adds zero size: it is a rank-1 weight edit, no new parameters.

Known behaviour

• At temperature 1.0 the model occasionally emits duplicate tool calls or a short repetition loop in streaming tool-call tests (all emitted calls remain valid JSON; the streaming parser handles them). At temperature 0 this has never been observed across many runs; production sampling (temp 0.9, min-p 0.05) suppresses it.

License

Apache-2.0, inherited from the base artifact chain. Abliteration and base-model licenses: see the credited repositories above — all credit to d0xin, ukisai, huihui-ai, z-lab and Neroued.