license: other license_name: swift-open-license-1.0 license_link: LICENSE base_model: d0xin/Swift-Qwen3.8-27B-Uncensored-BF16 base_model_relation: quantized pipeline_tag: image-text-to-text library_name: transformers language:

• en
• zh tags:
• qwen3.8
• swift
• abliterated
• uncensored
• w4a16
• 4-bit
• int4
• int8
• gptq
• compressed-tensors
• autoround
• vllm
• marlin
• mtp
• speculative-decoding
• efficient-thinking
• reasoning
• function-calling
• vision-language
• rtx-3090

Swift-Qwen3.8-27B-Uncensored-W4A16-fast

The "fast variant" of Swift-Qwen3.8-27B-Uncensored-W4A16: the same 4-bit body of d0xin/Swift-Qwen3.8-27B-Uncensored-BF16 (UkisAI's reasoning-efficient Swift-Qwen3.8-27B with its refusal direction removed), with the lm_head and the MTP module requantized to int4 GPTQ. It serves on one 24 GB GPU (RTX 3090 class) with the HyperQwen stack. The int4 lm_head makes every decode step read about 0.65 GB less; that is where the gain over the base build comes from.

Lineage

Qwen/Qwen3.8-27B                              Apache 2.0
 └─ ukisai/Swift-Qwen3.8-27b                  Swift Open License v1.0, reasoning-efficiency LoRA merged
     └─ d0xin/Swift-Qwen3.8-27B-Uncensored-BF16   rank-1 directional ablation, layer 38
         └─ ...-W4A16                         W4A16 g128 body, int8 lm_head / embed_tokens / MTP
             └─ this model                    int4 GPTQ lm_head / MTP

What changed against the base build

Relative error is ||dequant - w|| / ||w|| against the BF16 source weights. GPTQ uses 300,000 hidden states captured from this model's own generations, so it minimizes the output error on real activations rather than the weight error that this number shows. On those activations the KL divergence to the BF16 lm_head is 0.00251 for GPTQ int4, against 0.00558 for round-to-nearest int4. All groups are symmetric (no zero points).

The MTP module was evaluated on a held-out split of the captured data (greedy chain simulation, 2 draft positions): 2.269 tokens per step, first-position top-1 agreement 0.777, acceptance 0.845, n = 107,063 positions.

The draft vocabulary is the top 40,960 ids of 6,761 generations by this model (4.59 M output tokens, 3,119 of them with thinking on; prompts: English chat, code, Danish instructions and reasoning, GSM8K train). On the held-out 10 % of those generations it covers 97.68 % of tokens; the generic list that the official Qwen3.8-27B-W4A16-AutoRound checkpoint uses covers 98.18 %. The draft head only matters for SPEC=mtp.

How it was made

1. drafter/collect_prompts.py, drafter/gen_data.py, drafter/capture.py: generate with this model and capture its final hidden states.
2. drafter/gptq_lm_head.py --bits 4 --calib-rows 300000: GPTQ int4 lm_head.
3. drafter/train_mtp.py --eval-only 1 --dump-hessians, then drafter/requant_mtp_gptq.py --bits 4: GPTQ int4 MTP module.
4. prepare/build_draft_vocab.py: the draft head from the own-output id list.

The tools are in HyperQwen (drafter/, prepare/).

Properties of the BF16 source

These come from the source and were not measured again on this quantization:

• Refusals (d0xin, fixed 100-prompt set): 88 direct answers, 10 answers with a safety note, 2 other failures, 0 refusals.
• Capability (d0xin, fixed 298-example set against the original Swift BF16): 39.93 % against 38.26 % combined, +1.68 points, McNemar p = 0.44, no measurable loss.
• Method: rank-1 directional residual-stream ablation at layer 38; 131 of 1,199 tensors changed, vision tensors unchanged. Details in the source repository.
• Shorter reasoning (UkisAI): our GSM8K run shows 355 answer tokens on average against 379 for the official AutoRound base, with thinking off.

Measured results (single RTX 3090, 24 GB)

All numbers use the HyperQwen stack on vLLM 0.28.0, WSL2 + Docker, thinking off (protocol v2).

Single-user decode (bench/run_benchmarks.sh single), 2026-09-23, card at 250 W, SPEC=dflash2 (DFlash2 drafter, 7 draft tokens), PREFIX_CACHE=1, KV_MEM=4529848320 (4.22 GiB), MAX_LEN=49152. End-to-end throughput in tok/s, sampling T=default / T=0. One run per checkpoint after a warm-up request.

C1 accepts 3.67 / 3.75 tokens per verify step, mean time to first token 184 ms. Expect 3–5 % variation between sessions.

Batch serving (bench/run_benchmarks.sh batch --prefill --long), 2026-09-21, card at 350 W, KV=fp8, no speculation, second of two runs.

Quality (bench/quality_battery.py against the served model): perplexity over ~300-token windows of wikitext-2 (en), fineweb-2 Danish (da) and Python source (code); GSM8K exact match, first 200 test questions, greedy, thinking off.

The int4 heads cost 0.5 % perplexity. With n=200 the GSM8K standard error is about 1.2 points at this accuracy, so the one-point GSM8K difference is within noise.

How to serve

HyperQwen (Linux or WSL2, one 24 GB GPU), in .env:

MODEL=/app/models/Swift-Qwen3.8-27B-Uncensored-W4A16-fast
SPEC=dflash2          # or SPEC=mtp to draft with the int4 MTP head
PREFIX_CACHE=1

then docker compose --profile single up -d. bash verify.sh --no-server checks the directory before you serve it.

Plain vLLM (0.28 or later) loads the body and heads through compressed-tensors:

vllm serve TyroneNel/Swift-Qwen3.8-27B-Uncensored-W4A16-fast \
  --max-model-len 32768 --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

The truncated MTP draft head (mtp.draft_lm_head.* with mtp_draft_vocab_ids.pt) is read by HyperQwen's patches/qwen3_5-mtp-draft-vocab.patch. It is not tested on unpatched vLLM.

Sampling defaults (generation_config.json, from Swift): temperature 1.0, top_p 0.95, top_k 20, min_p 0, repetition_penalty 1.0. For thinking off, Qwen recommends temperature 0.7, top_p 0.8, presence_penalty 1.5. The chat template accepts reasoning_effort low, medium and xhigh (default), and maps minimal to low and high / max to xhigh.

Files

Safety

This is a refusal-reduced model. It can produce content that the upstream model refuses or handles with more care, and its output can be wrong, offensive or unsafe. Use it for research, development and other lawful purposes. Review its output where an error can cause harm. You are responsible for how you use it and for compliance with the licenses and the law. Fine-tuning should start from the BF16 source, not from these 4-bit weights.

License

The weights are distributed under the Swift Open License v1.0 (LICENSE). It permits use, modification and redistribution. Commercial use is licensed only while you, together with every entity that controls, is controlled by or is under common control with you, had gross revenue below US$1,000,000 in the most recently completed fiscal year (Section 5). Above that threshold, commercial use needs a Swift Enterprise License from UkisAI. The base model's Apache License 2.0 is included as LICENSE-APACHE-2.0 (Section 4(e)); NOTICE lists the attribution chain, UkisAI's notices and every change. This summary is not legal advice; LICENSE is the binding text.

Original Swift model: UkisAI. Abliteration: d0xin. Quantization and serving preparation: TyroneNel.