license: other license_name: swift-open-license-1.0 license_link: LICENSE base_model: d0xin/Swift-Qwen3.8-27B-Uncensored-BF16 base_model_relation: quantized pipeline_tag: image-text-to-text library_name: transformers language:

• en
• zh tags:
• qwen3.8
• swift
• abliterated
• uncensored
• w4a16
• 4-bit
• int8
• compressed-tensors
• autoround
• vllm
• marlin
• mtp
• speculative-decoding
• efficient-thinking
• reasoning
• function-calling
• vision-language
• rtx-3090

Swift-Qwen3.8-27B-Uncensored-W4A16

A 4-bit (W4A16) quantization of d0xin/Swift-Qwen3.8-27B-Uncensored-BF16: UkisAI's reasoning-efficient Swift-Qwen3.8-27B with its refusal direction removed. It serves on one 24 GB GPU (RTX 3090 class) with vLLM and the HyperQwen stack, with the vision tower, the MTP speculative-decoding head and tool calling kept.

A faster variant with int4 heads is Swift-Qwen3.8-27B-Uncensored-W4A16-fast.

Lineage

Qwen/Qwen3.8-27B                              Apache 2.0
 └─ ukisai/Swift-Qwen3.8-27b                  Swift Open License v1.0, reasoning-efficiency LoRA merged
     └─ d0xin/Swift-Qwen3.8-27B-Uncensored-BF16   rank-1 directional ablation, layer 38
         └─ this model                        W4A16 g128 body, int8 lm_head / embed_tokens / MTP
             └─ ...-W4A16-fast                int4 GPTQ lm_head / MTP

What is in this checkpoint

2,022 tensors, 67 safetensors files, 16.7 GB on disk. No group has zero points.

The draft vocabulary is the top 40,960 ids of 6,761 generations by this model (4.59 M output tokens). On the held-out 10 % of those generations it covers 97.68 % of tokens. For comparison, the generic list that the official Qwen3.8-27B-W4A16-AutoRound checkpoint uses covers 98.18 % of the same tokens. The draft head only matters for SPEC=mtp.

How it was made

1. run_quant.sh from HyperQwen: Intel AutoRound, --scheme W4A16 --bits 4 --group_size 128, 128 calibration samples x 2,048 tokens, lm_head not quantized, auto_round:llm_compressor export.
2. prepare/quant_lm_head.py, prepare/quant_embed.py, prepare/quant_mtp.py: int8 round-to-nearest heads. Relative error against the BF16 source weights (||dequant - w|| / ||w||): lm_head 0.0064, MTP linears 0.0066–0.0153.
3. prepare/build_draft_vocab.py: the draft head from the own-output id list.

Properties of the BF16 source

These come from the source and were not measured again on this quantization:

• Refusals (d0xin, fixed 100-prompt set): 88 direct answers, 10 answers with a safety note, 2 other failures, 0 refusals.
• Capability (d0xin, fixed 298-example set against the original Swift BF16): 39.93 % against 38.26 % combined, +1.68 points, McNemar p = 0.44, no measurable loss.
• Method: rank-1 directional residual-stream ablation at layer 38; 131 of 1,199 tensors changed, vision tensors unchanged. Details: ABLITERATION.json, STRUCTURAL_VALIDATION.json and INTELLIGENCE_VALIDATION.json in the source repository.
• Shorter reasoning (UkisAI): Swift is trained to avoid over-thinking. Our GSM8K run agrees: 356 answer tokens on average against 379 for the official AutoRound base and 384 for the abliterated OrcaRouter quant, with thinking off.

Measured results (single RTX 3090, 24 GB)

All numbers use the HyperQwen stack on vLLM 0.28.0, WSL2 + Docker, thinking off (protocol v2). The official Qwen3.8-27B-W4A16-AutoRound checkpoint is the reference.

Single-user decode (bench/run_benchmarks.sh single), 2026-09-23, card at 250 W, SPEC=dflash2 (DFlash2 drafter, 7 draft tokens), PREFIX_CACHE=1, KV_MEM=4529848320 (4.22 GiB), MAX_LEN=49152. End-to-end throughput in tok/s, sampling T=default / T=0. One run per checkpoint after a warm-up request.

C1 accepts 3.72 / 3.81 tokens per verify step, mean time to first token 188 ms. Expect 3–5 % variation between sessions.

Batch serving (bench/run_benchmarks.sh batch --prefill --long), 2026-09-21, card at 350 W, KV=fp8, no speculation, second of two runs.

Quality (bench/quality_battery.py against the served model): perplexity over ~300-token windows of wikitext-2 (en), fineweb-2 Danish (da) and Python source (code); GSM8K exact match, first 200 test questions, greedy, thinking off.

With n=200 the GSM8K standard error is about 1 point at 98 % and 1.6 points at 94.5 %.

How to serve

HyperQwen (Linux or WSL2, one 24 GB GPU), in .env:

MODEL=/app/models/Swift-Qwen3.8-27B-Uncensored-W4A16
SPEC=dflash2          # or SPEC=mtp to draft with the built-in MTP head
PREFIX_CACHE=1

then docker compose --profile single up -d. bash verify.sh --no-server checks the directory before you serve it.

Plain vLLM (0.28 or later) loads the body and heads through compressed-tensors:

vllm serve TyroneNel/Swift-Qwen3.8-27B-Uncensored-W4A16 \
  --max-model-len 32768 --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

The truncated MTP draft head (mtp.draft_lm_head.* with mtp_draft_vocab_ids.pt) is read by HyperQwen's patches/qwen3_5-mtp-draft-vocab.patch. It is not tested on unpatched vLLM.

Sampling defaults (generation_config.json, from Swift): temperature 1.0, top_p 0.95, top_k 20, min_p 0, repetition_penalty 1.0. For thinking off, Qwen recommends temperature 0.7, top_p 0.8, presence_penalty 1.5. The chat template accepts reasoning_effort low, medium and xhigh (default), and maps minimal to low and high / max to xhigh.

Files

Safety

This is a refusal-reduced model. It can produce content that the upstream model refuses or handles with more care, and its output can be wrong, offensive or unsafe. Use it for research, development and other lawful purposes. Review its output where an error can cause harm. You are responsible for how you use it and for compliance with the licenses and the law.

License

The weights are distributed under the Swift Open License v1.0 (LICENSE). It permits use, modification and redistribution. Commercial use is licensed only while you, together with every entity that controls, is controlled by or is under common control with you, had gross revenue below US$1,000,000 in the most recently completed fiscal year (Section 5). Above that threshold, commercial use needs a Swift Enterprise License from UkisAI. The base model's Apache License 2.0 is included as LICENSE-APACHE-2.0 (Section 4(e)); NOTICE lists the attribution chain, UkisAI's notices and every change. This summary is not legal advice; LICENSE is the binding text.

Original Swift model: UkisAI. Abliteration: d0xin. Quantization and serving preparation: TyroneNel.