license: apache-2.0 base_model: orcarouter/Qwen3.8-27B-Uncensored base_model_relation: quantized pipeline_tag: image-text-to-text library_name: transformers language:

• en
• zh tags:
• abliterated
• uncensored
• qwen
• qwen3
• qwen3.8
• 4-bit
• w4a16
• int8
• compressed-tensors
• vllm
• marlin
• mtp
• reasoning
• function-calling
• vision-language
• ampere
• rtx-3090

Qwen3.8-27B-Uncensored-W4A16-g128

Community W4A16 (4-bit weight) quant of orcarouter/Qwen3.8-27B-Uncensored — built with Intel AutoRound in llm-compressor compressed-tensors format, post-processed for vLLM Marlin serving and DFlash2 speculative decoding on a single RTX 3090 (24 GB, Ampere).

The abliterated (refusal-removed) Qwen3.8-27B at ~17 GB — fits one 24 GB card with room for KV cache, keeps the full vision tower, the MTP head, tool calling, and the flexible thinking control of the base. Weights-only quantization: the abliteration edit from the source release is carried through byte-for-byte.

Lineage

Qwen/Qwen3.8-27B                       (upstream, Apache 2.0)
  └─ orcarouter/Qwen3.8-27B-Uncensored (BF16 abliterated finetune — refusal direction removed)
       └─ Qwen3.8-27B-Uncensored-BF16  (56 GB, 18 shards — local quantization source)
            └─ THIS MODEL              (W4A16 g128 + int8 embed/lm_head/MTP, 2026-09-10)

Quantized from the local BF16 source tree G:\models\Qwen3.8-27B-Uncensored-BF16 (byte-identical to orcarouter/Qwen3.8-27B-Uncensored, revision 404ea47aaa5d8a8b00049c9e9750089aca011ab2 — that source repo is gated with auto-approval on HF; accept the agreement to download it).

What changed vs the BF16 source

The template translation is deliberate: the upstream template raises a TemplateError (vLLM returns HTTP 400) when a client sends reasoning_effort: "minimal" / "high" / "max". Everything else in the template is byte-identical to upstream. A .bak-effort copy of the original was kept locally and is not part of this release.

Quantization recipe

• Tool: Intel AutoRound (upstream main at commit 1139c323, plus a local pack-guard patch for the unquantized linear_attn/visual layers), scheme W4A16, --bits 4 --group_size 128, calibration on NeelNanda/pile-10k (AutoRound's default VL set — confirmed from the run log), --nsamples 128 --seqlen 2048, 200 iterations (tool default), --batch_size 4, format auto_round:llm_compressor.
• Hardware: one RTX 3090 (24 GB) + one RTX 3080 Ti (12 GB), block-streaming calibration.
• Post-quant (repo pipeline, syv-ai/qwen38-27b-rtx3090): int8 lm_head / embed_tokens / MTP-MLP packing, 40k draft-vocab build, verify.sh gate.
• 411 tensors packed — 400 W4A16 int4 layer linears + 11 int8 (lm_head, embed_tokens, 9 MTP); 790 tensors left BF16 (norms, vision, MTP embeddings); full tensor manifest via model.safetensors.index.json.

Quality on this exact artifact

Measured serving this checkpoint with vLLM 0.28.0 + DFlash2 speculative decoding (k=7) on an RTX 3090:

The capability-retention evals in the source card (MMLU / MMLU-Pro / CMMLU / safety suites) were measured on the BF16 source and its FP8/GGUF derivatives — quantization is the only change here; the 4-bit weights inherit those numbers up to the usual W4A16 cost, which GSM8K puts at none measurable (98.0 with thinking on vs 95.5 without on the same stack).

Daily-driver verified: Marlin W4A16 serving, multi-turn tool calling (qwen3_coder parser), thinking control, DFlash2 speculative decoding, loopback OpenAI-compatible API.

Serving (vLLM ≥ 0.28.0)

vllm serve /path/to/Qwen3.8-27B-Uncensored-W4A16-g128 \
  --served-model-name qwen3.8-27b \
  --max-model-len 131072 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

• RTX 3090 (24 GB): weights ~15.8 GiB on load; --gpu-memory-utilization 0.93 leaves a large KV pool for 128k context. On WSL2/Windows, 0.93 (not 0.97) avoids a startup OOM.
• Native context is 262,144; the official 1M recipe (static YaRN via --hf-overrides) applies.
• Speculative decoding — pair with the DFlash2 drafter per the official vLLM recipe: --speculative-config '{"method":"dflash","model":"incoai/Qwen3.8-27B-DFlash2","num_speculative_tokens":7}' (the repo pipeline builds mtp_draft_vocab_ids.pt for its drafter export — included here for completeness).
• --reasoning-parser qwen3 is effectively required: the template opens every assistant turn with <arg_key>, so without it the reasoning lands in content and eats the token budget.
• FP8/NVFP4 are not options here (Ampere has no FP8/NVFP4 tensor cores) — that is what this release is for.

Chat Completions / thinking control

Thinking is on by default. Toggle per request:

resp = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=messages,
    reasoning_effort="low",            # low | medium (default) | xhigh — plus OpenAI aliases minimal/high/max
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},  # direct answer, no <arg_key>
)

preserve_thinking defaults to true (prior-turn reasoning retained). generation_config.json ships the thinking-mode defaults (temperature=1.0, top_p=0.95, top_k=20, do_sample) — the same values upstream Qwen/Qwen3.8-27B ships; vLLM applies them as OpenAI-API request defaults. For non-thinking mode, override per request: temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5.

Files

Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream), inherited unchanged from the BF16 source.

• It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails.
• It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
• Use must comply with the Apache 2.0 License inherited from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm arising from this model. Its outputs do not reflect the views of the uploaders, of OrcaRouter, or of Qwen / Alibaba.

By downloading or using this model you acknowledge and accept the above.

Intended use / out of scope

• Intended: research into refusal mechanisms, alignment, interpretability; red-teaming and safety evaluation in controlled environments; single-user local inference on 24 GB Ampere hardware.
• Out of scope: any use violating the base model's Apache 2.0 license or applicable law; deployment to the public or end users without additional safety and moderation layers; content intended to harm.

Bias, risks, and limitations

• Safety guardrails removed — the model will produce harmful, biased, or offensive content on request.
• Inherits biases/limitations of the base Qwen3.8-27B and of the abliteration edit (see the BF16 source card for the full evaluation).
• 4-bit weights: fine-tuning on top of this checkpoint should start from the BF16 source instead.

License

Apache 2.0, inherited from Qwen/Qwen3.8-27B through the OrcaRouter abliterated release. Quantization does not change the underlying license obligations.

Acknowledgements

• Qwen/Qwen3.8-27B — base model (Apache 2.0)
• orcarouter/Qwen3.8-27B-Uncensored — abliterated BF16 source (OrcaRouter / Continuum-AI-Corp)
• incoai/Qwen3.8-27B-DFlash2 — DFlash2 drafter
• Tooling: syv-ai/qwen38-27b-rtx3090 (patch series, post-quant pipeline), Intel AutoRound