base_model: DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU base_model_relation: quantized library_name: transformers pipeline_tag: image-text-to-text license: apache-2.0 datasets:

• NeelNanda/pile-10k tags:
• qwen
• qwen3.8
• heretic
• uncensored
• finetune
• awq
• int4
• 4-bit
• auto-round
• vllm
• xpu
• intel-arc
• image-text-to-text
• vision
• mtp
• reasoning
• tool-calling quantized_by: derikn

Qwen3.8-27B TURBO-Fable Cold-Fusion 735-882 Heretic Uncensored AWQ INT4 (W4A16, group 128)

4-bit weight-only quantization of DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU, built with Intel's AutoRound and tested on Intel Arc Pro B70 GPUs.

This is an unofficial community quantization by derikn. DavidAU did not author it and does not endorse it. For the model itself, its capabilities and its evaluation results, read the base model card.

Two things to know about the base model before you use this: it is a community fine-tune of Qwen/Qwen3.8-27B, not the upstream model, and it is an uncensored build. Its outputs are not filtered for safety. Quality and behaviour come from the fine-tune, not from this quantization.

What this is

Only the linear weights are quantized. No fine-tuning and no extra training data went into this: it is the base model's weights at 4 bits.

101 modules are held at 16-bit, listed in quantization_config.modules_to_not_convert: the 96 linear-attention in_proj_a and in_proj_b projections, the 2 visual merger layers, the vision blocks, the MTP head, and lm_head.

Note on the parameter count

The Safetensors panel on this page reports a model size well below the real one. That is not the size of the model. The 4-bit weights are packed into int32 tensors, eight weights per int32, and the panel's total counts that packed storage instead of the parameters it encodes.

This checkpoint holds 27,781,427,952 parameters, the same count as the base model. The Hub API's own per-dtype breakdown adds up to that figure while the panel's single total does not. I32 and BF16 describe storage, not the precision of the model: packed 4-bit weights, and the tensors held at bfloat16.

How it was made

Quantized from the bf16 checkpoint with AutoRound 0.15.1 on a single Intel Arc Pro B70, with weights streamed to the card so VRAM stays bounded. The run was driven from the Quantize tab of a private dashboard, so this is the pipeline's invocation rather than a script you can run as-is:

python quantize.py \
  --model <base-model-dir> \
  --format auto_awq --device xpu:0 --low-gpu-mem \
  --n_samples 128 --seqlen 1024 --batch_size 8

Calibration used the quantizer's default set, NeelNanda/pile-10k, at 128 samples of 1024 tokens each. The wrapper is not published, so that exact command will not run for you. These are the AutoRound settings it passes, which are the whole recipe:

• export format auto_awq: AWQ, 4-bit weight-only, group size 128, symmetric, gemm version
• lm_head, the vision tower, the MTP head, the linear-attention projections and the visual merger excluded
• calibration on NeelNanda/pile-10k, 128 samples of 1024 tokens, batch size 8
• weights streamed to the device during tuning, to keep VRAM bounded

Running it

Tested on vLLM XPU (vllm/vllm-openai-xpu:nightly) with tensor parallelism across two Arc Pro B70s, an fp8 KV cache and the full 256K context:

vllm serve /models/Qwen3.8-27B-TURBO-Fable-AWQ-INT4 \
  --served-model-name qwen3.8-27b-turbo-fable-awq \
  --quantization awq --dtype bfloat16 \
  --max-model-len 262144 --kv-cache-dtype fp8_e4m3 \
  --block-size 64 --max-num-seqs 8 --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.91 \
  --enable-prefix-caching \
  --tool-call-parser qwen3_coder --enable-auto-tool-choice \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --trust-remote-code --tensor-parallel-size 2

Container settings that matter on Arc: --device=/dev/dri, --group-add for the render and video groups, --ipc=host, --security-opt seccomp=unconfined, and ZE_AFFINITY_MASK set to the cards you want to use. Without the device, group and seccomp combination Level Zero reports zero devices.

MTP speculative decoding

The MTP head ships at 16-bit beside the weights, and quantization_config declares it unquantized through the mtp entry in modules_to_not_convert.

That entry is required. AutoRound quantizes the module tree it loads, and the MTP head sits outside it, so without the exclusion vLLM builds the head as a packed 4-bit linear and the draft model fails at load:

ValueError: There is no module or parameter named 'fc.weight' in Qwen3_5MultiTokenPredictor

With the exclusion in place the head loads alongside the target model, and speculative decoding drafts. Measured on this checkpoint: about 77 percent draft acceptance (77,712 draft tokens against 59,609 accepted) with 3 speculative tokens.

Thinking and tool calling

Tested with the qwen3_coder tool parser and the qwen3 reasoning parser, with thinking controlled through the chat template:

--default-chat-template-kwargs '{"enable_thinking": true, "reasoning_preserve": true, "reasoning_effort": "low"}'

enable_thinking turns thinking on or off, and reasoning_effort accepts the OpenAI scale that vLLM forwards to the template.

What was tested, and what was not

Tested:

• 256K context loads and serves, at tensor parallel 2 with an fp8 KV cache.
• MTP speculative decoding loads and drafts at about 77 percent acceptance, from the retained MTP head.
• Tool calling returns a structured call, and thinking separates from the answer: vLLM reports the reasoning in message.reasoning_content and the reply in message.content.
• Text generation returns correct answers, and the vision tower (333 tensors) plus the Qwen2VL processor config load, so image input remains available.

Not tested:

• No accuracy or throughput numbers, and no image-input evaluation, are published with this repository. The quantizer's own measurements come from one home-lab setup (Arc Pro B70, XPU vLLM nightly, fp8 KV cache) and are not a fair comparison for anyone running different hardware or software. Model quality is the base model's, so read its card.

Licence and attribution

Apache License 2.0. The base model declares Apache 2.0 but its repository ships no licence file, so LICENSE carries the Apache 2.0 text published with the upstream model, and NOTICE.txt states the position in full.

• Base model: DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU (Apache 2.0 as declared)
• Upstream model: Qwen/Qwen3.8-27B (Apache 2.0)
• Quantization tooling: intel/auto-round (Apache 2.0)
• Quantizer: a private in-house pipeline around AutoRound (not published)