license: apache-2.0 base_model: DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored base_model_relation: quantized pipeline_tag: image-text-to-text library_name: transformers language:

• en tags:
• nvfp4
• fp4
• w4a4
• gptq
• quantized
• compressed-tensors
• llm-compressor
• vllm
• vision-language
• thinking
• mtp
• speculative-decoding
• math-calibration
• gsm8k
• kv-cache-calibrated
• chat-template-patched

Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L — NVFP4 (W4A4, KV-calibrated)

Quantization-only derivative of DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored (the 709-L "Stage 2b-rplus3" TWIN-TURBO checkpoint). No training, no additional abliteration. All credit for the model itself belongs upstream. One exception is the chat template, which is patched in this repo — see Chat template (patched in this repo).

Of the Qwen3.8-27B DavidAU derivatives we have published, this 709-L base measured out as the one we would recommend first: it is the strongest of the three on raw benchmarks, and the W4A4 cost here is the smallest we have seen on this architecture.

This is a vLLM-first build: the quant format, the KV-cache calibration, the MTP speculative config, and the chat-template patch below are all tuned for vLLM's compressed-tensors path. We have not tested llama.cpp / SGLang (and the einstein/spoon modes in particular are unverified outside vLLM).

For Blackwell users: 4-bit activations (the "A4" in W4A4) is the part a W4A16 build leaves at full precision, so on sm_120/sm_121-class hardware the 4-bit activation path is the one that unlocks the extra decode throughput — expect this build to run noticeably faster than a W4A16 quant of the same model at the quality measured below.

Quality Verification

All values below were benchmarked under identical conditions: lm-eval 0.4.12 (vLLM backend) in the vllm/vllm-openai:qwen38-x86_64-cu130 image, single RTX PRO 6000 Blackwell (96 GB), --gpu-memory-utilization 0.85, max_model_len 4096, greedy decoding. Prompts are 0-shot raw task prompts — no chat template, no thinking (equivalent to the "Instruct mode" the upstream card benches in). Worst relative degradation: 2.6 % (QC threshold 10 %).

Reading: every task is within ~2.5 pt of the BF16 source; gsm8k keeps 98 % of the source score. The only notable W4A4 cost is the wikitext PPL (+8.1 %), the expected 4-bit language cost on this base. Absolute values differ from third-party harness numbers (e.g. the upstream card's llama.cpp benches); compare against the BF16 column, which was measured under the same conditions.

Chat template (patched in this repo)

This repo ships a hardened chat_template.jinja as the default, and the unmodified upstream template as chat_template-upstream-original.jinja for reference. The exact patch is in template_patch/chat_template.hardened.diff.

Upstream base: the template as of commit 9cdc928b of the upstream repo (2026-09-12, the agentic/tool-calling fix). That fix is folded into this file; everything else in it is upstream.

Potential risks in the upstream template as shipped. It implements an in-message mode switch by scanning messages for a literal {REASON: tag. For many uses this is never a problem, but the design carries several independent sharp edges:

• Scope: it scans every message in the history — assistant, tool results, everything — not just what the user typed. Any text that merely quotes the feature (reading a file, fetching a URL, a RAG document, a pasted example) can silently switch the model's reasoning mode, or fail the request entirely.
• Hard failure: a value outside the whitelist (xhigh/medium/low/ einstein/spoon) makes the template raise_exception, aborting the whole request — instead of being ignored.
• Self-trigger: the template source itself contains the literal {REASON:, so the moment the file's own content appears in the conversation (e.g. an agent that inspects its own config), the parser extracts a code fragment as the "mode" and every later request in that session fails until a new chat is started.
• Vision path: the scan pass renders every message as if it were a system message, so a user message containing an image or video fails the whole request with System message cannot contain images before generation even starts. In other words, served via vLLM, image input was not usable at all with this template. We think the upstream GGUF card's "Vision: tested" refers to the GGUF/mmproj path (llama.cpp), which does not run this jinja scan and never hit this code path — so the issue was not visible from the GGUF side.

The patch closes all four of these with a 7-line change to the parse block; every other line is byte-identical to upstream.

Semantics that differ from the upstream README. The mode switch is now deliberately conservative and stateless:

The five modes themselves and their system-prompt blocks are untouched. Verification: 24/24 robustness/repro checks (including both incident repros on the unmodified template) + byte-for-byte output parity with the original template for all 5 thinking + 5 instruct modes (local jinja2; same engine family as vLLM).

API note. The OpenAI-compatible top-level reasoning_effort field is still validated by vLLM against the standard 7-value ladder, so reasoning_effort="einstein"/"spoon" at the API level is rejected regardless of template. Use extra_body={"chat_template_kwargs": {"reasoning_effort": "einstein"}}, or the in-message tag.

If you prefer the exact upstream template, serve with --chat-template <repo>/chat_template-upstream-original.jinja.

Reproducibility

Built with llm-compressor oneshot + GPTQModifier (recipe.yaml included: block size 128, dampening 0.01, actorder static, MSE observer, group size 16) with a domain-matched calibration blend (long reasoning + general chat + code + GSM8K train math), on a GB10 (sm_121, CUDA 13.0, torch 2.11.0+cu130, transformers 5.14.1, llm-compressor 0.13.0), with the KV-cache calibration pass enabled (kv_cache_scheme: fp8, symmetric, per-tensor, memoryless_minmax) riding on the same one-shot pass.

What the KV calibration adds. 32 scalar FP32 tensors — k_scale/v_scale (one per full-attention layer's k_proj/v_proj; this model has 16 full-attention layers, the rest being Gated DeltaNet linear attention, which keeps a recurrent state and has no paged KV cache) — plus quantization_config.kv_cache_scheme in config.json. vLLM's compressed-tensors path reads these into the attention layers and multiplies the loaded K/V entries by the per-layer scale when quantizing them into the fp8 paged cache.

Post-quantization structural fixes (MTP restoration from the source's model-mtp-restored.safetensors, index.json rebuild, ignore guarantee for mtp.* in config.json, tower split into model-towers.safetensors) are idempotent and re-derivable from the BF16 source. Structural check: PASS (source keys 1199, missing 0). MTP draft head verified working with the speculative config below; vision tower verified structurally (weights intact).

Serve with vLLM

vllm serve <this-repo> \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}' \
  --kv-cache-dtype fp8 --enable-prefix-caching \
  --max-num-batched-tokens 2048 --max-num-seqs 2 \
  --gpu-memory-utilization 0.44 \
  --reasoning-parser qwen3 --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

(Flag values are a tested GB10 / 128 GB unified-memory configuration — 256K context works at this setting. On a dedicated GB10 you can raise --gpu-memory-utilization toward ~0.8 for a larger KV pool; avoid ~0.95, which starves the OS on a unified pool.)

Engineering notes

• This quantization run (calibration, QC, card, upload) was performed with an LLM agent operating the operator's own toolchain.
• The chat template hardening (incident repro, patch, and test suites) was likewise done with the operator's agent; the diff is in template_patch/.

Lineage

Qwen/Qwen3.8-27B → DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU → DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored → this. Apache 2.0, unbroken from the base model.