base_model: DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP base_model_relation: quantized license: apache-2.0 library_name: transformers pipeline_tag: image-text-to-text language:

• en tags:
• nvfp4
• fp4
• w4a4
• gptq
• quantized
• compressed-tensors
• llm-compressor
• vllm
• qwen3_5
• vision-language
• thinking
• mtp
• speculative-decoding
• uncensored
• heretic

TL;DR: DavidAU's Qwen3.6-27B Fable-Fusion-711 (Uncensored/Heretic, with an MTP head), quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. ~20 GB, 256K thinking VL, uncensored (from the base). The MTP head and vision tower are kept at bf16, 4-bit for the 27B backbone, full precision for speculative decoding and image input.

Qwen3.6-27B Fable-Fusion-711 NVFP4

DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP, quantized to NVFP4 (W4A4) in the compressed-tensors nvfp4-pack-quantized format with llm-compressor (GPTQ + MSE, shared fused-layer scales). This is a quantization only, the model is already uncensored/heretic upstream; no further abliteration was applied here.

• Built for vLLM on NVIDIA Blackwell (4-bit weight + 4-bit activation). Pre-Blackwell GPUs run it weight-only.
• The MTP head (mtp.*) and vision tower (model.visual.*) are kept at bf16, so speculative decoding and image input are unaffected by 4-bit.
• Loading and generation verified in vLLM on an NVIDIA GB10 (Blackwell, sm_121).

Uncensored model (inherited from the base). It follows instructions without refusal guardrails. You are responsible for how you use it.

Why the MTP head is kept at bf16

The base ships a multi-token prediction (MTP) head (mtp_num_hidden_layers: 1) that lets the model self-speculate: it drafts the next token(s) and verifies in one pass, cutting decode latency without a separate draft model.

That head sits on the acceptance-rate critical path. Quantizing it to 4-bit would push its drafts away from what the full-precision main model predicts, the verifier would reject more of them, and the speedup would shrink. So the recipe ignores re:.*mtp.* and keeps the head at bf16 (it is tiny next to the 27B backbone). The 4-bit backbone does the heavy lifting; the full-precision MTP head keeps acceptance high. The vision tower (re:.*visual.*) is kept at bf16 for the same fidelity reason.

Fidelity

Near-lossless versus the bf16 source, ~20 GB vs 55.6 GB bf16 (~36%). GPTQ error compensation and an MSE observer keep the drop from bf16 minimal; the header lists the full characteristics and Quantization covers the recipe.

Quickstart

NVFP4 is auto-detected from config.json (compressed-tensors); no quantization flag needed. --reasoning-parser qwen3 splits the <think> block into reasoning_content.

vllm serve maci0/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4 \
  --served-model-name fable-fusion-27b-nvfp4 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3

• Supports up to 262144 tokens; keep at least 128K to preserve thinking quality.
• Add --language-model-only to skip the vision tower and free KV cache for text use.

Speculative decoding (MTP)

The bf16 MTP head enables vLLM's built-in self-speculative decoding:

vllm serve maci0/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4 \
  --served-model-name fable-fusion-27b-nvfp4 \
  --max-model-len 131072 --kv-cache-dtype fp8 --reasoning-parser qwen3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'

num_speculative_tokens matches the head depth (mtp_num_hidden_layers: 1). Acceptance is not penalized by quantization because the head is bf16.

Quantization

The model is loaded with AutoModelForImageTextToText so the vision tower is quantized-and-kept (bf16) rather than dropped; the MTP head, which lives outside the transformers model graph, is re-attached at bf16 from the base. GPTQ is a quantization-time cost only; inference speed and format are identical to plain round-to-nearest NVFP4, but it chooses better 4-bit values.

About the base model

DavidAU's Fable-Fusion-711 is a 27B Qwen3.5-family (qwen3_5) vision-language model with thinking mode, a 256K context window, and an MTP head, already run through Heretic uncensoring.

• 64 decoder layers; hybrid gated delta-net linear attention plus full attention; dense MLP; vision tower.
• MTP head (mtp_num_hidden_layers: 1) for speculative decoding.
• 256K context (max_position_embeddings 262144).

Recommended sampling

Thinking mode is the default.

• Thinking, precise: temperature=0.6, top_p=0.95, top_k=20
• Thinking, general: temperature=1.0, top_p=0.95, top_k=20
• Instruct / non-thinking: temperature=0.7, top_p=0.80, top_k=20

Related

• Base model: DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP
• Space: Rogue Quants
• Collection: NVFP4 Quants
• Sibling NVFP4 quants:
  • ThinkingCap-Qwen3.6-27B abliterated
  • Qwopus3.6-27B-Coder abliterated
  • Qwen3.6-40B Deckard

Notes

• Needs NVIDIA Blackwell (sm_121, e.g. GB10) for accelerated W4A4; pre-Blackwell GPUs run it weight-only.
• The MTP head and vision tower are bf16; enable speculative decoding via --speculative-config.
• --reasoning-parser is not auto-detected; pass it explicitly.
• No refusal guardrails; you are responsible for how you use it.

License

Apache-2.0, following the base model. Intended use and all responsibility for use follow the base model.

Credits

• Base model: DavidAU
• Uncensoring: Heretic by Philipp Emanuel Weidmann (applied upstream)
• Quantization tooling: llm-compressor / compressed-tensors