license: apache-2.0 tags:

• transformers
• gguf
• qwen3
• thinkingcap
• efficient-thinking
• reasoning
• token-efficient
• lora
• sft
• abliterated
• multi-token-prediction
• mtp
• llama.cpp
• text-generation base_model:
• hotdogs/Qwen3.8-27B-thinkingcap-abliterated model:
• hotdogs/Qwen3.8-27B-thinkingcap-abliterated-mtp-GGUF datasets:
• hotdogs/thinkingcap-sft-qwen38-27b pretty_name: Qwen3.8-27B ThinkingCap Abliterated (MTP GGUF)

Qwen3.8-27B ThinkingCap Abliterated (MTP GGUF)

llama.cpp GGUF from the ThinkingCap SFT-merged model (hotdogs/Qwen3.8-27B-thinkingcap-abliterated).

Trained from the on-policy oracle-verified dataset.

All 3 quantizations preserve the MTP layer (blk.64, 15 tensors including 4 nextn.* projection tensors). 866 tensors total, 27.3B params.

Files

MTP verification

import gguf
g = gguf.GGUFReader("Qwen3.8-27B-thinkingcap-abliterated-mtp-f16.gguf")
mtp = [t.name for t in g.tensors if 'blk.64' in t.name]
print(len(mtp))  # 15

Serve (llama.cpp)

CUDA_VISIBLE_DEVICES=0,1,2,3,4 llama-server \
  -m Qwen3.8-27B-thinkingcap-abliterated-mtp-Q6_K.gguf \
  --n-gpu-layers 999 --ctx-size 8192 --parallel 2 \
  --batch-size 4096 --flash-attn on \
  --temp 1 --top-k 20 --top-p 0.95 --min-p 0.0 --jinja

📊 Benchmarks

Tested with inspect_ai 0.3.260 on local llama.cpp, reasoning_effort=low, temperature=1, top_k=20, top_p=0.95. All runs are single-epoch (no multi-epoch averaging).

Benchmark comparison — this repo vs frontier models

¹ Official vendor-reported scores from the Qwen3.8-27B model card, reasoning_effort=xhigh. ² Standard Google IFEval via inspect_evals/ifeval (541 prompts, prompt-level strict accuracy). ⁴ inspect_evals/swe_bench_verified_mini: 50 issues (25 django + 25 sphinx), message_limit=100 steps, max_tokens=16384/turn, tool_timeout=210s, no internet, Docker sandbox. ⁶ External/independent evaluations (not the official model card). SWE-bench Verified (500) is a different, harder split than our 50-task mini. Treat as directional, not exact. ⁷ External comparison via serenitiesai.com (F16 reference values). Treat as directional. ⁸ From SmartScope Gemini 3.1 Pro Benchmark Analysis (2026). GPT-5.5 and Kimi K3 SWE scores from MorphLLM SWE-bench Pro Leaderboard. ⁹ 31/49 of-scored (1 out: sphinx-doc__sphinx-9229, step-limited at 117 msgs). Run status error (APITimeoutError at scheduler level) but all 50 samples completed in summaries. ¹⁰ 34/49 of-scored (1 out: sphinx-doc__sphinx-9229, timeout at 162 msgs / 190 min). Run status cancelled — last sample hit the 3600s timeout.

Reading it: Q6_K scores within 2.1 pp of the full BF16 Qwen3.8-27B on GPQA (87.1 vs 89.2) — with a quantized, abliterated, LoRA-merged GGUF running on consumer GPUs. On IFEval, Q6_K (88.2%) and Q4_K_M (88.4%) are near-identical and sit within ~3 pp of the F16 reference (~91.5% for Qwen3.8-27B, 95.0% for Opus 4.6). On SWE-bench, Q6_K leads at 69.4% (34/49) vs Q4_K_M 63.3% (31/49) — both timed out on the same hard issue (sphinx-9229); external runs of the full 500-task SWE-bench Verified put Qwen3.8-27B at ~82% (footnote ⁶).

⚠️ Hardware limitation: Two-box setup — Box A: 1× RTX 3090 24GB + 1× RTX 3060 12GB (36 GB), Box B: 1× RTX 4060 Ti 16GB + 1× RTX 3060 12GB (28 GB). Total 64 GB VRAM across both boxes. F16 (55 GB) fits on Box A with tight KV-cache; Q6_K (21 GB) and Q4_K_M (16 GB) are the practical daily-drivers. F16 / Q8_K benchmark runs are pending — this table will be updated when they land.

⚠️ Token & step limits: SWE-bench is the most constrained benchmark: message_limit=100 (agent turns) × max_tokens=16384/turn ≈ 1.6M output-token ceiling. The model is typically step-limited, not token-limited (agent runs out of turns before exhausting the token budget). GPQA is single-shot with max_tokens=65536, so the token ceiling is rarely hit (avg ~4K tokens/question).

Smoke test

Base model (pre-SFT) used 4000+ think-chars on the snail problem. ThinkingCap SFT brings it to 359 (-91%).