license: apache-2.0 base_model: Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16 base_model_relation: quantized library_name: gguf tags:

• gguf
• llama.cpp
• gsq
• rco
• iq3_xxs
• quantized
• abliterated
• uncensored
• 27b
• qwen3.8

Qwen3.8-27B Blackfrost-Abliterated · GSQ-RCO-IQ3_XXS GGUF (~9.8 GiB)

Base model Method Quant License Engine

TL;DR: First Blackfrost-lineage GSQ-RCO allocation quant. ISTA's published RCO allocation table (866 tensors) is applied 1:1 to the official Blackfrost abliterated BF16 weights — no grid training needed. Result: at 9.8 GiB it matches the 15.7 GiB daily-driver quant on behavioral batteries and is the fastest quant in our lineup on real-prompt replay (RTX 5090, llama.cpp b10930). Sized for the RTX 5090's 32 GB: weights + 256K context stay fully resident on one GPU, zero offloading.

Part of the collection Blackfrost-abliterated Qwen3.8-27B GGUF line (RTX 5090) — 9.8G speed tier · 15G fidelity tier · base weights · method sources.

中文摘要：全网首个 Blackfrost 血统的 GSQ-RCO 分配表量化档。把 ISTA 官方公布的 RCO 分配表（866 张量）直接打到官方 Blackfrost 去审查 BF16 权重上，无需任何网格训练。9.81 GiB 体积下行为级电池与本机 15.7G 主档同分，真实负载回放全场最快（138.9 tok/s @ RTX 5090），并为 5090 的 32G 显存量身定做——权重 + 256K 上下文单卡全驻留，零卸载。拒答 450 案例协议镜像实测：自动口径真拒答率 3.3%（复核上限 9.6%），75 条良性提示零拒答。所有数字均为本机实测，同 harness 同基线。

Why this exists

GSQ-RCO (ISTA / IST-DASLab) learns a per-tensor grid-format allocation under a bit budget. The allocation transfers to abliterated copies of the same architecture (same tensor shapes, same budget), so the learned table can be applied to the official Blackfrost abliterated weights with a single llama-quantize pass. Method credit goes to RentedNoodle, who first demonstrated this transfer on a Huihui-base abliterated model; this repo is the same allocation applied to the cleaner single-pass Blackfrost lineage — and the lineage difference turns out to be measurable (see benchmarks).

Built for the RTX 5090 (32 GB) — measured, not marketed

• One GPU, no offloading: 9.81 GiB of weights leaves ~22 GB of the 5090's 32 GB for KV cache — we run 256K-context sessions fully resident (202K deep-needle measured on this exact setup).
• Fastest quant we've ever measured on replay: 138.9 tok/s decode on 30 real user prompts, +19% over our 15.7 GiB mixed 4/5-bit daily driver — at one-sixth-less-the-bytes and equal HardBattery score. Decode is weight-bandwidth-bound, so the small file is the speed.
• The whole point of the size: this quant is the sub-10-GiB tier of a project that chased daily-driver behavioral quality (HardBattery 14/17, same as the 15.7 GiB main) at a size that leaves the 32 GB card headroom for long context. On smaller cards it also leaves room for big KV.
• Runs anywhere, not Blackwell-locked: the tensors are IQ/K-format (IQ3_XXS/IQ3_S/IQ2_S mix with Q4_K/Q6_K anchors) — any GPU that runs llama.cpp runs this. The RTX 5090 numbers above are simply where we measured. (Explicit note: this quant contains no NVFP4 tensors — NVFP4 is a different, Blackwell-native quant family.)

Memory by GPU tier

The 32 GB row is our bench (llama-server --ctx-size 262144, K=q8_0/V=q4_0 KV cache, parallel 1, BF16 DFlash draft). Other rows are derived from that measurement, not measured on those cards. KV grows slowly here because the Gated-DeltaNet hybrid carries full attention KV on only 16 of 64 layers.

Method (reproduce in one command)

Inputs: official Blackfrost abliterated BF16 GGUF + ISTA official imatrix + the allocation table shipped in this repo (gsq-rco-iq3xxs-allocation-table.txt, one tensor_name=ggml_type per line).

llama-quantize \
  --imatrix <ista-official.imatrix> \
  --tensor-type-file gsq-rco-iq3xxs-allocation-table.txt \
  <blackfrost-abliterated-bf16.gguf> \
  <output.gguf>

Benchmarks

All numbers measured locally on RTX 5090 32GB / llama.cpp b10930, same harness, same golden baseline, temp 0. Nothing is quoted from other people's cards.

Protocol notes:

• KLD is against saved BF16 golden logits (f16 KV, identical corpus/chunking for all quants). Treat it as a weight-fidelity indicator, not a quality ranking — within the 0.02–0.08 band our behavioral tests never separated quants; 0.1137 is outside that band and does reflect a real 3-bit cost.
• Easy battery: knowledge quiz + open-ended generation (auto-judged) + IFEval-style instruction following. HardBattery: math with brute-force verified ground truth + sandbox-executed coding + hard logic — the only battery that separated model generations in our setup.
• 202K needle: deep-context retrieval needle at 202K context.
• Replay: 30 real user prompts, paired across quants, temp 0 / 400 output tokens, with the same BF16 speculative draft model (decode tok/s and draft acceptance rate). This is the only speed comparison we trust; synthetic benchmarks lied to us before (+11% real-world delta measured as "4% noise" synthetically).
• vs the same-allocation RentedNoodle card: 69% lower KLD, better open-ended generation, +12% needle accuracy, higher draft acceptance, same speed — single-pass clean abliteration quantizes measurably better than multi-layer crude abliteration.

Perplexity (wiki.test.raw)

Measured with llama-perplexity (llama.cpp b10930, full GPU offload, f16 KV) on the official wiki.test.raw (ggml-org/ci). Every number below is ours, same harness, run back-to-back.

How to read this:

• Within the Blackfrost lineage (rows 1–3, same base weights): v16 pays +4.1% relative PPL over the daily driver — almost exactly what a standard uniform IQ3_XXS pays on the aligned base in bartowski's public measurements (+4.7% vs bf16, same dataset/tool). Non-uniform allocation doesn't beat PPL; it beats behavioral metrics at this size (see the batteries above).
• Cross-lineage PPL mostly measures the base model, not the quant (row 4 is a different abliterated fine-tune): Huihui-lineage weights simply read lower wiki perplexity. We hold behavioral quality to a stricter bar — our open-generation battery and refusal protocol are published above; lineage technical-hallucination differences are exactly what those catch and PPL cannot.
• The Blackfrost BF16 base's own PPL is not quoted here (we keep the BF16 golden for logit-KLD, not text PPL), so the quant-only share of these absolutes is not decomposable from this table alone.

Refusal bench — 450-case Blackfrost-protocol mirror

Same 450-case protocol Blackfrost uses for its release cards (AdvBench 150 + StrongREJECT 150 + XSTest 150 = 375 unsafe / 75 benign), clean serving template, on this exact quant + engine combo. Automated true-refusal classifier (refusal markers without substantive payload = refusal; disclaimer-then-substance = comply), mirroring the rubric.

Caveat, stated plainly: v16's figure is the automated classifier only — the 43-item human adjudication pass (the step that took v4 from 0% to 6.0%) is still pending, and 9.6% is the ceiling if every review item flips to refusal. Even at that ceiling it stays far below the official aligned card's 17.8%. De-refusal survives the 3-bit quantization.

Known limits

• 3-bit fidelity cost is real: KLD 0.1137 vs 0.0387 for our 14.95G mixed quant. Pick by VRAM budget.
• Long-context (202K) decode is still behind the bigger quants (63.2 vs 70.1 tok/s).
• A 450-case refusal-bench protocol has been run on this exact quant+engine combo: 3.3% automated true-refusal (43/450 total review items = 9.6% worst-case bound, human adjudication pending), vs 17.8% self-reported on the official aligned card. Automated numbers only so far — treat the adjudicated figure as TBD.

Usage

Standard GGUF — any recent llama.cpp build:

llama-server -m <this-model>.gguf --jinja -ngl 999 -c 65536

The chat template keeps the chain-of-thought channel, so keep API max_tokens ≥ 600.

Files

Credits & license

• Base weights: Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16
• Method: IST-DASLab GSQ + RCO (allocation table & imatrix); RentedNoodle for the allocation-transfer recipe
• License inherits the base model (Apache-2.0 per Qwen lineage) — check the base repo for the authoritative terms

Disclaimer

This is an abliterated (refusal-reduced) model intended for local research and personal use. It will comply with harmful requests more readily than the base alignment intends. You are responsible for what you do with it.