license: gemma pipeline_tag: text-generation base_model: HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced tags:
- gguf
- long-context
- yarn
- gemma4
- uncensored
- mtp
- speculative-decoding
- vision
- llama.cpp
- ollama
Gemma4-12B Uncensored 1.5M: the longest needle-certified context we know of
The same HauhauCS uncensored Gemma-4-12B trunk as our 1M repo, baked with YaRN factor 6 for a 1,572,864-token maximum context, and needle-certified far past the million mark. Every number below was measured on a single RTX 5090 (32GB); raw per-needle records ship in this repo.
The heatmap
10 needles per rung at depths 5 to 95 percent, temperature 0, seeded haystacks. The f16 tier is the certification; the q8 KV tier is the labeled budget config that fits the higher rungs into 32GB. At 1.57M with f16 KV the cache no longer fits a 32GB card, so that rung was run on an H200 (July 9, 2026): 6/10. Past the certified 1.31M mark the extreme edge degrades, and we report it as measured. The q8 KV budget tier at the same rung measured 9/10 on a different seed; per-needle records for both are in this repo.
Where the ceiling is, honestly
We also baked and tested a factor-8 variant (2.1M max) so nobody has to wonder:
| Factor | 1M | 1.57M | 2M |
|---|---|---|---|
| 6x (this repo) | 10/10 | 9/10 (q8 KV) | not reachable by design |
| 8x (tested, not shipped) | 9/10 | 9/10 (q8 KV) | 6/10 (q8 KV) |
Two findings worth stealing: the YaRN factor itself costs quality (8x drops a needle at 1M where 6x is clean, so bake the smallest factor that reaches your target), and 2M is where retrieval bends (6/10, degradation spread across depths). The 2M line of work is closed: 6/10 does not meet our bar, the 8x build stays unshipped, and the data is published instead. All four raw ladders are in the results jsonl files, misses included.
Next for this build: NVIDIA's RULER benchmark (multikey NIAH, variable tracking, aggregation) at lengths up to and beyond 1M. Results will be published here either way, sample counts stated.
Files
| File | Size | Role |
|---|---|---|
gemma4-12b-uncensored-1.5M-Q4.gguf | 7.4 GB | Trunk, 1.57M max context baked (Google QAT 4-bit) |
mtp-gemma-12b.gguf | 254 MB | MTP draft head, pair with -md (+51 percent decode measured on this trunk) |
mmproj-gemma12b-hauhau.gguf | 175 MB | Vision tower, pair with --mmproj |
niah_heatmap.png, results-*.jsonl | small | Verification evidence, all four ladders |
Run it
llama-server -m gemma4-12b-uncensored-1.5M-Q4.gguf \
-c 1310720 -np 1 --jinja \
-md mtp-gemma-12b.gguf --spec-type draft-mtp --spec-draft-n-max 3 \
--mmproj mmproj-gemma12b-hauhau.gguf
For the certified experience use up to -c 1310720 with default f16 KV. To reach the full 1.57M on a 32GB card add --cache-type-k q8_0 --cache-type-v q8_0 (the budget tier above, 9/10 measured). Ollama:
FROM ./gemma4-12b-uncensored-1.5M-Q4.gguf
RENDERER gemma4
PARSER gemma4
PARAMETER num_ctx 262144
Raise num_ctx as desired; Gemma's 5:1 sliding-window attention keeps the KV cache small enough that this entire certification ran inside a 32GB GPU.
How this was built
YaRN rope-scaling metadata (factor 6.0 over native 262,144) written into the GGUF header with gguf-py; weights bit-identical to the HauhauCS release, no fine-tuning. Certification harness: 10 needles per rung, depths 5 to 95 percent, temperature 0, f16 KV for certified rungs, q8 KV clearly labeled. Method and tooling: github.com/satindergrewal/aviary-1m. Sister repo with the 4x/1M build and full ladder: Gemma4-12B-Uncensored-HauhauCS-1M-GGUF.
How to actually use a 1M-context model
Habits that measurably help, from our RULER, hop and adherence testing across this fleet:
- Re-state standing instructions near the end of long prompts; recency beats depth.
- One big reference dump beats a long accumulated conversation. Fresh session per task.
- After any compaction or summarization, repeat your active rules yourself.
- Prefill at 500K+ takes real time on any hardware; stage your questions accordingly.
- Know your quant: the results tables on this card show what each quant actually holds at depth; pick the strongest one your memory allows.
Credits
Base model and QAT: Google (Gemma license; terms flow down). Uncensoring: HauhauCS. MTP head: Unsloth (via the HauhauCS repo). YaRN 1.5M extension, beyond-1M certification, and the 2M ceiling study: SatGeze.
Mirrors: Hugging Face | ModelScope