license: apache-2.0 language:
- en
- zh library_name: gguf pipeline_tag: text-generation base_model: FrostWolfAI/Spark-X2.6-4B-abliterated base_model_relation: quantized tags:
- gguf
- llama.cpp
- spark
- sparkx2_5
- spark-x2.6
- on-device
- agentic
- tool-calling
- reasoning
- heretic
- ara
- uncensored
- decensored
- abliterated
- local-llm
Spark-X2.6-4B-abliterated-GGUF
GGUF quantizations of FrostWolfAI/Spark-X2.6-4B-abliterated, a decensored variant of XHToken/Spark-X2.6-4B produced with Heretic v2 using Arbitrary-Rank Ablation (ARA).
Refusals drop from 100/100 to 2/100 at a KL divergence of 0.0177. Architecture, 1M-token context,
controllable thinking and the <tool_call> format are all intact. Full details, abliteration parameters and
the reproduction bundle live on the
safetensors model card.
Who this is for: local agents, coding and security research with llama.cpp on a consumer GPU, a laptop or CPU-only. At 4-bit the weights are about 2.4-2.6 GB.
Files
Full quantization set (BF16 + 14 quants, 44.53 GB total), converted from the BF16 safetensors with
upstream llama.cpp convert_hf_to_gguf.py (master, 2026-10-29) and
quantized with llama-quantize (build 11669, 620d2225c). No importance matrix was used.
| File | Format | Size | Notes |
|---|---|---|---|
Spark-X2.6-4B-abliterated-BF16.gguf | GGUF BF16 | 8.23 GB | Lossless, reference |
Spark-X2.6-4B-abliterated-Q8_0.gguf | GGUF Q8_0 | 4.38 GB | Near-lossless |
Spark-X2.6-4B-abliterated-Q6_K.gguf | GGUF Q6_K | 3.38 GB | Recommended for quality |
Spark-X2.6-4B-abliterated-Q5_K_M.gguf | GGUF Q5_K_M | 2.98 GB | |
Spark-X2.6-4B-abliterated-Q5_K_S.gguf | GGUF Q5_K_S | 2.88 GB | |
Spark-X2.6-4B-abliterated-Q4_K_M.gguf | GGUF Q4_K_M | 2.60 GB | Recommended default |
Spark-X2.6-4B-abliterated-Q4_1.gguf | GGUF Q4_1 | 2.64 GB | |
Spark-X2.6-4B-abliterated-Q4_K_S.gguf | GGUF Q4_K_S | 2.43 GB | |
Spark-X2.6-4B-abliterated-Q4_0.gguf | GGUF Q4_0 | 2.41 GB | ARM / older runtimes |
Spark-X2.6-4B-abliterated-IQ4_XS.gguf | GGUF IQ4_XS | 2.39 GB | Fastest 4-bit on CUDA |
Spark-X2.6-4B-abliterated-Q3_K_L.gguf | GGUF Q3_K_L | 2.39 GB | |
Spark-X2.6-4B-abliterated-Q3_K_M.gguf | GGUF Q3_K_M | 2.17 GB | |
Spark-X2.6-4B-abliterated-IQ3_S.gguf | GGUF IQ3_S | 1.98 GB | |
Spark-X2.6-4B-abliterated-Q3_K_S.gguf | GGUF Q3_K_S | 1.90 GB | |
Spark-X2.6-4B-abliterated-Q2_K.gguf | GGUF Q2_K | 1.79 GB | Last resort - noticeable factual slips |
[!IMPORTANT] These files use the
spark2_5GGUF architecture. You need a llama.cpp build with Spark2.5 support (tested with build 11669); older builds will fail to load the model. Apps that bundle llama.cpp (Ollama, LM Studio, Jan, ...) work once their bundled engine includesspark2_5.
Quickstart
# llama.cpp - downloads the quant straight from this repo
llama-server -hf FrostWolfAI/Spark-X2.6-4B-abliterated-GGUF:Q4_K_M --jinja -ngl 99 -fa on -c 32768
# OpenAI-compatible request; thinking is ON by default
curl -s http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
"messages": [{"role": "user", "content": "Explain how a TLS 1.3 handshake works."}],
"temperature": 1.0, "top_p": 0.95, "top_k": 0,
"chat_template_kwargs": {"enable_thinking": false}
}'
# CLI chat
llama-cli -hf FrostWolfAI/Spark-X2.6-4B-abliterated-GGUF:Q6_K --jinja -ngl 99 -cnv
Always pass --jinja: the embedded chat template handles thinking control and tool calls.
Generation parameters
XHToken's recommended settings: temperature=1.0, top_p=0.95, top_k=0 (disabled), repetition_penalty=1.0.
Thinking
Spark-X2.6 thinks by default; llama-server returns the reasoning in reasoning_content and the answer in
content. Control it per request with chat_template_kwargs:
{"enable_thinking": false}- no reasoning, direct answer{"reasoning_effort": "low"}/"high"(default) - reasoning budget
With thinking on, give it room: short max_tokens budgets can be spent entirely inside <think>.
Tool use
Send OpenAI-style tools with --jinja; llama-server parses Spark's native
<tool_call>name<arg_key>k</arg_key><arg_value>v</arg_value></tool_call> output into standard
tool_calls with JSON arguments, ready for agent frameworks.
Tested
Each quant below was served with llama-server --jinja -ngl 99 -fa on -c 8192 and checked for a
direct answer (thinking off), a reasoned answer (thinking on: 17 x 23 = 391) and a structured tool call
(get_weather(city="Paris"), finish_reason=tool_calls):
| Quant | Thinking off | Thinking on | Tool call |
|---|---|---|---|
| Q8_0 | ✅ | ✅ | ✅ |
| Q4_K_M | ✅ | ✅ | ✅ |
| Q2_K | ⚠️ coherent, one factual slip | ✅ | ✅ |
Speed
llama-bench (-ngl 99 -fa 1 -p 512 -n 128, 3 reps) on an RTX 4090 power-limited to 360 W:
| Quant | Prefill (pp512) | Decode (tg128) |
|---|---|---|
| BF16 | ~12,300 tok/s | 99 tok/s |
| Q8_0 | ~16,100 tok/s | 165 tok/s |
| Q6_K | ~14,800 tok/s | 199 tok/s |
| Q4_K_M | ~16,800 tok/s | 240 tok/s |
| IQ4_XS | ~17,300 tok/s | 253 tok/s |
Runs anywhere
| Device | Recommended quant | Weights |
|---|---|---|
| Phone / SBC (8 GB RAM) | Q4_K_M or IQ4_XS | 2.4-2.6 GB |
| Laptop, CPU-only (16 GB) | Q4_K_M | 2.60 GB |
| Laptop / Apple Silicon (16 GB) | Q6_K or Q8_0 | 3.4-4.4 GB |
| RTX 3060 / 4060 (8-12 GB) | Q6_K / Q8_0 | 3.4-4.4 GB |
| RTX 3090 / 4090 (24 GB) | Q8_0 or BF16 | 4.4-8.2 GB |
Weights only. Three of every four layers use a 512-token sliding window, so the KV cache grows far more slowly than in a pure global-attention 4B; at the full 1M-token window it is still a server workload.
Model details
| Architecture | spark2_5 (GGUF) / Spark2_5ForCausalLM (HF) |
| Parameters | 4.11B |
| Layers | 36: 27 sliding-window attention + 9 global attention (3:1 pattern) |
| Attention | 16 heads / 4 KV heads, head dim 256, fused QKV, head-wise sigmoid output gate |
| Hidden / intermediate | 2560 / 10240, GELU MLP |
| Position embedding | RoPE; global layers theta 5e6 (partial rotary 0.25), sliding layers theta 1e4 |
| Sliding window | 512 |
| Context length | 1,048,576 |
| Vocab | 131,072 |
| Tensors | 290 (tied embeddings, no separate output tensor) |
| Languages | English, Chinese |
Responsible use
Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it should not. There is no safety filtering layered on top. You are responsible for how you deploy it - don't put it behind an unmoderated public-facing endpoint serving third parties. It inherits Spark-X2.6-4B's factual limitations; abliteration removes refusal directions, it does not add capability or judgment.
Made by FrostWolfAI - the uncensored base behind the upcoming Frost Cyber Mini 1.1.
License
Apache-2.0, inherited from XHToken/Spark-X2.6-4B.
Related
- FrostWolfAI/Spark-X2.6-4B-abliterated - BF16 safetensors, abliteration details, reproduction bundle
- XHToken/Spark-X2.6-4B - the original post-trained model
- p-e-w/heretic - the abliteration tool
- ggml-org/llama.cpp - inference engine