license: apache-2.0 base_model:

• XHToken/Spark-X2.6-4B base_model_relation: finetune pipeline_tag: text-generation library_name: gguf language:
• en
• zh tags:
• gguf
• llama.cpp
• abliterated
• heretic
• spark

Spark-X2.6-4B-Abliterated — GGUF

An abliterated build of XHToken/Spark-X2.6-4B (dense 4.1B, a thinking model with a low/high effort dial). It was produced with Heretic and quantized with llama.cpp. Every claim below is measured against XHToken's own Q8_0, which the same converter reproduces bit for bit.

Modification notice. These weights are a modified derivative. Directional ablation altered 53 tensors to reduce refusals (see What changed). Nothing else was retrained. XHToken did not produce, endorse, or review this build, and its behavior is not theirs.

Read this first

This is an escape hatch for over-refusal, not an uncensored model. Abliteration removes a refusal direction from the residual stream; it adds no knowledge. This build knows what stock Spark knows and nothing more. What changed is its willingness to say it. If stock answers your prompt, use stock.

With thinking on, it deliberates at length before answering. On the refusal prompts below, thinking-on answers average ~5k tokens (stock: 529), and some run out of an 8192-token budget while still thinking. A system prompt that tells it not to refuse or add disclaimers roughly halves that (see System prompt). Give it a generous max_tokens.

The hazard worth naming: refusals double as hedging. Remove them and you get a model answering fluently in domains where it is unreliable, with none of the cues that used to mark the boundary. Factual reliability is unchanged; only the signalling is gone. You are accountable for what you generate and for applicable law.

Measurements

All on Q8_0 against XHToken's published Q8_0 (sha256 145c2aa6…31d58b81). Converting the stock safetensors with the same converter reproduced that file exactly (all 290 tensors), so it serves as the stock arm. Sampling is temp 1.0 / top-p 0.95 / top-k 20, seeded per item so both builds see the same randomness. Refusals use mlabonne/harmful_behaviors test[60:], the 44 prompts Heretic's optimiser never saw (it scored test[:60]), with Heretic's default system prompt. Budgets: 1536 tokens thinking off, 8192 thinking on.

Refusals (lower = complies more). An answer counts as refused if it opens with a refusal. Heretic's marker list also fires on disclaimer wording inside answers that comply in full, and on this model it flagged 11 (thinking off) and 10 (thinking on) such answers. Both counts are shown:

• Non-degenerate. Answers don't loop: in each regime only one answer has more than 10% repeated 20-grams. A few thinking-on reasoning traces circle a little (3/44 below 90% unique 20-grams, the worst at 78%), which is part of the deliberation cost below. With thinking off, answers run ~1000 tokens (stock: 221), because the model now writes the answer it used to decline. 8 of them reached the 1536-token budget.
• Classified, not hand-graded. "Opens with a refusal" is a check on the first 300 characters. A model that refuses partway through an answer would slip past it.

GSM8K, paired, n=100, enable_thinking: false:

Discordant pairs split 5/2 (McNemar exact p = 0.45), so no math regression is detectable. Abliteration damage tends to land in math reasoning, which is why this is the check that matters. n=100 rules out a large regression, not a 2–3 pp one.

Unmeasured: MMLU, coding, long context, non-English, agentic/tool-calling loops.

System prompt

The deliberation cost responds to the system prompt. With a short no-refusal, no-disclaimer system prompt (an "unrestricted" preset from a local chat client, ~1 KB), thinking on, same 44 prompts:

The prompt alone does little for stock: with thinking off it still refused 43/44, just more tersely. The weights remove the refusal; the prompt shortens the argument about it.

Files

All quantized from one bf16 GGUF, without an imatrix (Q4_K_M and up don't require one). Only Q8_0 was evaluated. build/ holds the search record, not model files: the log (run.log), the full Optuna study journal (heretic-study.jsonl, all 100 trials), and SHA256SUMS.

Usage

The spark2_5 architecture needs llama.cpp b10828 or newer (built and tested at b11780, 1242be66). Sampling is XHToken's card values (top-k off):

llama-server -m Spark-X2.6-4B-Abliterated-Q8_0.gguf \
  -ngl 99 -c 131072 -fa on --jinja \
  --temp 1.0 --top-p 0.95 --top-k 0

Thinking is on by default at effort high. Per request, "chat_template_kwargs": {"enable_thinking": false} turns it off and {"reasoning_effort": "low"} shortens it. The template raises an error on any effort other than high or low. 27 of 36 blocks use a 512-token sliding window, so the KV cache is small (36 KiB/token f16); the card's 1M context still needs ~36 GiB of KV at f16.

What changed

Diffing the Q8_0 against XHToken's (same converter, so only the ablation can differ):

Layers 0–7 are untouched entirely, and attention in 8–10 is untouched too.

Reproducing

Stock Heretic cannot run this model under its own dependencies. Spark ships remote modeling code written for transformers 4.57, and Heretic 1.4 requires transformers 5.x. Three things break there:

• The tied-weights key list. The v4 list form of _tied_weights_keys raises in post_init.
• The mask helpers. create_causal_mask renamed input_embeds and dropped cache_position.
• Hidden states, silently. output_hidden_states is ignored: the model never returns hidden states, so Heretic would read none.

Heretic also cannot pass template kwargs, and this template thinks by default.

This build loaded a small in-process patch that fixes the three breaks and renders prompts with enable_thinking=False. The scored prompt was checked byte-identical to llama-server's rendering. The patch also passed causality, padded-vs-unpadded (KL ≤ 1.3e-4) and cached-vs-uncached generation checks before the search. The search itself is stock Heretic 1.4.0 (transformers 5.19.0, peft 0.21.2), merged straight onto the original bf16 safetensors. Run bare, the command below fails at model load:

heretic XHToken/Spark-X2.6-4B \
  --seed 1234 --max-batch-size 128 --max-response-length 64 \
  --n-trials 100 --n-startup-trials 40 \
  --good-evaluation-prompts '{"dataset":"mlabonne/harmless_alpaca","split":"test[:60]","column":"text"}' \
  --bad-evaluation-prompts '{"dataset":"mlabonne/harmful_behaviors","split":"test[:60]","column":"text"}'

Abliteration is not bit-reproducible (the TPE search lands on a different trial each run). The selected trial was 80: 2/60 refusals @ KL 0.1289, from a baseline of 59/60, with a single interpolated direction. Its parameters, as Heretic reports them:

direction_index: 15.10
attn.o_proj    : max_weight 0.83  max_weight_position 28.69  min_weight 0.79  min_weight_distance 17.81
mlp.down_proj  : max_weight 1.49  max_weight_position 22.02  min_weight 1.39  min_weight_distance 14.11

Verifying

cd1afa19048ba58b09bee406aaa6865b1d91ca3e2cd66409c779cbc45ec1fbcb  Spark-X2.6-4B-Abliterated-Q4_K_M.gguf
9e59dcdf6fa6b18cbf146f1b2f5ae47ba30fdcb21e6903b327fc726d050460a1  Spark-X2.6-4B-Abliterated-Q5_K_M.gguf
763be1b053665d77e43ed2ac522ebec1b3d34182a8b36efeb3bd21fe308a168a  Spark-X2.6-4B-Abliterated-Q6_K.gguf
152dec01dca198b1a10295582acfbae6b475d4f2b12b348fbfe407adc91a22a9  Spark-X2.6-4B-Abliterated-Q8_0.gguf

License

Apache-2.0, inherited from XHToken/Spark-X2.6-4B. The LICENSE here is the base repo's file, copied verbatim (sha256 525c465a…, identical to upstream); it is the standard Apache License 2.0 text with no added terms. No NOTICE file exists upstream, so none is propagated. As required by §4(b), this card states that the files were modified (see the notice at the top). Abliteration and quantization change nothing about the license.

Credits

• XHToken: the Spark-X2.6-4B base model
• p-e-w/heretic: the abliteration method and tooling
• ggml-org/llama.cpp: conversion and quantization
• mlabonne: the prompt sets used for directions and scoring

Supporting my work

If these quants are useful to you, consider supporting them via Ko-fi. It goes toward local hardware for quantizing and testing larger models. I also build Reika, a coding agent CLI designed around small local models first.