license: apache-2.0 base_model:

• XHToken/Spark-X2.6-4B pipeline_tag: text-generation library_name: gguf tags:
• gguf
• llama.cpp
• abliterated
• spark
• sharp-template language:
• en
• zh

Spark-X2.6-4B-Abliterated — Sharp template baked in (GGUF)

The Q5_K_M GGUF from alexwkleung/Spark-X2.6-4B-Abliterated-GGUF with the peculiar-ragdoll Sharp-Spark chat template baked into the file as tokenizer.chat_template, so runtimes pick it up without --chat-template-file.

This is a metadata-only edit. Weights are byte-identical to the source Q5_K_M — the tensor-data region of both files hashes to 57ce12c4…95f15e5. Exactly three GGUF keys differ from the source file:

Everything else was copied verbatim with llama.cpp's gguf_new_metadata.py at c7fc509.

The model

Spark-X2.6-4B (XHToken): dense 4.1B thinking model, spark2_5 architecture — hybrid attention (27 of 36 blocks use a 512-token sliding window, so the KV cache grows with only 9 full-attention layers), 16 Q / 4 KV heads of 256 dims, 131072 vocab. The abliterated build removed a refusal direction from 53 tensors with Heretic; see the source card for the method and full measurements.

Notice. The weights are a modified derivative: abliteration removes refusals, not the model's knowledge or its failure modes — you are accountable for what you generate and for applicable law. XHToken did not produce, endorse, or review this build. Factual reliability is unchanged; only the signalling is gone.

The template

Sharp-Spark v1.0.0 — Spark's own control tokens and <think> conventions preserved, with four fixes to the stock render (caller's system prompt replaces rather than trails the default, the tools block is separated and ordered, reasoning history is read from reasoning/thinking as well as reasoning_content, tool results render like every other role) plus the force-appended Sharp terseness prompt. Template kwargs:

• enable_thinking (default true) — false pre-emits </think>
• terse (default true) — the force-appended terseness system prompt
• suppress_tool_instructions (default: auto-detect)

Usage

The spark2_5 architecture needs llama.cpp b10828+ (built and tested here at c7fc509, 2026-10-21):

llama-server -m Spark-X2.6-4B-Abliterated-Q5_K_M.gguf \
  -ngl 99 -c 131072 -fa on --jinja --cache-prompt \
  --temp 0.6 --top-p 0.95 --top-k 20

Recommended serving defaults: temp 0.6 / top-p 0.95 / top-k 20 (the source card's evaluations used XHToken's 1.0 / 0.95 / off — match that if you want to reproduce their numbers).

No --chat-template-file needed — the template ships in the GGUF. Per-request kwargs:

{"chat_template_kwargs": {"enable_thinking": false, "terse": false}}

Pre-fill KV cache after response

Keep llama-server's prompt caching on (--cache-prompt, the default): when a response finishes, the conversation stays encoded in the slot's KV cache, so the next turn only processes the new tokens instead of re-encoding the whole prompt. On this model only 9 of 36 layers grow with context, so cached multi-turn chat is cheap.

Stateless clients that re-send the full conversation every turn can pre-fill while the user reads the response: after receiving it, re-submit the conversation with n_predict: 0 (OpenAI-compatible endpoint: "max_tokens": 0) — the prompt is evaluated into the cache and no tokens are generated, so the real follow-up request starts generating immediately.

KV cache quantization works well on this arch (only 9 full-attention layers grow with context):

  -ctk q8_0 -ctv q8_0   # ~2.5 GiB at 131k ctx
  -ctk q8_0 -ctv q4_0   # ~1.9 GiB at 131k ctx

Files

Verification

source Q5_K_M sha256:       9e59dcdf6fa6b18cbf146f1b2f5ae47ba30fdcb21e6903b327fc726d050460a1
template sha256:            4b17e14468e458809f5b24dfbfb5e84d26249d115b1a70c68d547e44108f0dd8
tensor-data sha256 (both):  57ce12c4c918364829e39a5765f884a1d34cf9d741caeb47f457ead9795f15e5

Credits

• XHToken — Spark-X2.6-4B
• alexwkleung — abliteration and quants (source card has the method and evals)
• peculiar-ragdoll — the Sharp-Spark template
• ggml-org/llama.cpp — quantization and tooling

License

Apache-2.0, inherited from XHToken/Spark-X2.6-4B through the abliterated build. Per §4(b): this distribution modifies the source GGUF's chat-template metadata only, as stated above; the weights are an unmodified copy of the abliterated build, which is itself a modified derivative of the base model.