base_model: OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated library_name: gguf pipeline_tag: text-generation tags:

gguf
llama-cpp
imatrix
quantized
spark
spark-x2.6
heretic
abliterated
uncensored
decensored
reasoning
roleplay
edge-ai
q4_k_m
q5_k_m
q6_k
q8_0
f16

Spark-X2.6-1.7B-Abliterated-GGUF (iMatrix Quantizations)

This repository provides Importance Matrix (imatrix) calibrated GGUF quantizations of OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated (an uncensored variant of XHToken/Spark-X2.6-1.7B abliterated with Heretic v2.0.0.dev0).

All quantized weights were calibrated using a 2,000-sample multi-domain dataset (spark_2000_mixed_dataset.jsonl) formatted strictly in the native Spark-X2.6 ChatML template (covering uncensored instruction following, coding, mathematics, science, and roleplay) to maximize perplexity retention and reasoning quality at low bit-widths.

Available GGUF Files

File Name	Quant Type	iMatrix	Notes
`Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf`	`Q4_K_M`	Yes	Best balance of speed, low RAM/VRAM usage, and accuracy
`Spark-X2.6-1.7B-Abliterated-Q5_K_M-imatrix.gguf`	`Q5_K_M`	Yes	High-quality 5-bit quantization with minimal perplexity loss
`Spark-X2.6-1.7B-Abliterated-Q6_K-imatrix.gguf`	`Q6_K`	Yes	Near-lossless 6-bit quantization
`Spark-X2.6-1.7B-Abliterated-Q8_0-imatrix.gguf`	`Q8_0`	Yes	Extreme precision 8-bit quantization
`Spark-X2.6-1.7B-Abliterated-F16.gguf`	`F16`	N/A	Full 16-bit unquantized GGUF weights
`Spark-X2.6-1.7B-Abliterated-imatrix.dat`	`imatrix`	Yes	Precomputed Importance Matrix calibration data

Prompt Format (Spark-X2.6 ChatML)

Spark-X2.6-1.7B uses a custom prompt structure with configurable thinking effort (low or high):

<｜start▁of▁sentence｜><|System|>
you are a helpful assistant.
`thinking_effort` guides on how much to think in your thinking channel (not including the response channel), supported values include `low` and `high`.
Now the system is invoked with `thinking_effort=high`.<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|User|>{prompt}<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|Bot|><think>

How to Use

1. Using llama-cli (llama.cpp)

./llama-cli \
  -m Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf \
  -ngl 99 \
  -c 4096 \
  --temp 0.3 \
  --top-p 0.9 \
  --top-k 50 \
  --repeat-penalty 1.1 \
  -cnv

2. Using llama-server (OpenAI-Compatible Local API)

./llama-server \
  -m Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf \
  -ngl 99 \
  -c 4096 \
  --port 8080

3. Using llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated-GGUF",
    filename="Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf",
    n_gpu_layers=-1,
    n_ctx=4096
)

response = llm.create_chat_completion(
    messages=[
        {"role": "user", "content": "Explain the difference between stack memory and heap memory in 3 concise bullet points."}
    ],
    temperature=0.3,
    top_p=0.9,
    top_k=50,
    max_tokens=512
)

print(response["choices"][0]["message"]["content"])

Spark-X2.6-1.7B-Abliterated-GGUF (iMatrix Quantizations)

Available GGUF Files

File Name

Quant Type

iMatrix

Notes

Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf

Q4_K_M

Yes

Best balance of speed, low RAM/VRAM usage, and accuracy

Spark-X2.6-1.7B-Abliterated-Q5_K_M-imatrix.gguf

Q5_K_M

Yes

High-quality 5-bit quantization with minimal perplexity loss

Spark-X2.6-1.7B-Abliterated-Q6_K-imatrix.gguf

Q6_K

Yes

Near-lossless 6-bit quantization

Spark-X2.6-1.7B-Abliterated-Q8_0-imatrix.gguf

Q8_0

Yes

Extreme precision 8-bit quantization

Spark-X2.6-1.7B-Abliterated-F16.gguf

F16

N/A

Full 16-bit unquantized GGUF weights

Spark-X2.6-1.7B-Abliterated-imatrix.dat

imatrix

Yes

Precomputed Importance Matrix calibration data

Prompt Format (Spark-X2.6 ChatML)

Spark-X2.6-1.7B uses a custom prompt structure with configurable thinking effort (low or high):

<｜start▁of▁sentence｜><|System|> you are a helpful assistant. `thinking_effort` guides on how much to think in your thinking channel (not including the response channel), supported values include `low` and `high`. Now the system is invoked with `thinking_effort=high`.<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|User|>{prompt}<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|Bot|><think>

How to Use

1. Using llama-cli (llama.cpp)

./llama-cli \ -m Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf \ -ngl 99 \ -c 4096 \ --temp 0.3 \ --top-p 0.9 \ --top-k 50 \ --repeat-penalty 1.1 \ -cnv

2. Using llama-server (OpenAI-Compatible Local API)

./llama-server \ -m Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf \ -ngl 99 \ -c 4096 \ --port 8080

3. Using llama-cpp-python

from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated-GGUF", filename="Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf", n_gpu_layers=-1, n_ctx=4096 ) response = llm.create_chat_completion( messages=[ {"role": "user", "content": "Explain the difference between stack memory and heap memory in 3 concise bullet points."} ], temperature=0.3, top_p=0.9, top_k=50, max_tokens=512 ) print(response["choices"][0]["message"]["content"])