license: apache-2.0 base_model: InfinimindCreations/K2-Horizon-MoVA-36B-A4B-uncensored base_model_relation: quantized tags:
- gguf
- llama-cpp
- uncensored
- abliterated
- k2-horizon
- mova
- moe language:
- en pipeline_tag: text-generation
K2-Horizon-MoVA-36B-A4B Uncensored GGUF
GGUF quantizations of InfinimindCreations/K2-Horizon-MoVA-36B-A4B-uncensored, the rank-3 abliteration of IFM K2-Horizon-MoVA-36B-A4B. Method, leak correction, capability and negative-control results are documented there.
Files
| File | Size | Notes |
|---|---|---|
| K2-Horizon-MoVA-36B-A4B-uncensored-Q8_0.gguf | 37.1 GiB | largest quantization, not measured separately |
| K2-Horizon-MoVA-36B-A4B-uncensored-Q6_K.gguf | 28.7 GiB | not measured separately |
| K2-Horizon-MoVA-36B-A4B-uncensored-Q4_K_M.gguf | 20.8 GiB | refusal behavior re-measured, see below |
Compatibility
Upstream llama.cpp does not yet support the k2_horizon architecture. These files were produced
and tested with the
MBZUAI-IFM fork of llama.cpp,
branch model/K2Horizon, commit 35999d1. Tools built on llama.cpp (Ollama, LM Studio) can load
them only once their bundled llama.cpp includes the architecture.
llama-server -m K2-Horizon-MoVA-36B-A4B-uncensored-Q4_K_M.gguf --jinja -c 32768
Does the abliteration survive quantization?
Quantization can partly restore refusal behavior, so the smallest file was measured again rather
than assumed. Q4_K_M, greedy decoding, reasoning_effort="low", on the same leak-free validation
stages 4 and 5 and with the same rule-based counting as the full-precision release:
| Metric | Original | Abliterated, NF4 | Abliterated, Q4_K_M (this repo) |
|---|---|---|---|
| Safety refusals, harmful prompts (n = 102) | 96/102 (94.1 %) | 1/102 (1.0 %) | 1/102 (1.0 %) |
| Over-refusal, harmless prompts (n = 10) | 1/10 | 0/10 | 0/10 |
| Truncated or incoherent responses | 0/102 | 7/102 | 7/102 |
Paired against the original: 88 refuse to comply, 0 comply to refuse, exact McNemar p = 6.5e-27. The responses themselves differ from the NF4 run (none of the 112 is identical), but the counts do not: quantization to Q4_K_M did not bring refusals back on these prompts. The original and NF4 columns were run with transformers and bitsandbytes, the Q4_K_M column with llama.cpp.
Capability. Same MMLU and ARC-Challenge protocol as the full-precision release (5-shot, no chat template, answer-letter probabilities), on identical prompt tokens, paired question by question against the original:
| Benchmark | Original | Abliterated, NF4 | Abliterated, Q4_K_M (this repo) |
|---|---|---|---|
| MMLU, 285 questions | 77.9 % | 80.0 % | 77.5 % |
| ARC-Challenge, 200 questions | 91.0 % | 92.0 % | 91.5 % |
Q4_K_M against the original: MMLU -0.4 points (95 % bootstrap interval [-4.6, +3.9], exact McNemar p = 1.0), ARC-Challenge +0.5 points ([-3.0, +4.0], p = 1.0). No measurable capability loss. The probability mass on the answer letters was 1.000 for every question, so the format was understood throughout.
The larger quantizations were not measured separately. Their error relative to bfloat16 is smaller than that of Q4_K_M.
Disclaimer
This model is provided for research purposes. With refusal behavior removed, it will respond to prompts the original model declines. Users are responsible for how they use this model.
About
Built by Infinimind Creations, Germany.