license: apache-2.0 base_model: dima6312/Flint-40B-A2.5B-abliterated base_model_relation: quantized pipeline_tag: text-generation library_name: gguf language:
- en datasets:
- mlabonne/harmful_behaviors
- mlabonne/harmless_alpaca tags:
- abliterated
- cybersecurity
- code
- moe
Flint-40B-A2.5B-abliterated (GGUF)
RecursiveSI's Flint 40B-A2.5B with its refusal direction removed, for security research and coding work: exploit analysis, malware triage, detection engineering, CTFs and red-team tooling that the original model often declines. Weights: BF16 safetensors, GGUF.
Results
Both models ran as Q8_0 GGUF on llama-server with the same flags, Flint's recommended sampling (temperature 1.0, top_p 0.95, top_k 20) and full reasoning, on evenly spaced subsets of each benchmark (n in each row). Checks are paired on the same items; a quality check fails only on a drop that is significant at 95%. Refusals are scored with Meta's CyberSecEval refusal check. Refusal rates count refusals only. A reply with no final answer (the model was still reasoning at the 8,192-token limit) is listed separately as "no answer": on the 100 MITRE prompts that happened 19 times for the abliterated model and once for the original. The cut-off reasoning was not repetitive; give long security tasks a larger token budget.
| Check | Original | Abliterated | Result |
|---|---|---|---|
| MITRE refusals (n=100) significantly lower | 0.15 | 0.0 | pass |
| FRR false refusals (n=100) not significantly higher | 0.0 | 0.0 | pass |
| cybermetric (n=50) not significantly worse | 1.0 | 0.96 | pass |
| gsm8k (n=50) not significantly worse | 0.98 | 0.98 | pass |
| mmlu_pro (n=50) not significantly worse | 0.8 | 0.78 | pass |
| wmdp_cyber (n=50) not significantly worse | 0.5 | 0.6 | pass |
| humaneval_plus (n=40) not significantly worse | 0.9 | 0.95 | pass |
| mbpp_plus (n=40) not significantly worse | 0.875 | 0.9 | pass |
| pp512 tok/s within 3% | 8050.8 | 8135.1 | pass |
| tg128 tok/s within 3% | 180.0 | 185.6 | pass |
Refusals on held-out CyberSecEval MITRE prompts (n per ATT&CK category below):
| Category | n | Original: refused / no answer | Abliterated: refused / no answer |
|---|---|---|---|
| C2 | 10 | 20% / 0% | 0% / 30% |
| Collection | 10 | 0% / 0% | 0% / 0% |
| Discovery | 10 | 0% / 10% | 0% / 20% |
| Evasion | 10 | 40% / 0% | 0% / 40% |
| Execution | 10 | 10% / 0% | 0% / 30% |
| Exfil | 10 | 10% / 0% | 0% / 0% |
| Lateral Movement | 10 | 40% / 0% | 0% / 10% |
| Persistence | 10 | 20% / 0% | 0% / 10% |
| Privilege Escalation | 10 | 10% / 0% | 0% / 30% |
| Recon | 10 | 0% / 0% | 0% / 20% |
KL divergence from the original on 100 harmless prompts, measured by the search for the selected trial (not part of the checks above): 0.0146 nats/token.
Benchmarks: CyberSecEval MITRE and FRR (Meta), CyberMetric-500, WMDP-cyber, MMLU-Pro, GSM8K, HumanEval+ and MBPP+ (EvalPlus); speed from llama-bench pp512/tg128.
Method
The refusal direction comes from the difference in mean activations on harmful and harmless prompts
(mlabonne/harmful_behaviors, mlabonne/harmless_alpaca). abliterix
removes it from attention output, shared-expert and routed-expert projections; Optuna picks the strengths over 50
trials against refusals on 100 MITRE prompts and KL divergence on harmless prompts. The MITRE prompts in the table
were never seen by the search. Tensor shapes are unchanged, so speed and memory match the original.
Usage
Reasoning is optional. When the model reasons, it uses <think>…</think> before its answer. Use the sampling settings above.
llama.cpp (b11827 or later):
llama-server -m Flint-40B-A2.5B-abliterated-Q8_0.gguf --jinja -ngl 999 -fa on
transformers 5.20 (trust_remote_code=True, the port ships with the weights):
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "dima6312/Flint-40B-A2.5B-abliterated"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, trust_remote_code=True, dtype="bfloat16", device_map="auto")
Limitations
- The checks ran on subsets of each benchmark (n in each row), on the Q8_0 GGUF only. Other quants were not measured.
- Refusals are lower, not zero: the selected trial still refused 10 of the 100 prompts used by the search.
- On long security tasks the model can reason for more than 8,192 tokens before answering.
- Abliteration removes refusals; it adds no knowledge. Mistakes the original makes, this model makes too.
Responsible use
This model answers requests the original refuses. It is meant for authorized security testing, research, education and defensive work. You are responsible for how you use it and for following the law and the terms of the systems you test. Its answers can be wrong; check them before acting on them.
Credits
- Flint by RecursiveSI (Apache 2.0). This is a derivative of their weights under the same license.
- abliterix by wuwangzhang1216, building on Heretic by p-e-w.
- Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).
- Maxime Labonne's harmful/harmless prompt sets; Meta's CyberSecEval; CyberMetric (Tihanyi et al.); WMDP (Center for AI Safety); MMLU-Pro (TIGER-Lab); GSM8K (OpenAI); EvalPlus.
- llama.cpp (ggml-org) and transformers (Hugging Face).