license: apache-2.0 base_model: dima6312/Flint-40B-A2.5B-abliterated-Rec-4bit base_model_relation: quantized pipeline_tag: text-generation language:

• en tags:
• abliterated
• uncensored
• code
• moe
• reap
• expert-pruning
• rec
• 4-bit

Flint-40B-A2.5B-abliterated REAP-224 (Rec 4-bit)

dima6312/Flint-40B-A2.5B-abliterated-Rec-4bit with 32 of the 256 routed experts removed in every MoE layer (256 → 224), chosen by REAP (router-weighted expert activation pruning) on coding and coding-agent text. The layout is unchanged (affine int4, group size 64, the same as RecursiveSI's int4 release); only the expert tensors and router rows of the 47 MoE layers are sliced.

Code and full write-up: github.com/RINNECODER/flint-research: pruning scripts, masks, routing stats, eval logs, and the Locals patches for DFlash 2.

It is about 12% smaller (19 GB instead of 21 GB). When experts are streamed from SSD it decodes faster, and it scored the same as the 256-expert model on a small HumanEval+ test.

Results

All measurements are on an M5 Pro Mac with 24 GB, using Locals with SSD expert streaming (a 6 GB expert cache) and DFlash 2 speculative decoding where marked.

• How the tok/s numbers were taken: 2 coding prompts × 2 runs, 1,024 tokens each. Where two numbers are shown, they come from separate sessions.
• KL baseline: the KL figure is not near zero. Even changing only the chunked-prefill numerics of the 256 model gives a KL of 0.42 against itself.
• Correction: re-scoring the saved evalplus results on only the 40 sampled tasks gives 36/40 for the 256-expert model, not 35/40. The table above predates that re-count. REAP-224 stays at 35/40; the one task it lost is HumanEval/48. Details are in the repo's docs/eval.md.
• Small eval: with only 40 tasks this is a sanity check, not proof that quality is unchanged. Long agentic sessions were not tested.

Deeper cuts were tried and dropped:

Applying the published RESMP RCO-224 mask to this model kept a different set of experts. On coding text it did no better than dropping 32 experts at random:

How the experts were chosen

• Calibration data: 217,606 tokens taken from codeparrot/codeparrot-clean-valid, nebius/SWE-agent-trajectories and SWE-Gym/OpenHands-SFT-Trajectories. Evaluation used different rows of the same datasets.
• Scoring: for each expert, the mean over routed tokens of gate weight × the L2 norm of the expert's output. The 224 highest-scoring experts in each layer were kept.
• What was removed: the experts dropped carried 10.1% of routing slots on the calibration text (a random 32 carry 12.6%). REAP keeps some rarely used experts that matter when they fire, so it is not the same as dropping the least-used experts (that set carries 3.5% but tracks the original model less closely: KL 0.98 vs 0.92).
• Expert ids: keep_ids.json maps the new expert indices to the original ones. keep_ids[layer - 2][j] is the original id of new expert j; layers 0–1 are dense.

Coding was the target, so other domains (non-English text, security knowledge, maths) may lose more than the coding numbers above show. Refusal behaviour comes from the abliterated base and was not re-tested after pruning.

Running it

It loads wherever the base Rec 4-bit build loads (num_experts is 224 in config.json). It was tested only with Locals:

locals start --model /path/to/this/repo

It does not load in mlx-lm, LM Studio or llama.cpp; it uses RecursiveSI's Rec int4 layout.

Credits

• Flint by RecursiveSI.
• The abliteration and Rec 4-bit build by dima6312.
• REAP by Lasby et al. (Cerebras), arXiv:2510.13999.

License: Apache-2.0, the same as the base model.