license: mit library_name: mlx pipeline_tag: text-generation base_model: apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8 base_model_relation: quantized tags:
- mlx
- omlx
- deepseek-v4
- mixture-of-experts
- abliterated
- quantized
DeepSeek-V4-Flash-0731 Abliterated MLX Mixed 2-bit/3-bit g64
Preliminary experimental release. The full paired refusal and capability evaluation is still in progress.
This is a mixed-precision MLX quantization of
apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8,
designed to run on a 128 GB Apple Silicon system. Weight files occupy about
99.46 GiB.
Quantization profile
| Tensor class | Storage |
|---|---|
| Routed expert gate/up | affine 2-bit, group size 64, importance weighted |
| Routed expert down | affine 3-bit, group size 64, importance weighted |
| Attention projections | MXFP8, group size 32 |
| Shared experts | MXFP8, group size 32 |
| Output head | MXFP8, group size 32 |
| Embeddings, routers, norms and auxiliary tensors | protected BF16/FP32 |
The importance matrix was imported from
jedisct1/DeepSeek-V4-Flash-imatrix-aligned.
The 129 routed modules have architecture-compatible dimensions, but this is an
aligned transfer from the earlier DeepSeek-V4-Flash checkpoint, not a fresh
0731-native calibration run.
This variant omits the attached speculative weights and sets
num_nextn_predict_layers=0. It is intended for standard trunk generation;
it is not a DSpark release. The preferred local runtime is
oMLX with DeepSeek-V4 support.
Validation status
- All 21 shards and 1,931 indexed tensors passed structural validation.
- The imported importance matrix was applied to all 129 routed modules.
- The full paired refusal and capability benchmark is pending.
Limitations
This is an aggressive mixed 2/3-bit quantization. Quality should be measured against both the abliterated FP8 checkpoint and the clean quantized control. Abliteration can affect behavior beyond refusals and does not guarantee a particular response.
License and attribution
The upstream DeepSeek-V4-Flash-0731 weights are MIT licensed. This repository retains that license and documents both the abliteration and quantization provenance above.