library_name: mlx pipeline_tag: text-generation license: mit base_model:

• deepseek-ai/DeepSeek-V4.1-Flash-1017
• pqhaz/DeepSeek-V4.1-Flash-1017-heretic tags:
• mlx
• omlx
• mlx-serve
• deepseek-v41
• apple-silicon
• safetensors
• quantized
• abliterated

UPLOADING — INCOMPLETE CHECKPOINT. DO NOT DOWNLOAD OR START IT YET. The 53 weight shards are still transferring. Wait for PUBLICATION.json and release tag v1-oQ2.5e-mtp, which are created only after all remote shard sizes and SHA256 values are verified. The recipes below are for the completed release.

DeepSeek V4.1 Flash 1017 — Abliterated MLX

A baked, mixed-precision oQ2.5e checkpoint for Apple Silicon, served with oMLX or mlx-serve. The writer modifications are already in the weights: no runtime steering adapter is needed.

Start with oMLX 0.7.2 on a 256 GiB Mac. That is the main serving path used for this release. Keep the Engram tables on a fast local SSD; do not try to load the entire package into unified memory. mlx-serve is an alternative with a more limited validation history; see below.

Quick start: oMLX (recommended)

1. Install the tested runtime

Use macOS 15 or newer on Apple Silicon, Python 3.12 and Git (normally provided by Apple Command Line Tools). The official wheel includes native kernels, but installation also fetches pinned engine dependencies from Git repositories; it is not an offline, self-contained wheel. Some dependency ranges remain flexible, so this is not a complete environment lock. Do not arbitrarily upgrade MLX beyond the engine requirements.

git --version   # Install Git / Apple Command Line Tools first if missing.
python3.12 -m venv "$HOME/.venvs/omlx-ds41"
source "$HOME/.venvs/omlx-ds41/bin/activate"
python -m pip install --upgrade pip
python -m pip install huggingface_hub \
  "https://github.com/jundot/omlx/releases/download/v0.7.2/omlx-0.7.2-cp312-cp312-macosx_15_0_universal2.whl#sha256=5485347ed66ee64982d4f5efa519585d5bfa565c93f4a2abf8a6d99f5299593d"
python -m pip check

Alternatively use the official oMLX macOS app, which ships precompiled kernels. The Python helper below must run inside the Python environment containing oMLX; an app CLI shim by itself does not install the Python package into an unrelated interpreter.

2. Download the complete model

Allow space for the 342.7 GB package, plus your chosen SSD prefix-cache budget. Downloading only the small config or only the non-Engram shards is not sufficient.

MODEL_ROOT="$HOME/models/DeepSeek-V4.1-Flash-1017-Abliterated-MLX"
hf download apetersson/DeepSeek-V4.1-Flash-1017-Abliterated-MLX \
  --local-dir "$MODEL_ROOT"

# Optional full integrity check: reads all 342.7 GB once.
(cd "$MODEL_ROOT/oQ2.5e-mtp" && shasum -a 256 -c SHA256SUMS)

For reproducibility, add --revision <commit-sha> to hf download and record the chosen commit. SHA256SUMS binds the exact published variant files.

3. Start the server

Stop any other copy of this large model first. The serving helper creates isolated oMLX state, enables Engram SSD offload and native adaptive MTP, and enables the reusable SSD prefix cache. It binds to loopback only.

python "$MODEL_ROOT/serve_omlx.py" \
  --model-dir "$MODEL_ROOT/oQ2.5e-mtp" \
  --state-dir "$HOME/.omlx-ds41-public" \
  --cache-dir "$HOME/Library/Caches/omlx/ds41" \
  --context 32768 --concurrency 1 --port 8000

Choose an external SSD cache directory if preferred. Do not use --no-cache for a growing chat: it disables the reusable oMLX prefix cache and can make every turn reprocess the entire history. The recipe uses a 100 GiB disk-cache ceiling and no extra RAM hot cache.

The helper's 256 GiB profile retains the tested 212 GiB native MLX memory/wired limits and 220 GiB application memory guard. These are limits, not memory preallocation or a guarantee that every context fits. It does not change macOS wired-memory sysctls or install an external RAM watchdog. Keep OS/application headroom; stop if memory pressure becomes unsafe. See the detailed recipe for state handling and operating notes.

4. Connect an OpenAI-compatible client

Base URL: http://127.0.0.1:8000/v1. Model ID: deepseek-v41-abliterated.

curl --fail-with-body http://127.0.0.1:8000/v1/models
curl --fail-with-body http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"deepseek-v41-abliterated","messages":[{"role":"user","content":"2+3? Reply only the integer."}],"temperature":0,"max_tokens":32,"stream":false,"chat_template_kwargs":{"enable_thinking":false}}'

Expected arithmetic answer: 5. Thinking is off in this recipe. The default output budget is 2,048 tokens, separate from the total context ceiling; clients can request a different max_tokens, subject to remaining context and runtime/memory limits. The first request includes loading the weights, often around 90 seconds on the release host; set a suitable client timeout. Do not mistake cold-load latency for decode speed. SDKs that require an API-key field can use a nonsecret placeholder such as local for this keyless loopback setup; if you enable authentication, use your configured key instead.

Longer chats, caching and parallel requests

• Context: the downloaded architecture declares a larger native window, but this serving recipe deliberately starts at 32K. Set --context 262144 only with sufficient memory headroom. A configured ceiling is not a full-window stress-test result.
• Cache: SSD prefix reuse was verified on this checkpoint. In a growing-prompt test, 8,192 of 8,588 tokens were reused, reducing request wall time from 14.62 s to 1.45 s. An exact short repeat reused 787/789 tokens. A changed short prompt missed; not every common text prefix is reusable. The observed block size was 8,192 tokens.
• Concurrency: --concurrency 2 enables two active oMLX request slots, chunked prefill and decode fairness in one shared model instance. The installed adapter supports multi-request MTP with independent per-request verification. Two full 256K contexts and their throughput remain unqualified; do not expect twice the single-stream speed.
• llama-swap: if routing through llama-swap, its model concurrencyLimit counts admitted HTTP requests, including queued work. Use 4 router admissions with 2 oMLX active slots to leave room for a main chat and background work. More than four simultaneous admissions can still return HTTP 429; honor Retry-After and use bounded retry/backoff. This does not load four copies of the weights.
• Memory: the two-request/256K planning estimate was approximately 215–220 GiB for oMLX, plus 25–35 GiB for OS/apps, not a measured peak. Existing memory guards can throttle prefill or reject a request. Do not disable them or blindly raise system limits to force admission.
• Speed: see the reproducible short-request observations below. They are not guaranteed per-thread rates or evidence of full-context throughput; cold-load and prefill time are separate. A different novel-prompt trial was slower.

Measured short-request reference

The exact prompt is in the detailed recipe. These measurements describe this host and workload, not all Apple Silicon or every prompt.

mlx-serve alternative

See mlx-serve instructions and exact feature limits. This is the native MLX backend, not GGUF, EXL3 or the separate mlx-stream expert repack. Use a release that explicitly supports DeepSeek V4.1 and this mixed oQ layout; a generic MLX loader is not enough. Do not assume oMLX's SSD prefix cache, MTP scheduling or memory guards carry over to another engine.

What was baked and quantized

• Base: deepseek-ai/DeepSeek-V4.1-Flash-1017, revision 1bae0e1088993b9b82de08f59de5be5a34e42170.
• Writer patch: pqhaz/DeepSeek-V4.1-Flash-1017-heretic, revision 12d58ad0878b5008e80d720251adb969a0069412. Credit to DeepSeek, pqhaz and the Heretic method; attribution is not endorsement.
• 38 writer tensors were replaced before final quantization. This is a real baked checkpoint, not a base-model alias or runtime adapter.
• Fresh calibration processed 1,024 × 512-token samples using an explicitly approximate calibration-only proxy. Final weights were read from the native baked master, not exported from the proxy.
• 9 of 46,080 expert/projection positions were unvisited at the stock calibration maximum; source precision was retained for them. This is not complete calibration coverage.
• Quantization is mixed: affine 2/3-bit and 8-bit group-64 tensors, native MXFP4/MXFP8 group-32 tensors, and required affine 4-bit group-32 Engram tables. oQ2.5e is a recipe name, not a claim of uniform bits per parameter.
• The package accounts for 763,205,315,794 logical tensor parameters, including retained vision/MTP and large embedding tables. This is not the number of dense parameters evaluated for each token.

See source provenance, build provenance and quantization report. Public metadata copies replace private calibration-machine paths with provenance labels; model weights, tokenizer and runtime geometry are unchanged. The public checksum manifest reflects those metadata-only changes.

Validation scope and limitations

The exact checkpoint has served real text requests under stock oMLX 0.7.2 with Engram SSD offload and adaptive MTP. Earlier paired tests with the unchanged base under stock oMLX v0.7.1rc1 / 7a5a3d75ce7463300bdfa5fedccd34348f60ff46 passed repeated arithmetic, translation, short code and label retrieval at 2,841 prompt tokens. Narrow thinking, tool-call/tool-result and red/blue image probes also passed; these are not broad capability-preservation tests.

The abliteration label describes the patch lineage. There is no comprehensive refusal-rate, safety or general-quality benchmark in this release, and no proof of bit-identical MTP-on/off output. It can generate inaccurate, harmful or unsafe content. Treat outputs accordingly. Retained vision/MTP tensors do not imply arbitrary client support or feature parity across engines.

The MIT notice is preserved in LICENSE. No separate Native-Engram overlay is required for this package. No smaller-memory tier, GGUF conversion, universal client RAM badge or full two-user 256K guarantee is claimed.