license: mit base_model: pqhaz/DeepSeek-V4.1-Flash-1017-heretic pipeline_tag: image-text-to-text language:

• en
• zh tags:
• deepseek
• deepseek-v4.1
• uncensored
• abliterated
• heretic
• reap
• expert-pruning
• moe
• mxfp4
• fp8
• 4-bit
• a100
• cmp-170hx
• rtx-pro-6000

DeepSeek V4.1 Flash 1017 · Uncensored · REAP-256 · 4-bit

DeepSeek V4.1 Flash 1017 (uncensored "heretic" build) with 256 of its 384 routed experts kept in every layer, so it fits four 64 GB GPUs plus 256 GB of system RAM. No weight was re-quantized: the routed experts stay DeepSeek's own 4-bit MXFP4, the dense layers FP8, and every kept byte is identical to the base.

Every number on this card is labelled. Measured means we ran it. Forecast means it is calculated, never run. All CMP 170HX speeds are forecasts.

Scorecard

Why this exists

A small business owner wanted a frontier-class uncensored AI running at home.

• The long-term box is 4 x RTX PRO 6000, which runs the full uncut model.
• The bridge, on a budget, is 4 x NVIDIA CMP 170HX: ex-mining cards from the A100 family, unlocked to 64 GB each, PCIe x16-modded and water-cooled. That is 256 GB of VRAM. It is a temporary solution.

The challenge: fit DeepSeek's newest model into 256 GB at 4-bit without making it dumber or re-censoring it. The full uncensored model is 510 GB, and its routed experts are already 4-bit, so squeezing bits further was off the table. Instead, a third of the experts were removed with REAP, and everything kept was left byte for byte as DeepSeek made it.

The question behind it: is 256 GB worth it over 128 GB? A common view is that the big models only make sense at full size (400-500 GB), and that cutting one down to 256 GB loses so much that you may as well stay at 128 GB with a model built for that size. This project tested that directly. The best 128 GB-class model available, Qwen 3.8 Flash Next Uncensored (97.5 GB at 4-bit), was run against this 256 GB cut on identical questions with identical grading. For this model the answer was yes, 256 GB is worth it:

• Better: reasoning (+4.3), following instructions (+10.3), competition maths (+12 to +22) and a blind judge's verdict on open-ended answers (111 wins to 65).
• Probably better: coding, +8 (81 vs 73), but at the edge of the error bars on 100 problems.
• Tied: tool calling, long documents, web research and real business tasks.
• Worse: images and charts (-6.4).
• What the cut gave up against the full 510 GB model: graduate-level science and languages other than English and Chinese (details below). For those, full size still matters.

What we learned:

• Choosing experts by watching the model's own answers beat generic calibration text. An earlier cut ranked experts on generic text, code and chat. It lost business arithmetic (90.6% vs the uncut model's 99%) and brought back refusals (63 vs 45 flagged of 100). This cut added the uncut model's own answers in 9 domains to the ranking: business arithmetic 100%, refusals 52 (within noise of the uncut model). Measured on the same dev sets.
• Pruning costs show up where the calibration was thin: graduate-level science and languages other than English and Chinese (below).
• Speed came from many small, verified steps, each pre-registered with a pass bar before it ran. Several promising ideas failed on real traffic, and we say which.
• We made mistakes; they are listed near the end.

How it was built: the engineering (cutting, testing, speed work, recipes and this card) was done by AI coding agents, Anthropic's Claude running in Claude Code. The owner set the goals, made the calls and paid for the hardware.

At a glance

Download

This repo holds 46 of the 48 weight shards. The last two are the Engram lookup tables (203.1 GB). They are DeepSeek's own 1017 tables, untouched by uncensoring or pruning, so they are not duplicated here. Fetch them from DeepSeek's official repo into the same folder:

hf download BUCKHU/DeepSeek-V4.1-Flash-1017-Uncensored-REAP256-4bit --local-dir ./ds1017-reap256
hf download deepseek-ai/DeepSeek-V4.1-Flash-1017 \
  model-00047-of-00048.safetensors model-00048-of-00048.safetensors \
  --revision 1bae0e1088993b9b82de08f59de5be5a34e42170 --local-dir ./ds1017-reap256
cd ds1017-reap256 && sha256sum -c SHA256SUMS

The Engram shards must have these sha256 values:

76a05106ece259be83d1bfea0dcb54f23377864ee60fd321690f35e6698a2af5  model-00047-of-00048.safetensors
9dc51433e195bc78b5ad846b68643cc0351d3b174e5e4309222d3b7402c83afd  model-00048-of-00048.safetensors

The same bytes are in pqhaz/DeepSeek-V4.1-Flash-1017-heretic and dealignai/DeepSeek-V4.1-Flash-1017-UNCENSORED-FP8. Do not use dealignai/DeepSeek-V4.1-Flash-UNCENSORED-FP8 (the older, non-1017 release). Its tables differ, and the model loads and runs with them while being quietly worse. recipes/fetch-engram.sh does the download and the check.

Results

All quality tests were run on 17-18 Oct 2026 on rented 4 x RTX PRO 6000 (SGLang), with the same frozen items for every model, each maker's own sampling, thinking at the middle of each model's scale (DeepSeek effort 75, Qwen "medium"), a 12,000-token thinking cap and a 16k answer cap. Paired 95% ranges cover item choice only (one sample per item). Aggregates are in results/.

Against the uncut model (what pruning cost)

Measured, same harness. "Uncut" = the heretic 1017 base with all 384 experts.

Against Qwen 3.8 Flash Next (the model it replaces)

Qwen3.8-Flash-Next-Uncensored at IQ4_XS (97.5 GB GGUF), llama.cpp. Measured, paired on identical items.

The business tasks and web research were machine-graded for this cut (the earlier cut was also hand-checked; the grader held up). About 26 of the cut's Arena-Hard losses were replies in the wrong language.

Weaknesses, plainly

• Graduate-level science. GPQA Diamond fell from 90.4 (uncut) to 78.8. Most misses are guesses after the thinking cap: the cut hit the 12k cap on 43% of GPQA items (the official model on 17%), and was right 95.5% of the time when it finished thinking. Give it a 24k thinking budget for hard science: GPQA rose to 84.3 (measured), level with Qwen (86.9 at 24k, p 0.40).
• It thinks longer than the full model. GPQA median answer length 7,875 tokens vs 2,163 for the official model. On ordinary tasks pruning, not uncensoring, about doubled the thinking of our earlier cut; this cut thinks about 10% less than that one.
• Languages: the language bug. Prompts in Russian (0/14), Vietnamese (0/9), Japanese (0/3) and Korean (0/1) were answered in another language: Spanish, French, Chinese or English. The uncut model got 42 of 44 right. English and Chinese are fine (Chinese 15/17, the same as the uncut model). Cause: almost no text in those languages in the calibration data, so their experts were cut. Fixing it needs a new cut.
• Images. 79.5 vs Qwen's 86.0. Most of that gap is the base model: the uncut 1017 scored 80.0-81.7 on our vision dev set where Qwen scored about 86-87.
• Some refusals came back. Heretic's keyword counter flagged 52 of 100 harmful prompts for this cut, 45 for the uncut heretic model (difference within noise, p 0.21) and 99 for the official censored model. The counter also flags disclaimers, so these are upper bounds.

Speed

How the numbers were made

The CMP box was not available, so the CMP 170HX was rehearsed on rented 4 x A100 80 GB: each card limited to 64 GiB, and every tensor-parallel transfer slowed to the CMP's link speed (6.7 GB/s with the x16 mod, 1.7 GB/s stock). The A100 numbers are measured. CMP = A100 / 1.4 (range / 1.2 to / 1.55), from the A100 vs CMP compute and memory bandwidth: a forecast, not a measurement. Engine: the vLLM SM80 backport with our patches.

Before and after the A100 speed program (x16-mod links, 6.7 GB/s)

• Stock links (1.7 GB/s), final stack: two 500K chats 79.3 tok/s each (A100 measured; CMP forecast about 57), one 500K chat 112.0 (CMP forecast about 80).
• Time to first word, A100 measured: a cold 500K-token chat 282.5 s. Two cold 500K chats 283 s and 564 s (the second waits for the first). Two cold 1M chats 14.7 min and 29.3 min. With stock links the cold times double (561 s and 1,121 s for two 500K chats). CMP not measured: reading a prompt is compute-bound, so the / 1.4 factor does not apply, and the CMP has less compute.
• Follow-up latency, A100 measured: a new question in an already-read chat starts answering in 2.4-4.6 s at 500K and 4.8 / 7.8 s at 1M (the prefix is cached).
• Per-chat tok/s swings 10-15% between boots with the draft acceptance; the time per decode step is the stable number (about 1%).

Context capacity

The shipped setting (max_model_len 1,048,576, max_num_seqs 2, VLLM_PREFILL_BUFFER_MULT=4) gives a KV pool of 3,669,475 tokens on four 64 GiB cards (A100 measured). Two ~1M-token chats ran at once: 0 preemptions, both hidden "needles" found, 110.7 and 95.5 tok/s while both generated. Before the memory fix the pool at that setting was 1,350,349 tokens and only one 1M chat fit. The fix: upstream reserves a prefill workspace of 40 x max_model_len (5.5 GB per card at 1M), sized for Hopper; 4 x is enough while at most 4 chats run.

How to run it

Full recipes in recipes/; patches in patches/.

The A100/CMP server in one line (after bash patches/vllm-sm80/apply.sh <vllm package dir>):

MODEL_DIR=./ds1017-reap256 PORT=8000 GMU=0.92 bash recipes/serve-vllm-sm80.sh   # GMU: see the recipe for your card

Requests: thinking is on at effort 75 (1017's default; the backport would otherwise use 50). Send "thinking_token_budget": 12000 per request (24000 for hard science). DeepSeek recommends temperature 1.0 and top_p 1.0.

curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "local", "temperature": 1.0, "top_p": 1.0, "thinking_token_budget": 12000,
  "messages": [{"role": "user", "content": "Explain the CAP theorem in three sentences."}]}'

CMP 170HX notes (details and links in recipes/cmp170hx.md): unlock with cmpunlocker (8 GB Hynix cards -> 64 GB); check every card at every boot (64 GB visible, unlocked tensor rate, link width, no power-brake slowdown); run memtest_vulkan at 64 GB for at least 10 minutes per card and accept 0 errors, because there is no ECC and errors corrupt output silently; the x16 capacitor mod was worth +29% prose / +17% code for one chat in the rehearsal but has caused memory errors on some cards, so re-test after it; the cards have no fans and need forced air or water; cap power at 140-180 W; 256 GB of host RAM minimum.

What we tried, and why it stops here

Adopted (each passed a pre-registered A/B on step time plus quality gates)

• Memory: exact-size pinning of the Engram tables (a 256 GB host fits); expert padding 128 -> 64 (-5 GiB per card); smaller prefill and RoPE buffers; the prefill-buffer fix above. Together: 262K -> 1M context, and two 1M chats.
• Indexer decode-sharding (built into the backport, off by default): two 500K chats +18.5% per chat.
• Hyper-connection merge (one kernel where DSv4.1 ran three at every sublayer boundary): step time -3.2%.
• Bundle v2 (Triton HC-pre kernel, router GEMV at 2-8 tokens, Triton SwiGLU-clamp, wo_a kept in FP8 at decode): step time -4.2%. The four microbenchmark savings (about 0.8 ms) add up to the measured 0.79 ms.

Rejected or exhausted

• One-collective draft selection: 100% agreement on synthetic draft logits, but real draft acceptance fell 22%.
• Candidate-only indexer (third-party SM80 overlay): one chat -4.9%; with decode-sharding the extra gain was inside noise.
• Hand-written FP8 GEMM kernels: three attempts, best 0.92x Marlin's time. Route exhausted.
• Retraining the DSpark draft on the cut: +0.27% acceptance against a +4% bar. The released draft already fits; the cut is simply harder to predict.
• Draft lengths 3, 4 and 6 (6 is illegal), adaptive verification (unsupported on SM80), CUDA-graph and scheduling variants, atomic-add, local-argmax verify, fixed splits, removing the five small draft collectives: no gain or below the 3% bar. Turning off the shared-expert stream or serving FP8 layers from a BF16 copy: 5% slower.
• Full list with numbers: results/speed_techniques.csv.

What is left, and why it is hard

Final profile, one short chat, A100 with emulated links: 19.4 ms per decode step against a physics floor (bytes that must be read, plus link time) of about 8-9.6 ms.

• Link traffic, about 5 ms per step. 88 tensor-parallel all-reduces per step are structural (two per layer), each about 44 us at 6.7 GB/s. Only faster links (working peer-to-peer) can cut this.
• FP8 dense GEMMs, about 4 ms. The FP8 kernel route is exhausted on SM80 (above).
• FP4 experts, about 3.6 ms: already near their memory floor.
• About 2,000 small kernels per step: 1.6-1.85 ms of launch gaps plus about 1.6 ms of small copy and elementwise kernels. SM80 has no programmatic dependent launch, so only fusing kernels helps.
• The hardware cannot run the faster kernels this model uses on newer GPUs (FlashMLA sparse FP8, DeepGEMM, FP8/FP4 tensor-core MoE, NCCL FP8): they need SM90 or later.
• The software cannot: tree verification (not in the SM80 backends), longer drafts (block of 5; 10 is legal but loses), vLLM's custom all-reduce (fails CUDA-graph capture; refuses more than 2 PCIe GPUs).
• The open item: torch.compile. The model runs uncompiled in this backport. Fusing the small kernels is estimated at about 1 ms per step (about 5%), but it needs every custom kernel registered for tracing first. Deferred to real CMP hardware, where it should be measured anyway (the CMP's FP32 FMA rate is very low).

Method lessons

• Pre-register. Every pass bar was written down, with a timestamp, before the result it judged. Rulings made after a result are marked as such.
• Judge on step time, not tokens/s. Tokens/s moves about 3.5% run to run and moves with draft acceptance, which moves with whatever text the model happens to write. Time per decode step from the server's own counters is stable to about 1%. At least 20 requests per arm, arms alternated across boots, 3% beyond 3 standard deviations to adopt.
• Check draft acceptance on real traffic. The one-collective draft selection agreed 100% with the exact method on synthetic logits and still cut real acceptance by 22%. Real draft logits are not peaked enough.
• Gate every speed change on quality. Mean KL to a reference over 31 prompts, measured over two boots (it moves about 0.01 between identical boots; one prompt that swings 0.09-0.66 was excluded in advance), plus a 120-item coding and instruction-following check: 75/120 on the final stack vs 75/120 for the reference setup.
• Measure the uncut model on the same harness before blaming the cut. The vision gap and the long thinking turned out to be base model and pruning respectively, not uncensoring.

Mistakes along the way

• The first calibration set contained no thinking text and its long-document rows had no answers in them. Found in review; the long rows were rebuilt with answers, and thinking came in with the model's own answers in this cut. The same review found the image ranking kept image experts poorly; a photo-focused re-cut only tied, so images remain a weakness.
• Graders had bugs: a regex rubric missed correct business answers (mostly Qwen's) and a web-research grader was too strict. A blind hand review corrected both; an early "DeepSeek wins 5 of 8 business tasks" became 2.
• Our first refusal tests could not tell censored from uncensored: the censored official model answered them all.
• A crashed decision script deleted the winning cut and its base on the rented GPU server. It was rebuilt from the saved expert list and verified identical (KL 0.0 on 20 prompts).
• One speed spot check ran without the thinking cap and two draft-length tests used mismatched CUDA-graph sizes; all three were marked invalid and re-run.
• The first local copy's config.json named the wrong base repo. Following it would have fetched the wrong Engram tables. Corrected in this release.
• An early claim that this cut "keeps more of its original than Qwen's 4-bit keeps of its own" did not survive error bars and was withdrawn.

Limitations

• CMP numbers are forecasts. The A100 rehearsal emulates memory and link speed, not the CMP's slower compute.
• Benchmarks are subsets with one sample per item; the ranges cover item choice only. Business tasks and web research were machine-graded for this cut.
• Uncensored. It will answer requests the official model refuses. You are responsible for how you use it and for following the law where you are.
• Tested engines only: the vLLM SM80 backport and SGLang as described. Not GGUF or llama.cpp. The vLLM recipe runs text only (--language-model-only); images were tested on SGLang.
• keepset_mean_k256.json lists the kept expert ids per layer. Its text_tokens / image_tokens fields are placeholders written by the conversion tool; the real calibration is described in config.json under pruned_from.

Files

License and attribution

License: MIT, as for DeepSeek V4.1 Flash 1017 and the heretic base. Our patches and scripts are MIT as well; the engine files they modify keep their own licenses (Apache-2.0).

• DeepSeek-AI: DeepSeek V4.1 Flash 1017, weights, Engram tables and reference code (MIT).
• pqhaz: DeepSeek-V4.1-Flash-1017-heretic, the uncensored base (MIT), made with the method of Heretic by p-e-w.
• dealignai: DeepSeek-V4.1-Flash-1017-UNCENSORED-FP8 hosts identical Engram shards; dealignai's earlier V4.1 uncensored release was the base of this project's first cuts.
• REAP: Lasby et al., Cerebras, "REAP the Experts: Why Pruning Prevails for One-Shot MoE compression" (arXiv:2510.13999, code); the DeepSeek V4.1 implementation by LibertAI, deepseek-v41-flash-reap (MIT).
• killy: dsv41-flash-rtx-pro-6000, the SGLang setup for RTX PRO 6000 (Apache-2.0).
• wtdcode: vllm-backport, vLLM for SM80 (Apache-2.0), built on vLLM (Apache-2.0). SGLang (Apache-2.0).
• StellarVoyager: the SM80 overlay we tested (not adopted).
• cmpunlocker by amoghmunikote (GPL-2.0), referenced, not included.
• Benchmarks: LiveBench, IFBench (AllenAI), LiveCodeBench, BFCL, LongBench v2, BrowseComp-Plus, Arena-Hard v2.0, AIME 2025 and HMMT Feb 2025 (via MathArena), GPQA, Humanity's Last Exam, RealWorldQA, MMStar, OCRBench, ChartQA, MathVista, and mlabonne/harmful_behaviors. Only aggregate scores are published.
• Compared against: Qwen3.8-Flash-Next-Uncensored as the i1-IQ4_XS GGUF by mradermacher.