license: mit base_model: pqhaz/DeepSeek-V4.1-Flash-1017-heretic base_model_relation: quantized tags:

• gguf
• quantized
• vector-quantization
• deepseek
• moe
• abliterated
• uncensored

DeepSeek-V4.1-Flash-1017 abliterated (pqhaz) — lecore vq11 calibrated GGUF (106.8 GB)

cover

A 106.8 GB single-file GGUF of pqhaz/DeepSeek-V4.1-Flash-1017-heretic, the pqhaz abliteration of deepseek-ai/DeepSeek-V4.1-Flash-1017, quantized with lecore activation-calibrated 11-bit vector quantization of the routed experts.

Credit: all abliteration work belongs to pqhaz (source model). The base model is DeepSeek's DeepSeek-V4.1-Flash-1017. This repo only re-quantizes it.

What is in this build

• Routed experts (557B params): VQ, 11-bit codebooks (2048 entries) everywhere, activation-weighted (calibrated) k-means on 65,536 calibration tokens; 1.39 bpw, weighted cosine 0.909.
• Shared/common experts: VQ 11-bit, cosine 0.910.
• Attention skeleton: q4_K (AMD-supported GEMV path), cosine 0.997. Output head: FP8 32x32 blocks.
• Engram n-gram tables: DeepSeek's original shards, used unchanged (download separately, ~203 GB).
• File type-scan passed; decode-parity check passed against the source weights.

Evaluation

Teacher = the unquantized source model (pqhaz/DeepSeek-V4.1-Flash-1017-heretic), same official forward pass. WikiText-2 test set. Tail metrics over the 1,233 reserved token ids in the tokenizer (place_holder / control ids).

For reference, the earlier uncalibrated lecore-vq10sh12 build of the original model: PPL 3.615, KL 0.797 mean / 0.195 median, top-1 74.5% (2k).

Recommended sampling

temperature 0.6, top_p 0.95, min_p 0.05, thinking/reasoning off for agent use. Greedy (temperature 0) also works.

Serving (Windows / Radeon 8060S, native HIP)

Measured on a Radeon 8060S (gfx1151, HIP SDK 7.2, 128 GB unified memory) with the patched ds4-server from the companion repo. Paths are examples.

Start

cd ds4-winhip
.\hip\win\run_ds41.ps1 -Engram D:\models\ds41\deepseek-engram      # OpenAI-compatible API on http://127.0.0.1:8008/v1

run_ds41.ps1 sets these unless already present in the environment:

Load is ~6 minutes to the first answer: the weight copy, then the first request pays ~110 s of one-off kernel and graph warm-up. Do not benchmark the first request. One request is served at a time; a second one queues.

Speed on this hardware (measured on the vq10sh12 build): prefill 135-190 tok/s (real agent prompts vs synthetic), decode 14-17 tok/s. Both are kernel-bound, not format-bound.

Sampling

The earlier uncalibrated vq10sh12 build produced word salad with reserved tokens (e.g. place_holder_no_362) under T=1.0 with no truncation at 14k+ prompts. This build's tail metrics (junk mass, escape mass, top-p stability) are in the Evaluation section. Recommended regardless:

• temperature 0 (greedy), or 0.6-0.7 with top_p 0.95 and min_p 0.05;
• reasoning off for agent use (reasoning_effort: "none" or thinking: {"type":"disabled"}).

Hermes Agent

~/.hermes/config.yaml (Windows: %LOCALAPPDATA%\hermes\config.yaml):

provider: custom            # not "deepseek": that name selects the cloud provider and demands its key
base_url: http://127.0.0.1:8008/v1
api_key: local
model: local-ds41           # any name that does not prefix-match a cloud provider
reasoning_effort: "none"

Hermes's system prompt plus tool schemas is ~14k tokens, so a cold turn is 100-140 s to first token. Follow-up turns are cheap only if the KV checkpoint survives.

KV prefix reuse

The server keeps two live checkpoints: the state at the end of the last prompt and at the end of the last reply. A new request whose tokens start with the last reply's key prefills only the new tokens. Any other request in between overwrites both, and the next turn re-prefills everything (measured: 19.3k tokens, 138 s). While an agent is working, nothing else should use the port.

--prefix-reuse off|exact|auto controls it (exact is the default). The disk store (--kv-disk-dir <dir> --kv-disk-space-mb <n>) keeps many checkpoints keyed by token prefix; a 14k-token checkpoint is ~30 MB. Present but not yet validated on the Windows build.

The KV cache is stored 4-bit (FP4 values, E4M3 scales) regardless of the weight quant.

Gotchas

• Rebuilding: cargo build -p ds4-server compiles and then fails at link (no error[ lines); expected, link with hip\win\link_server.ps1. A running server keeps its own exe copy.
• Stopping: kill ds4-server, then wait for Windows commit to drop before starting again, or the loader thrashes at the commit limit.
• PowerShell 5.1 reads a BOM-less UTF-8 .ps1 as ANSI, which mangles the fullwidth bar in <｜...｜> tokens. Save scripts that contain them with a BOM.
• usage.prompt_tokens tells you whether a special token in your text was parsed as one id (5 with the chat template for a lone token) or split into text (18).

Runtime

Needs the patched ds4 / ds4-server runtime (companion ds4-winhip repo). llama.cpp, vLLM and Ollama cannot load the VQ blobs.

License

MIT, following the base model and the source abliteration.