license: apache-2.0 base_model: swdq/MiniCPM-V-4.7-35B-A3B-abliterated tags: [exl3, abliterated, uncensored, minicpm-v, moe]
MiniCPM-V-4.7-35B-A3B-abliterated EXL3 3.5 bpw
這個目錄是 EXL3 量化結果，不是原始 bf16 權重。
來源
- 量化來源：swdq/MiniCPM-V-4.7-35B-A3B-abliterated
- 該權重的上游：openbmb/MiniCPM-V-4.7-35B-A3B
- 量化：exllamav3 1.4.2，
-b 3.5（不含 head 的實測位元率 3.50），head 6 bit - 視覺塔
model.vision_tower與model.merger以原始 bf16 原樣複製，沒有再量化 - 來源沒有
mtp.權重，因此沒有做 MTP 量化
下面是來源模型卡原文。其中的 vLLM 指令用來載入 bf16 來源，不能直接拿來載入這個 EXL3 目錄。
MiniCPM-V-4.7-35B-A3B-abliterated
Refusal-ablated version of openbmb/MiniCPM-V-4.7-35B-A3B, made with abliterix (LoRA steering incl. MoE expert down_proj, merged into bf16 weights; no quantization). Trial 28 of 40 (Optuna TPE).
| refusals (keyword detector, 100 harmful eval prompts) | KL divergence (harmless prompts) | |
|---|---|---|
| base | 100/100 | 0 |
| this model | 5/100 | 0.101 |
Only the language model was edited; the vision tower is unchanged and image input was not tested. Refusal counting is keyword-based (no LLM judge). Weights keep the original fused-expert layout (experts.gate_up_proj/down_proj).
Warning: safety alignment is intentionally weakened. The model may produce harmful, offensive or illegal content. You are responsible for its use and for complying with applicable laws and the base model license.
Serving the bf16 source with vLLM + DSpark (speculative decoding)
下面的指令下載並啟動的是 bf16 來源 swdq/MiniCPM-V-4.7-35B-A3B-abliterated，不是這個 EXL3 目錄。
dspark/ contains the unmodified official draft model openbmb/MiniCPM-V-4.7-35B-A3B-DSpark (Apache-2.0), trained against the base model. Tested with the vLLM nightly (0.32.0rc1) on one DGX Spark (GB10): MiniCPM-V 4.7 is not in vLLM v0.31.0 and SGLang support is not merged yet.
huggingface-cli download swdq/MiniCPM-V-4.7-35B-A3B-abliterated --local-dir ./model
vllm serve ./model --served-model-name minicpm-abliterated --dtype bfloat16 \
--max-model-len 131072 --max-num-seqs 8 --trust-remote-code \
--enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
--default-chat-template-kwargs '{"enable_thinking": false}' \
--speculative-config '{"method":"dspark","model":"./model/dspark","num_speculative_tokens":15,"enable_adaptive_verification":false}'
Measured decode speed (single stream, greedy): 25.5 tok/s without DSpark; with DSpark about 86 tok/s on code and 102 tok/s on math, but ~25-30 tok/s on prose/chat. At ~100k-token context decode was slower with DSpark (10 vs 20 tok/s). Claude Code works via ANTHROPIC_BASE_URL=http://localhost:8000.