license: mit base_model:
- orcarouter/DeepSeek-V4-Flash-Vision-Uncensored base_model_relation: quantized pipeline_tag: image-text-to-text tags:
- exl3
- exllamav3
- quantized
- vision-language
- reasoning
- moe
DeepSeek-V4-Flash-Vision-Uncensored — EXL3 4 bpw
EXL3 (trellis) quant of orcarouter/DeepSeek-V4-Flash-Vision-Uncensored, tuned for TabbyAPI / exllamav3 with CPU MoE offload. 23 safetensors shards, ~146 GB, measured final bitrate 4.04 bpw.
| component | bits |
|---|---|
| expert linears | 4 |
| output head | 6 |
| MTP / DSpark draft | 4 |
| vision tower | 6 |
| embeddings | unquantized (fp16) |
| codebook | mul1 |
Serving notes
- All expert tensors use the mul1 codebook, which is what exllamav3's
cpu_moe_split_expertsrequires; packs built with themcgcodebook cannot CPU-offload. - Verified on 56 GB VRAM (2 GPUs):
cpu_moe_split_experts: 192,cpu_moe_threads: 16,cache_size: 131072→ ~23 tok/s generation, offloading all 40 MoE layers, 64 experts resident / 192 on CPU per layer. max_seq_lenbeyond 131072 needs more VRAM than the 2-GPU split provides at these settings.- Text and vision both verified against the base template shipped in
chat_template.jinja.