license: apache-2.0 license_link: https://ai.google.dev/gemma/docs/gemma_4_license library_name: transformers pipeline_tag: image-text-to-text base_model: edwixx/diffusiongemma-26B-A4B-it-HERETIC-Uncensored base_model_relation: quantized tags:
- nvfp4
- fp4
- compressed-tensors
- vllm
- llm-compressor
- heretic
- diffusion
- moe
diffusiongemma-26B-A4B-it-HERETIC-Uncensored-NVFP4
NVFP4 quantization of edwixx/diffusiongemma-26B-A4B-it-HERETIC-Uncensored.
This is not a quant of google/diffusiongemma-26B-A4B-it. Heretic changed the expert weights, so those weights were quantized from the HERETIC checkpoint. The official Red Hat and NVIDIA NVFP4 checkpoints are quants of the original Google model.
Format
compressed-tensors NVFP4 (W4A4), nvfp4-pack-quantized.
- MoE experts and the dense MLP linears are NVFP4, group size 16.
- Attention, embeddings, the router, the vision tower, self-conditioning, and
lm_headstay BF16. - Experts are stored as per-expert packed linears:
experts.{i}.{gate,up,down}_proj.weight_packed.
Calibration used 128 samples from HuggingFaceH4/ultrachat_200k (train_sft), truncated to 2048 tokens, with every expert calibrated. Text only. The vision tower was not quantized.
Serve with vLLM
VLLM_USE_V2_MODEL_RUNNER=1
vllm serve Zewsic/diffusiongemma-26B-A4B-it-HERETIC-Uncensored-NVFP4 \
--trust-remote-code \
--max-num-seqs 4 \
--hf-overrides '{"diffusion_sampler": "entropy_bound", "diffusion_entropy_bound": 0.1}' \
--default-chat-template-kwargs '{"enable_thinking": true}'
Native NVFP4 needs a Blackwell GPU (or newer). On older GPUs vLLM falls back to a slower path.
Inherited behavior
The source checkpoint can emit the token own in place of other words. Quantization does not remove that. Compare outputs with the HERETIC checkpoint, not with the original Google model.
License
Apache 2.0, and the Gemma 4 terms that apply to the base weights.