base_model: TrevorJS/gemma-4-E2B-it-uncensored pipeline_tag: image-text-to-text library_name: openvino language:

• en
• zh license: apache-2.0 tags:
• gemma-4
• openvino
• int4
• uncensored
• multimodal
• vision-language-model
• efficient

Gemma-4 E2B (5.1B) INT4 OpenVINO

Uncensored Gemma-4 E2B exported to OpenVINO IR with INT4 weight compression via NNCF.

The E2B variant is Gemma-4's efficient architecture with shared KV layers and smaller hidden dimensions, optimized for long-context scenarios.

Model Details

Usage

With openvino_genai (recommended)

import openvino_genai as g

pipe = g.VLMPipeline("gemma-4-E2B-int4-ov", "GPU")
result = pipe.generate("Hello", max_new_tokens=100)
print(result.texts[0])

Multimodal example

result = pipe.generate(
    "Describe this image",
    images=["image.jpg"],
    max_new_tokens=256,
)
print(result.texts[0])

OpenAI-compatible API server

OV_MODEL=./models_trevorjs_e2b_int4_ov_new OV_PORT=8092 python serving/ov_server.py

Performance (Arc A770 16GB, KV_CACHE=f16)

Best value: E2B achieves the best price/performance ratio. With only 4.3GB VRAM, it supports up to 14K context on a 16GB GPU.

Why E2B?

The E2B variant uses 20/35 shared KV layers and 1 KV head to dramatically reduce KV cache memory. This allows longer context windows on limited VRAM compared to the 12B model.

Export Process

Same as other Gemma-4 OpenVINO exports:

1. Export FP16 via optimum-cli export openvino --task image-text-to-text --weight-format fp16
2. Compress to INT4 with nncf.compress_weights (INT4_ASYM, group_size=128, ratio=1.0)

License

Apache 2.0

Links

• Original Gemma-4 E2B