library_name: openvino license: apache-2.0 base_model:

• huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated pipeline_tag: image-text-to-text tags:
• openvino
• int8
• qwen2.5-vl
• vlm
• abliterated
• uncensored
• intel
• arc

Qwen2.5-VL-7B-Instruct-abliterated — OpenVINO IR (int8)

OpenVINO IR conversion of huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated, an abliterated (decensored) build of Qwen2.5-VL-7B-Instruct.

This is a full VLM export: vision tower, vision-language merger, text embeddings, language model, tokenizer and detokenizer are all present.

Sibling build: int4

Details

Because the text graph has more than the two inputs LLMPipeline expects, load this through VLMPipeline, or serve it with OVMS which dispatches automatically.

Benchmark

Intel Core Ultra 7 258V (Arc 130V/140V iGPU), 30 GB RAM, OpenVINO Model Server 2026.4.0 on GPU, greedy decoding, 128 max new tokens, mean of 3 runs.

Decode here is memory-bandwidth bound, so throughput tracks model size closely.

Serving

cat > ovms_config.json <<'EOF'
{
  "model_config_list": [
    {
      "config": {
        "name": "qwen2.5-vl",
        "base_path": "/absolute/path/to/this/model",
        "target_device": "GPU",
        "nireq": 8,
        "plugin_config": { "PERFORMANCE_HINT": "THROUGHPUT", "NUM_STREAMS": "2" }
      }
    }
  ]
}
EOF

# graph.pbtxt must also exist in the model directory.
PYTHONPATH=$OVMS_ROOT/lib/python ovms --config_path ./ovms_config.json --rest_port 11436

Conversion notes

• Stage 1: optimum-cli export openvino --task image-text-to-text --weight-format fp16
• Stage 2: nncf.compress_weights on the language-model IR.
• Requires transformers 5.0 — the qwen2_5_vl exporter rejects newer versions (MAX_TRANSFORMERS_VERSION = "5.0"). Newer transformers fails with a version-gate error.
• optimum-intel 2.2.0 / OpenVINO 2026.4.0.

Limitations

• Abliterated: reduced refusal behavior. Evaluate outputs before deployment.
• Quantized; re-export at a smaller group size or fp16 for maximum accuracy.
• Throughput depends on runtime kernels, hardware, and prompt distribution.

Attribution

Base model: huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated. Qwen2.5-VL is Apache-2.0. Conversion via optimum-intel + NNCF.