base_model: DavidAU/Qwen3.5-9B-The-Defiant-Fable-DARK-ROAST-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP library_name: openvino pipeline_tag: image-text-to-text tags:

• qwen3.5
• openvino
• ovms
• int4
• quantized
• mtp
• speculative-decoding

Qwen3.5-9B Uncensored INT4 OpenVINO

This repository contains an OpenVINO INT4 asymmetric quantized conversion of:

DavidAU/Qwen3.5-9B-The-Defiant-Fable-DARK-ROAST-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP

It is intended for efficient local inference with OpenVINO and OpenVINO Model Server (OVMS).

Converted From:

• Base model: DavidAU/Qwen3.5-9B-The-Defiant-Fable-DARK-ROAST-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP
• Format: OpenVINO IR
• Weight precision: INT4
• Quantization: Asymmetric
• Task: image-text-to-text
• MTP: Included when openvino_mtp_model.xml/bin are present
• Recommended device: Intel GPU

No additional fine-tuning was performed during conversion.

OVMS Requirement

This model uses the experimental/recent Qwen3.5 MTP speculative decoding path.

A recent OVMS development build is recommended and may be required. Older stable OVMS releases may not contain the necessary Qwen3.5 MTP support.

OVMS development builds are available at:

https://storage.openvinotoolkit.org/repositories/openvino_model_server/packages/weekly/

For bare-metal OVMS installation and deployment instructions, see:

https://docs.openvino.ai/2025/model-server/ovms_docs_deploying_server_baremetal.html

When troubleshooting MTP initialization issues, first update to the latest available weekly OVMS build before assuming the converted model is incompatible.

OVMS Deployment

Example:

ovms `
  --rest_port 8000 `
  --model_name Qwen3.5-9B-uncensored-int4-ov `
  --model_path "path/to/model" `
  --target_device GPU `
  --draft_model_path . `
  --enable_prefix_caching false `
  --pipeline_type VLM

--draft_model_path . tells OVMS to use the bundled openvino_mtp_model.xml as the MTP draft model for speculative decoding.

Prefix caching should remain disabled and pipeline_type should be VLM when using the MTP configuration.

OpenAI-Compatible API

After OVMS starts, the model can be accessed through:

http://127.0.0.1:8000/v3/chat/completions

Example:

{
  "model": "Qwen3.5-9B-uncensored-int4-ov",
  "messages": [
    {
      "role": "user",
      "content": "Hello!"
    }
  ],
  "max_tokens": 1024,
  "num_assistant_tokens": 5,
  "stream": true
}

num_assistant_tokens controls how many tokens the MTP draft model proposes during speculative decoding.

A value such as 3 or 5 is a reasonable starting point. The optimal value depends on draft-token acceptance rate and target hardware.

Notes

This is a community OpenVINO conversion. Model behavior, capabilities, limitations, and licensing are inherited from the original model.

INT4 quantization may introduce small differences compared with the original floating-point checkpoint.

MTP performance depends on:

• OVMS/OpenVINO version
• target hardware
• draft-token acceptance rate
• num_assistant_tokens
• prompt and generation length

Because Qwen3.5 MTP support is evolving rapidly, using the latest weekly OVMS build is strongly recommended.

Please refer to the original model repository for full model details and usage terms.