library_name: openvino license: apache-2.0 base_model:

• guglxni/Qwen3.5-9B-abliterated-DFlash
• lukey03/Qwen3.5-9B-abliterated
• z-lab/Qwen3.5-9B-DFlash pipeline_tag: text-generation tags:
• openvino
• int8
• dflash
• speculative-decoding
• draft-model
• block-diffusion
• abliterated
• uncensored
• qwen
• intel
• arc

Qwen3.5-9B-abliterated-DFlash int8 OpenVINO IR

OpenVINO IR conversion of guglxni/Qwen3.5-9B-abliterated-DFlash, an abliterated DFlash speculative-decoding draft for the Qwen3.5-9B family.

Sibling builds: int4 · BF16

What this is

A DFlash draft model — it is not run standalone. It attaches to a Qwen3.5-9B target and proposes candidate tokens that the target verifies.

Measured performance

Intel Core Ultra 7 258V (Arc 130V/140V iGPU), 30 GB RAM, OpenVINO Model Server 2026.4.0, GPU device, single-stream, 128 max new tokens, best of 3 runs. Target: lukey03/Qwen3.5-9B-abliterated at int8.

The draft matches the upstream reference draft on this target.

Serving

Attach the draft to a target with --draft_model_path. DFlash requires prefix caching to be disabled:

ovms --configure \
  --model_path <target model dir> \
  --task text_generation \
  --draft_model_path <this model dir> \
  --enable_prefix_caching false

ovms --config_path <target dir>/ovms_config.json --rest_port 11436

A successful pairing logs Draft model strategy: DFlash followed by state: AVAILABLE.

Conversion notes

The upstream repo ships custom modeling code (dflash.py) whose auto_map points at itself, which blocks optimum-cli — its forward takes target_hidden and does not accept the hidden_states / inputs_embeds contract optimum's dummy-input generators produce.

Export directly through optimum's patched class instead:

from optimum.exporters.openvino import model_patcher
from optimum.exporters.openvino.model_configs import Qwen3OpenVINOConfig
from optimum.exporters.openvino.convert import export as ov_export
from safetensors.torch import load_file

cfg = AutoConfig.from_pretrained(SRC, trust_remote_code=False)
cfg.tie_word_embeddings = False
cfg.dflash = True

# Qwen3DFlashForCausalLM, not the base Qwen3DFlashDraftModel:
# it takes `inputs_embeds` natively and drops the seed position so emitted
# hidden states align 1:1 with candidate tokens.
model = model_patcher.Qwen3DFlashForCausalLM(cfg)
model.load_state_dict(load_file(f"{SRC}/model.safetensors"), strict=False)
model.eval()

ov_export(
    model=model,
    config=Qwen3OpenVINOConfig(
        cfg,
        task="text-generation",       # not the CLI's "text-generation-with-past"
        use_past=True,
        use_past_in_inputs=True,      # required: see below
    ),
    output="openvino_model.xml",
    stateful=True,
    library_name="transformers",      # skip library inference, which expects .model
)

Three things that silently break this export

1. Wrong patcher class. The base Qwen3DFlashDraftModel exports a stateless graph that omits the seed-position slice. It loads and generates, but every proposal is rejected — measured slower than no draft (8.5 vs 9.0 tok/s). Qwen3DFlashForCausalLM is the class optimum-cli itself uses.

2. use_past_in_inputs defaults to False. Without it the graph has present.* outputs but no matching past_key_values inputs, so patch_stateful_decoder returns early and the model stays stateless. The working reference IR exposes a beam_idx input; a stateless export does not.

3. library_name inference expects model.model. Passing library_name="transformers" avoids an AttributeError during export.

A quick check that a draft IR is correctly stateful — it must expose beam_idx and a single last_hidden_state output:

import openvino as ov
m = ov.Core().read_model("openvino_model.xml")
names = [i.get_any_name() for i in m.inputs]
assert "beam_idx" in names, "not stateful — proposals will be rejected"
print(names, [o.get_any_name() for o in m.outputs])

Attribution

• Abliterated draft: guglxni/Qwen3.5-9B-abliterated-DFlash (Apache-2.0)
• Abliterated target: lukey03/Qwen3.5-9B-abliterated (Apache-2.0)
• Upstream DFlash draft: z-lab/Qwen3.5-9B-DFlash
• DFlash paper: arXiv:2602.06036
• Conversion: OpenVINO IR via optimum-intel + NNCF