library_name: ninfer license: apache-2.0 base_model:

• Qwen/Qwen3.8-27B pipeline_tag: image-text-to-text inference: false language:
• en
• zh tags:
• ninfer
• qwen3.8
• abliterated
• uncensored
• vision
• multimodal
• image-to-text
• mtp
• dflash2
• speculative-decoding
• mixed-precision
• low-bit

Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2

English | 简体中文

NInfer v3 artifacts (.ninfer) of the abliterated Qwen3.8-27B. The text tower, token embedding, output head and MTP head are the blocks of the GSQ-RCO mixed-precision IQ2-class GGUF (Underdog-Saluki Swift 1.5), stored byte for byte; the Vision tower and the DFlash2 adapter are re-encoded from their own sources. Each file is self-contained: weights, tokenizer, chat template, generation config, image/video preprocessor configs, logical bindings. The container is not a GGUF file: transformers, llama.cpp and GGUF runtimes cannot read it, an NInfer v3 engine loads it.

Files

File B is file A plus the DFlash2 drafter (+2.13 GiB). Download one.

Download

hf download NateVoid/Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-Ninfer \
  Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp.ninfer --local-dir models

hf download NateVoid/Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-Ninfer \
  Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp-dflash2.ninfer --local-dir models

Composition

GDN controls (ssm_alpha, ssm_beta) are stored as BF16; norms, convolution, A_log and dt_bias are exact, with the exporter's conventions restored (w instead of 1 + w, A_log from -exp(A_log)). The GDN output projection keeps llama.cpp's tiled value-head order as an input_columns Use auxiliary.

7.905 GiB of file A (96.1%) is stored in gguf_blocks_v1; the rest is the Vision tower in NInfer's Vision formats and the BF16/F32 parameter tensors. File B adds the DFlash2 adapter, which is not part of the GGUF either.

Runtime

Engine: iamwavecut/ninfer-all. Kernels are selected from the architecture, configuration and bindings stored in the artifact.

Quick start

File A:

ninfer-serve Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp.ninfer \
    --model-id huihui-qwen3.8-27b-iq2 --host 127.0.0.1 --port 1145 \
    --max-context 262144 --kv-capacity auto --kv-dtype int8 --gdn-state-fp16 \
    --spec mtp --draft-tokens 4 --adaptive-mtp --lm-head-draft \
    --vision --vision-residency resident --vision-max-merged 1024 \
    --prefill-cublas

File B:

ninfer-serve Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp-dflash2.ninfer \
    --model-id huihui-qwen3.8-27b-iq2-dflash2 --host 127.0.0.1 --port 1145 \
    --max-context 32768 --kv-capacity auto --kv-dtype rk8v4 --gdn-state-fp16 \
    --spec dflash2 --draft-tokens 5 --lm-head-draft \
    --vision --vision-residency overlay --vision-max-merged 1024

API: OpenAI-compatible under /v1. On Windows the runtime path, model path and model name must be ASCII.

Conversion

Official recipe qwen3_8_27b_gguf:

python3 -m tools.convert `
  --model /path/to/Qwen3.8-27B `
  --recipe qwen3_8_27b_gguf `
  --source gguf=Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp.gguf `
  --source vision=mmproj-model-bf16.gguf `
  --components text,vision,mtp `
  --resource chat_template.jinja=tools/chat_templates/qwen3_8.jinja `
  --proposal `
  --name huihui-qwen3.8-27b-abliterated-underdog-saluki-swift15-iq2 `
  --out models/Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp.ninfer

# file B adds:
#   --source dflash2=/path/to/Qwen3.8-27B-DFlash2 --components text,vision,mtp,dflash2

Each conversion writes a .conversion.json report beside the artifact with the source, method and format of every object: 1194 objects in file A (1188 tensors and 6 resources), 1260 in file B.

Limitations

• Text tower at 2.351 bits/weight. No perplexity or KLD measurement is published.
• Abliterated model; refusal behaviour is intentionally altered.
• Video input is not supported by the engine.
• DFlash2 is trained against the base model and is reused here unchanged.

Credits

Apache-2.0, inherited from the base model.

简体中文

English | 简体中文

去审查版 Qwen3.8-27B 的 NInfer v3 产物（.ninfer）。文本塔、token 嵌入、输出头与 MTP 头就是 GSQ-RCO 逐张量混合精度 IQ2 级 GGUF（Underdog-Saluki Swift 1.5）的块，逐字节存入；视觉塔与 DFlash2 适配器则各自从其来源重新编码。每个文件自包含：权重、分词器、对话模板、生成配置、图像/视频预处理 配置、逻辑绑定。容器不是 GGUF 文件：transformers、llama.cpp 与 GGUF 运行时都读不了它，需由 NInfer v3 引擎加载。

文件

文件 B = 文件 A + DFlash2 草稿头（+2.13 GiB）。二者取一。

下载

hf download NateVoid/Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-Ninfer \
  Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp.ninfer --local-dir models

hf download NateVoid/Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-Ninfer \
  Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp-dflash2.ninfer --local-dir models

构成

GDN 控制量（ssm_alpha、ssm_beta）以 BF16 存储；范数、卷积、A_log、dt_bias 为精确值，并还原导出器约定（w 而非 1 + w，A_log 由 -exp(A_log) 得到）。GDN 输出投影保留 llama.cpp 的分块 value head 顺序，以 input_columns Use 辅助量表达。

文件 A 有 7.905 GiB（96.1%）以 gguf_blocks_v1 存储；其余是 NInfer 视觉格式的视觉塔与 BF16/F32 参数张量。文件 B 另加 DFlash2 适配器，它同样不属于那个 GGUF。

运行时

引擎：iamwavecut/ninfer-all。内核由产物内的架构、配置和绑定决定。

快速开始

文件 A：

ninfer-serve Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp.ninfer \
    --model-id huihui-qwen3.8-27b-iq2 --host 127.0.0.1 --port 1145 \
    --max-context 262144 --kv-capacity auto --kv-dtype int8 --gdn-state-fp16 \
    --spec mtp --draft-tokens 4 --adaptive-mtp --lm-head-draft \
    --vision --vision-residency resident --vision-max-merged 1024 \
    --prefill-cublas

文件 B：

ninfer-serve Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp-dflash2.ninfer \
    --model-id huihui-qwen3.8-27b-iq2-dflash2 --host 127.0.0.1 --port 1145 \
    --max-context 32768 --kv-capacity auto --kv-dtype rk8v4 --gdn-state-fp16 \
    --spec dflash2 --draft-tokens 5 --lm-head-draft \
    --vision --vision-residency overlay --vision-max-merged 1024

API：/v1 下 OpenAI 兼容接口。Windows 下运行时路径、模型路径与模型名须为 ASCII。

转换

官方 recipe qwen3_8_27b_gguf：

python3 -m tools.convert `
  --model /path/to/Qwen3.8-27B `
  --recipe qwen3_8_27b_gguf `
  --source gguf=Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp.gguf `
  --source vision=mmproj-model-bf16.gguf `
  --components text,vision,mtp `
  --resource chat_template.jinja=tools/chat_templates/qwen3_8.jinja `
  --proposal `
  --name huihui-qwen3.8-27b-abliterated-underdog-saluki-swift15-iq2 `
  --out models/Huihui-Qwen3.8-27B-abliterated-Underdog-Saluki-Swift-1.5-IQ2-mtp.ninfer

# 文件 B 追加：
#   --source dflash2=/path/to/Qwen3.8-27B-DFlash2 --components text,vision,mtp,dflash2

每次转换在产物旁写出 .conversion.json，记录每个对象的来源、方法与格式：文件 A 1194 个 （1188 个张量 + 6 个资源），文件 B 1260 个。

限制

• 文本塔 2.351 位/权重。未发布困惑度或 KLD 测量。
• 去审查模型，拒答行为被有意改变。
• 引擎不支持视频输入。
• DFlash2 针对基座模型训练，此处原样复用。

出处

Apache-2.0，继承自基座模型。