language:

• en
• de
• zh license: apache-2.0 library_name: ninfer pipeline_tag: image-text-to-text tags:
• qwen3.8
• qwen3.8-27b
• huihui
• uncensored
• abliterated
• ninfer
• ninfer-v3
• gguf-blocks
• gsq-rco
• iq3_s
• mixed-precision
• vision
• multimodal
• dflash2
• mtp
• speculative-decoding
• cuda
• rtx-4090
• sm89

Huihui Qwen3.8 27B — Uncensored · GSQ-RCO IQ3_S · NInfer v3

Vision · DFlash2 · MTP · RTX 4090 / SM89 tuned · No Swift1.5

A community-packaged, mixed-precision Huihui-Qwen3.8-27B abliterated release in the native NInfer v3 (.ninfer) format. The uploaded conversion record lists a Huihui abliterated GSQ-RCO IQ3_S GGUF as the source of text weights and contains Vision, MTP, and DFlash2 components. The benchmarked deployment uses a custom CUDA 13.4 build and an RTX 4090 (SM89) device profile.

[!IMPORTANT] NInfer artifact, not Safetensors or a directly loadable GGUF. Use a compatible NInfer-all build or SPK fork. DFlash2 and MTP are separate, selectable speculation backends (--spec dflash2 or --spec mtp). Vision must be enabled separately with --vision.

RTX 4090 — measured inference

Measurement: NVIDIA RTX 4090 24 GB · CUDA 13.4 · SM89 custom build/profile · DFlash2 with 5 draft tokens · real requests logged October 30–31, 2026 (UTC). Rates are calculated from request timings; not end-to-end throughput, a standardized benchmark, or a demonstrated speedup over stock NInfer. MTP and image input were not benchmarked. Full methodology, additional statistics, exact build and launch configuration: BENCHMARK-RTX4090.md.

Model at a glance

Quantization is mixed-precision: IQ3_S is part of the model's quantization family, not the format used by every tensor. The conversion record reports the individual formats. The quantized weights are not restricted to RTX 4090; optimization refers to the tested build, calibration profile and deployment configuration. Other supported NVIDIA GPUs may work with appropriate binaries, profiles and memory budgets.

Quick start

1. Download the model

Install the Hugging Face CLI:

py -m pip install -U huggingface_hub

Then download the native NInfer model (PowerShell):

$repo = 'SPK1337/Huihui-Qwen3.8-27B-Uncensored-GSQ-RCO-IQ3S-Vision-DFlash2-MTP-NInfer'
$file = 'Huihui-Qwen3.8-27B-GSQ-RCO-IQ3S-Vision-DFlash2.ninfer'
hf download $repo $file --local-dir '.\models'

2. Run with DFlash2 — Windows / RTX 4090

Build or download a compatible ninfer-serve.exe from NInfer-all or the SPK fork. Replace the executable path below with the location of your build:

$server = 'C:\path\to\ninfer-serve.exe' # CHANGE THIS
$model = (Resolve-Path '.\models\Huihui-Qwen3.8-27B-GSQ-RCO-IQ3S-Vision-DFlash2.ninfer').Path

& $server $model `
  --model-id huihui-qwen3.8-27b `
  --host 127.0.0.1 --port 8080 `
  --max-context 32768 --kv-capacity auto `
  --kv-dtype rk4v4 --gdn-state-fp16 `
  --spec dflash2 --draft-tokens 5

This 32K-context example is a conservative starting point, not the full benchmark configuration. For the measured 176,128-token ceiling, RTX 4090 device profile, concurrency and cache settings, see the benchmark launch recipe.

Use MTP instead: replace the last line with --spec mtp --draft-tokens 3. Do not enable MTP and DFlash2 simultaneously; compare each on your own workloads.

Use image input: add --vision. On tighter memory budgets, try --vision-residency overlay and reduce context as necessary. The Vision component is recorded in the converter metadata, but Vision quality/latency were not measured in the supplied benchmark.

3. Linux example

Once ninfer-serve is built or installed, the same model can be launched with equivalent flags:

ninfer-serve ./models/Huihui-Qwen3.8-27B-GSQ-RCO-IQ3S-Vision-DFlash2.ninfer \
  --model-id huihui-qwen3.8-27b --host 127.0.0.1 --port 8080 \
  --max-context 32768 --kv-capacity auto --kv-dtype rk4v4 \
  --gdn-state-fp16 --spec dflash2 --draft-tokens 5

Linux and other GPUs require a build that supports their CUDA architecture; the RTX 4090 results above are not Linux performance measurements.

4. Send an OpenAI-compatible request

$body = @{
  model = 'huihui-qwen3.8-27b'
  messages = @(@{ role = 'user'; content = 'Hello! Introduce yourself briefly.' })
  max_tokens = 128
} | ConvertTo-Json -Depth 10

Invoke-RestMethod -Uri 'http://127.0.0.1:8080/v1/chat/completions' `
  -Method Post -ContentType 'application/json' -Body $body

For the multimodal request schema, runtime options, cache modes and supported GPUs, consult NInfer serving, recipes and GPU support.

Conversion provenance: what is actually recorded

The published *.ninfer.conversion.json identifies:

This is stronger evidence than the repository name that the conversion used a Huihui-labelled abliterated GGUF, and that the optional components were assembled. It is not an independent checksum or audit of the original GGUF's training history, censorship behavior, or upstream revision. Source paths in the JSON are local converter paths; they are not downloadable dependency links.

Relevant upstream projects and credits:

• Qwen/Qwen3.8-27B — base architecture / original model family.
• huihui-ai/Huihui-Qwen3.8-27B-abliterated — Huihui abliterated variant and attribution reference.
• ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF — quantization methodology/reference, not asserted as the shipped Huihui GGUF itself.
• z-lab/Qwen3.8-27B-DFlash2 — DFlash2 reference.
• NInfer-all, original NInfer, SPK fork — conversion and inference tooling.

See NOTICE for the full attribution discussion. This is a community release, not an official publication or endorsement by the upstream projects.

Release files and integrity

After downloading, compare your local model's SHA-256 digest with the published SHA256SUMS:

Get-FileHash '.\models\Huihui-Qwen3.8-27B-GSQ-RCO-IQ3S-Vision-DFlash2.ninfer' -Algorithm SHA256

The sidecar conversion JSON is for provenance/documentation; it is not necessary to run the .ninfer model. Changing either file requires regenerating its checksum entry.

Limitations, safety and licensing

• Abliteration / “uncensored” does not establish accuracy, safety, legal suitability or unconditional compliance with requests. Model outputs may be wrong, biased or unsafe; verify high-stakes information independently.
• This release is not independently evaluated here for Vision correctness, multilingual ability, coding quality, reasoning or standardized academic benchmark scores.
• DFlash2 was measured; MTP was not. No controlled comparison against stock NInfer or other GPU models was performed, so no quantitative tuning advantage is claimed.
• Memory requirements vary by context, KV encoding, Vision residency and active draft backend. The benchmarked context limit does not guarantee full utilization in every workload.
• Respect applicable laws, privacy obligations and all upstream licenses/notices. Distribution under the repository's Apache-2.0 designation does not override component-specific conditions.

Published by SPK1337. Credit for the original model, fine-tuning, quantization, drafting and runtime belongs to the upstream authors and contributors.