library_name: transformers pipeline_tag: image-text-to-text inference: false license: apache-2.0 base_model:

d0xin/Swift-Qwen3.8-27B-Uncensored-BF16 tags:
qwen3.8
nvfp4
w4a4
compressed-tensors
vllm
sglang
blackwell
rtx-5090
abliterated
uncensored

Swift (1.0) abliterated Qwen3.8-27B NVFP4 (HF format / vLLM-ready)

HF-format quantized weights (compressed-tensors NVFP4, W4A4 group-16) of d0xin's Swift-Qwen3.8-27B-Uncensored-BF16 (huihui-style abliteration) — this is the exact pre-conversion source of our NInfer v3 container Qwen3.8-27B-swift-abliterated-nvfp4full-dflash2-NInfer-v3.

Published for structured-output users: NInfer does not support JSON-schema output; vLLM does. DFlash2 speculative decoding is engine-specific and lives only in the NInfer container.

Sibling: dragooy's Swift-Qwen3.8-27B-abliterated-NVFP4 (a different quantization of the same base — credit to Dragoy for the HF-format-first idea).

What's inside

model.safetensors — 17.1 GiB NVFP4 (W4A4 gs16) text stack + vision tower, W8G32 token embedding + output head; calibrated on 512 Ultrachat samples (seq 2048)
model_mtp.safetensors — MTP head (used by the NInfer container; ignored by vLLM)
recipe.yaml — the llm-compressor recipe, for reproducibility
Packaging note: per-packing-group global scales are unified (shrink-only E4M3 re-encode) — mathematically equivalent to the raw llm-compressor output within E4M3 re-encode precision.

Run with vLLM

pip install -U vllm

vllm serve kaushikvira/Qwen3.8-27B-swift-abliterated-NVFP4-HF \
  --max-model-len 131072     # 32 GB GPU: 131k fits comfortably; 262k on 48 GB+

JSON-schema structured output

from vllm import LLM, SamplingParams
from vllm.sampling_params import GuidedDecodingParams

llm = LLM(model="kaushikvira/Qwen3.8-27B-swift-abliterated-NVFP4-HF", max_model_len=131072)

schema = {
    "type": "object",
    "properties": {"answer": {"type": "string"}, "confidence": {"type": "number"}},
    "required": ["answer"],
}

sp = SamplingParams(temperature=0.0, guided_decoding=GuidedDecodingParams(json=schema))
out = llm.chat([{"role": "user", "content": "What is 2+2? Answer in JSON."}], sp)
print(out[0].outputs[0].text)

Credits & license

All model credit to d0xin (uncensored BF16), huihui-ai (abliteration lineage), ukisai (Swift post-training), Qwen team (base, Apache-2.0).
License: Apache-2.0, inherited through the artifact chain.
Quantization/packaging by kaushikvira.

Benchmarks — not measured yet on this packaging

The quantization is identical to our NInfer v3 container of the same weights (which carries full gate/needle/llama-benchy numbers on a single RTX 5090), but the vLLM/SGLang-specific numbers for this repo are not measured yet — this is a fresh packaging.

If you run it, please share your numbers in the Community tab (tool + version, GPU, context length, pp/tg tok/s) — we'll collect them here for everyone.

Swift (1.0) abliterated Qwen3.8-27B NVFP4 (HF format / vLLM-ready)

Published for structured-output users: NInfer does not support JSON-schema output; vLLM does. DFlash2 speculative decoding is engine-specific and lives only in the NInfer container.

Sibling: dragooy's Swift-Qwen3.8-27B-abliterated-NVFP4 (a different quantization of the same base — credit to Dragoy for the HF-format-first idea).

What's inside

model.safetensors — 17.1 GiB NVFP4 (W4A4 gs16) text stack + vision tower, W8G32 token embedding + output head; calibrated on 512 Ultrachat samples (seq 2048)

model_mtp.safetensors — MTP head (used by the NInfer container; ignored by vLLM)

recipe.yaml — the llm-compressor recipe, for reproducibility

Packaging note: per-packing-group global scales are unified (shrink-only E4M3 re-encode) — mathematically equivalent to the raw llm-compressor output within E4M3 re-encode precision.

JSON-schema structured output

from vllm import LLM, SamplingParams from vllm.sampling_params import GuidedDecodingParams llm = LLM(model="kaushikvira/Qwen3.8-27B-swift-abliterated-NVFP4-HF", max_model_len=131072) schema = { "type": "object", "properties": {"answer": {"type": "string"}, "confidence": {"type": "number"}}, "required": ["answer"], } sp = SamplingParams(temperature=0.0, guided_decoding=GuidedDecodingParams(json=schema)) out = llm.chat([{"role": "user", "content": "What is 2+2? Answer in JSON."}], sp) print(out[0].outputs[0].text)

Benchmarks — not measured yet on this packaging

If you run it, please share your numbers in the Community tab (tool + version, GPU, context length, pp/tg tok/s) — we'll collect them here for everyone.