library_name: ninfer pipeline_tag: image-text-to-text inference: false license: apache-2.0 base_model:

• Qwen/Qwen3.8-27B
• orcarouter/Qwen3.8-27B-Uncensored
• orcarouter/Qwen3.8-27B-Uncensored-NVFP4 base_model_relation: quantized tags:
• ninfer
• qwen3.8
• nvfp4
• fp8
• w4a4
• blackwell
• multimodal
• conversational
• cuda
• rtx-5090
• abliterated
• uncensored
• not-for-all-audiences

Qwen3.8-27B Uncensored NVFP4 for NInfer

This repository contains an abliterated (refusal-removed) build of Qwen3.8-27B in the native NInfer .ninfer artifact format. It combines the BF16 weights of orcarouter/Qwen3.8-27B-Uncensored with the packed NVFP4/FP8 Text weights of orcarouter/Qwen3.8-27B-Uncensored-NVFP4. The artifact is intended only for NInfer; it is not a Transformers checkpoint, Safetensors distribution, or GGUF file.

This is the registered nvfp4 weight profile of the qwen3_8_27b target. Text layers 0–55 use NVFP4 MLP weights, while the token embedding, attention input/output projections, GDN Q/K/V/Z and output projections, full output head, and Text layers 56–63 MLP weights use row-scaled FP8. BF16 control weights and the registered MTP and Vision allocations are retained.

[!WARNING] Built from weights whose safety alignment has been substantially removed. On a sibling FP8 build, the upstream author reports harmful-prompt refusal collapsing from 64–99% to 0–6%. See the disclaimer below.

[!WARNING] This artifact cannot be reproduced from stock NInfer. Building it required a converter patch that is not upstream: orcarouter/Qwen3.8-27B-Uncensored-NVFP4 ships lm_head unquantized, and the registered recipe requires it as row-scaled FP8, so normal conversion fails at preflight. Running it needs no patch: the modification is in the Python converter, not the engine. See Converter modification.

[!IMPORTANT] This artifact reports the same identity as the official build. Its model_id is qwen3.8-27b and its weights_id is nvfp4, and it is the same size as neroued/Qwen3.8-27B-nvfp4-NInfer. Nothing at load time distinguishes them. Keep the _uncensored filename, and pass --model-id to separate them when serving.

Artifact

Objects by name prefix: text 773 · vision 333 · dflash2 66 · mtp 12 · frontend 6. Tensor formats: BF16 579 · FP32 208 · FP8_E4M3FN_ROW_BF16S 146 · NVFP4 112 · Q4G64_F16S 55 · Q5G64_F16S 54 · W8G32_F16S 28 · Q6G64_F16S 1 · I32 1.

The file contains the registered Text, Vision, MTP, DFlash2, optimized proposal-head, tokenizer, chat-template, generation, and media-processor objects required by NInfer.

Verify a downloaded file with:

printf '%s  %s\n' \
  '43025bb64f2cb558d9ede6269f8a3c2ed8ebfa4619344fe1ba8e955ba4979218' \
  'qwen3_8_27b_nvfp4_uncensored.ninfer' | sha256sum --check

Requirements

• NInfer built from source;
• 64-bit Linux;
• NVIDIA GeForce RTX 5090 (sm_120a);
• CUDA Toolkit 13.1 or newer.

Observed to load, generate and serve on NInfer revision a16b6442856620b7e4856acb25215acbf7e3c750. The minimum compatible revision has not been determined; earlier revisions are untested for this artifact.

NInfer does not provide an install target or packaged binary. See the repository README for source-build dependencies.

Download and run a CLI example

hf download JMVRoill/Qwen3.8-27B-Uncensored-nvfp4-NInfer \
  qwen3_8_27b_nvfp4_uncensored.ninfer \
  --local-dir models

./build/apps/ninfer models/qwen3_8_27b_nvfp4_uncensored.ninfer \
  --prompt "Explain prefill and decode in three sentences." \
  --max-context 32768 \
  --max-new 8192 \
  --kv-dtype fp8 \
  --spec mtp --draft-tokens 3 \
  --lm-head-draft

For images, videos, and structured chat history, see the CLI guide.

Start a local server

./build/apps/ninfer-serve models/qwen3_8_27b_nvfp4_uncensored.ninfer \
  --host 127.0.0.1 \
  --port 8080 \
  --model-id qwen3.8-27b-uncensored \
  --max-context 240000 \
  --kv-capacity 240000 \
  --max-concurrency 2 \
  --kv-dtype fp8 \
  --device-state-slots 2 \
  --host-state-slots 8 \
  --host-kv-mib 8192 \
  --spec mtp --draft-tokens 3 \
  --lm-head-draft \
  --preserve-thinking

Each request has a 240,000-token logical ceiling. The shared 240,000-token Device KV pool admits two active requests when their combined completion reservations fit; either request may use the full pool while running alone. Two extra Device checkpoint slots, eight pinned Host State slots, and 8 GiB of pinned Host KV retain reusable continuations under resource pressure.

--model-id is what distinguishes this artifact from the official one over /v1/models; the artifact identity in the engine log cannot tell them apart.

See the HTTP serving guide for the API surface and the resource scheduling reference for cache and admission semantics.

Supported use

The artifact supports:

• text generation in thinking and non-thinking modes;
• image, multi-image, video, and mixed multimodal messages;
• MTP speculative decoding with draft windows from one to five;
• DFlash2 with draft windows from one to fifteen using the included DFlash2 companion weights (--spec dflash2 --draft-tokens 7, optionally --lm-head-draft);
• BF16, INT8, FP8, NVFP4, and K8V4 KV cache;
• CUDA Graph decode and compatible-prefix reuse;
• startup-bounded small-scale concurrent serving with true batched decode;
• the NInfer CLI;
• OpenAI Responses Core, OpenAI Chat Completions, and Anthropic Messages serving.

Directly exercised on this artifact: the CLI and server invocations above, --kv-dtype fp8 and --kv-dtype nvfp4, --spec mtp --draft-tokens 3, --spec dflash2 --draft-tokens 6, --lm-head-draft, and --max-concurrency 2. The remaining entries are capabilities of the registered qwen3_8_27b target whose objects are present in the file but which were not individually exercised here.

Performance

No benchmark has been run to NInfer's published methodology, and no performance figures are claimed.

The artifact loads, generates and serves. An ad-hoc K sweep exists on the maintainer's machine but is not comparable to any published NInfer result: it omitted --no-prefix-reuse, left thinking enabled under a short token cap, used invented prompts rather than the committed fixture corpus, used no fixed seeds, computed decode rate as completion_tokens / decode_seconds where NInfer uses (completion_tokens - 1) / decode_seconds, and ran at --device-state-slots 2 which is a deployment value rather than NInfer's benchmark profile. Those rates are biased upward and are deliberately omitted here.

Numbers will be added only when produced by the reproduction command in docs/performance/methodology.md.

What was actually observed

These are single runs without fixed seeds or a fixture corpus. The acceptance figure is one sample and is not a benchmark result.

Evaluation

No capability evaluation has been run on this artifact. No EvalScope suite, no perplexity measurement, and no quality comparison against the official build.

Two smoke tests have been run over HTTP with enable_thinking:false: one coherence check (merge two sorted lists; correct output) and one refusal probe (profane limerick; complied, no refusal). Those establish that it loads, is coherent, and did not refuse on that prompt. Two prompts cannot support a quality or safety claim.

The figures below are orcarouter's, measured on their separate FP8 build, not on the NVFP4 checkpoint this artifact derives from, and not on this artifact. They argue the numbers transfer because abliteration is "a weight edit shared across all releases" of the model:

orcarouter states that capability retention and perplexity for the NVFP4 + FP8 checkpoint have not yet been measured, and that a mixed 4-bit/8-bit build is expected to trade some accuracy against the FP8 build. The additional effect of conversion to .ninfer is likewise unmeasured.

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:

• It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails.
• It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
• Use must comply with the Apache 2.0 License inherited from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm arising from this model. Its outputs do not reflect the views of the uploaders or of Qwen / Alibaba.

This disclaimer is carried forward from the upstream model card; the abliterated weights it describes are the weights in this artifact, and conversion to .ninfer does not restore any refusal behaviour.

Method

The upstream author describes the abliteration as following Arditi et al. (2024), Refusal in Language Models Is Mediated by a Single Direction, using a single refusal direction (k = 1), and states that the vision tower is untouched and the MTP head is abliterated consistently with the main model.

Refusal behaviour of this converted artifact has not been systematically measured. The only direct evidence here is a single refusal probe (profane limerick) which it complied with. One prompt is not a safety evaluation.

Converter modification

The registered qwen3_8_27b_nvfp4-v2 recipe sources text/output_head from the quantized checkpoint as row-scaled FP8, but orcarouter/Qwen3.8-27B-Uncensored-NVFP4 leaves lm_head unquantized: it ships lm_head.weight with no lm_head.weight_scale, so preflight fails.

Two changes were made:

1. text/output_head is sourced from the BF16 checkpoint and encoded locally with MAXABS_BF16S_RECIP_E4M3FN_RNE_V1, the same canonical row-scaled FP8 encoder the recipe already applies to text/token_embedding. Both are FP8 (248320, 5120) in the inventory.
2. config_groups.targets comparison accepts explicit module lists as well as the registered re: pattern forms. The upstream checkpoint spells its targets as 232 and 168 fully expanded module names, verified to select exactly the same module set as the patterns, with none missing and none extra.

Provenance

The six frontend resources (tokenizer.json, tokenizer_config.json, chat_template.jinja, generation_config.json, preprocessor_config.json, video_preprocessor_config.json) are the official Qwen files, SHA-256-verified against the converter's pinned digests. The converter requires them byte-exact, and NInfer re-verifies chat_template.jinja against a runtime whitelist at load.

[!NOTE] The .ninfer.conversion.json report emitted alongside this artifact records Qwen/Qwen3.8-27B and unsloth/Qwen3.8-27B-NVFP4 as its sources. Those strings are hard-coded constants in the recipe module and are not the sources used for this build. The table above is authoritative.

License

This NInfer artifact is distributed under the Apache License 2.0. The base repository orcarouter/Qwen3.8-27B-Uncensored, the quantized source, the DFlash2 companion, the upstream Qwen/Qwen3.8-27B, and NInfer are all licensed under Apache-2.0. Users remain responsible for complying with the license and applicable laws.