base_model: OS-Software/ThinkingCap-Qwen3.8-27B-Uncensored-Heretic base_model_relation: quantized library_name: transformers pipeline_tag: image-text-to-text license: other license_name: polyform-small-business-1.0.0 license_link: LICENSE tags:

• auto-round
• autoround
• w4a16
• int4
• 4-bit
• quantized
• vllm
• qwen3_5
• qwen3_8
• image-text-to-text
• heretic
• uncensored
• decensored
• abliterated
• efficient-thinking

[!IMPORTANT] This is a derivative work. It is an AutoRound W4A16 quantization of OS-Software/ThinkingCap-Qwen3.8-27B-Uncensored-Heretic (revision 160894b1f9e60db2c548d9e46715760cc34da07c). No weights were trained or edited here; the only change is the 4-bit compression described below. All credit for the model belongs to the upstream authors.

ThinkingCap-Qwen3.8-27B-Uncensored-Heretic: AutoRound W4A16

Lineage

1. Qwen/Qwen3.8-27B: original base model by the Qwen team (Apache-2.0).
2. bottlecapai/ThinkingCap-Qwen3.8-27B: token-efficient fine-tune by BottleCap AI (PolyForm Small Business 1.0.0 plus a BottleCap personal-use grant).
3. OS-Software/ThinkingCap-Qwen3.8-27B-Uncensored-Heretic: decensored (abliterated) version made with Heretic by OS-Software, released under the base model's licence.
4. This repository: AutoRound W4A16 quantization of (3).

Please read the upstream model cards for what the model is, how it was trained and how it was evaluated. Benchmarks on those cards describe the BF16 models, not this quantized checkpoint.

What was done

Command used (inside the vLLM 0.29.0 container, CUDA_VISIBLE_DEVICES=0,1):

python3 -m auto_round --model <bf16 source> --scheme W4A16 --group_size 128 --format auto_round \
  --nsamples 512 --seqlen 2048 --iters 200 --batch_size 1 --gradient_accumulate_steps 8 \
  --low_gpu_mem_usage --device_map auto \
  --ignore_layers mtp.fc,linear_attn.in_proj_a,linear_attn.in_proj_b \
  --disable_torch_compile --enable_alg_ext

Known limits

• No MTP head. The source checkpoint ships no mtp.* tensors, so this one has none either: multi-token-prediction speculative decoding is not available.
• Checked for loading, not for quality. The checkpoint was verified structurally (every source tensor accounted for, no duplicate or doubled-prefix keys, vision tensors identical to the source) and smoke-tested in vLLM: a text prompt, a one-image prompt and a single tool call all returned correct, coherent output. No benchmark, perplexity or KL comparison against the BF16 source was run. Expect some quality loss from 4-bit quantization and do not assume the upstream benchmark numbers carry over.
• Built and tested only with vLLM 0.29.0 on 2x RTX 3090 at tensor parallel 2.

Usage (vLLM)

vllm serve <this-repo-id> --quantization auto_round --tensor-parallel-size 2 \
  --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_xml

Thinking, sampling and chat-template options follow the upstream ThinkingCap card (reasoning_effort: xhigh | medium | low; recommended sampling there is temperature=1.0, top_p=0.95, top_k=20).

⚠️ Important notice (carried over from the upstream Heretic model)

This model has undergone substantial reduction of its safety alignment. As a result, it is more likely than standard models to generate harmful, inaccurate, biased, offensive, or otherwise inappropriate content.

Intended use: research and experimentation only, including safety research, alignment studies, and red-teaming. Please avoid deploying it in public or end-user-facing services.

User responsibility: all outputs should be treated as untrusted and independently verified before use. Users are solely responsible for evaluating the accuracy and suitability of generated content, implementing appropriate safeguards and human oversight, and complying with applicable laws, regulations, licenses, and ethical standards. Use of this model is entirely at your own risk.

Disclaimer: the upstream providers and the publisher of this quantization provide the model without warranties of any kind and assume no liability for any direct or indirect damages, losses, misuse, or legal consequences arising from its use.

License

This quantization is a derivative of ThinkingCap and carries its licence: PolyForm Small Business 1.0.0 plus the BottleCap personal-use grant (see LICENSE and LICENSE-PolyForm-Small-Business-1.0.0.txt). Commercial use by organizations that do not qualify as a "Small Business" requires a separate licence from BottleCap AI. Upstream Qwen materials remain under Apache-2.0 (LICENSE-Apache-2.0-Qwen.txt, NOTICE). The licence and notice files are copied unmodified from the ThinkingCap repository.

Acknowledgements

Qwen team (Qwen3.8-27B), BottleCap AI (ThinkingCap), OS-Software and p-e-w (Heretic), and Intel (AutoRound).