license: apache-2.0 base_model: Qwen/Qwen3.6-27B tags:
- abliteration
- uncensored
- qwen3
- qwen3.6
- hybrid-attention
- deltanet
- vlm
- multimodal
- gguf
- llama-cpp
- ollama
- mtp model_type: qwen3_5 pipeline_tag: image-text-to-text library_name: gguf
Qwen3.6-27B-abliterated-GGUF
GGUF quantizations of Qwen/Qwen3.6-27B with refusal behavior removed via abliteration. For use with llama.cpp, Ollama, LM Studio, and other GGUF-compatible runtimes.
Blog post: Abliteration: Uncensoring LLMs via Weight Surgery
Standard Quantizations (No MTP)
Maximum compatibility — works with Ollama, LM Studio, KoboldCPP, llama.cpp out of the box.
| File | Size | RAM Required | Notes |
|---|---|---|---|
...-F16.gguf | 53.8 GB | 64+ GB | Full precision |
...-Q4_K_M.gguf | 26.9 GB | 32+ GB | Recommended (F16 vision embed) |
...-Q4_K_M-Q8.gguf | 19.7 GB | 24+ GB | Q4 language + Q8 vision embed |
MTP Quantizations (Multi-Token Prediction)
Include MTP draft head for speculative decoding. Larger files but faster inference with compatible runtimes.
| File | Size | RAM Required | Notes |
|---|---|---|---|
...-MTP-F16.gguf | 54.7 GB | 64+ GB | Full precision + MTP |
...-MTP-Q8_0.gguf | 29 GB | 36+ GB | Near-lossless + MTP |
...-MTP-Q6_K.gguf | 22.4 GB | 32+ GB | Very high quality + MTP |
...-MTP-Q5_K.gguf | 19.5 GB | 24+ GB | Recommended for 48GB + MTP |
...-MTP-Q4_K.gguf | 16.8 GB | 20+ GB | Good quality + MTP |
...-MTP-Q3_K.gguf | 13.5 GB | 16+ GB | 16GB VRAM + MTP |
...-MTP-Q2_K.gguf | 10.9 GB | 12+ GB | 2-bit + MTP |
Vision Encoder
| File | Size | Notes |
|---|---|---|
...-mmproj-f16.gguf | 928 MB | Required for multimodal (image understanding) |
MTP (Multi-Token Prediction)
MTP files include a draft head for speculative decoding — faster token generation:
./llama-server -m Qwen3.6-27B-abliterated-MTP-Q5_K.gguf \
--jinja --spec-type draft-mtp --spec-draft-n-max 6 -ngl 99
Runtime compatibility: MTP requires llama-server b9180+. Ollama does not support MTP yet. Use the standard (non-MTP) files for Ollama/LM Studio.
Multimodal (Vision)
The mmproj-f16.gguf file is the vision encoder for image understanding:
./llama-mtmd-cli -m Qwen3.6-27B-abliterated-Q4_K_M.gguf \
--mmproj Qwen3.6-27B-abliterated-mmproj-f16.gguf \
-p "Describe this image" --image photo.jpg
Quickstart — Ollama
# Recommended for Apple Silicon 48GB+ (M4 Pro, M4 Max)
ollama run hf.co/Bahushruth/Qwen3.6-27B-abliterated-GGUF:Q4_K_M
# For 24GB systems
ollama run hf.co/Bahushruth/Qwen3.6-27B-abliterated-GGUF:Q4_K_M-Q8
Usage — llama.cpp
huggingface-cli download Bahushruth/Qwen3.6-27B-abliterated-GGUF \
Qwen3.6-27B-abliterated-Q4_K_M.gguf --local-dir .
./llama-cli -m Qwen3.6-27B-abliterated-Q4_K_M.gguf \
-p "You are a helpful assistant." \
--chat-template chatml -cnv -c 262144
Architecture Notes
Qwen3.6-27B is a dense hybrid-attention multimodal model:
- Hybrid attention: 16 blocks of (3x Gated DeltaNet + 1x Gated Attention) = 64 layers
- Parameters: 27B dense (all active per token)
- Hidden size: 5120
- Context: 262K native, extensible to 1M+ via YaRN
- Multimodal: Vision encoder for image understanding
- MTP: Multi-token prediction draft head for speculative decoding
Disclaimer
This model has had safety guardrails removed. Released for research purposes. The creator assumes no responsibility for downstream use.