language:

• en
• zh license: apache-2.0 tags:
• qwen3_5
• gguf
• vision
• mtp
• qwen3.8
• distilled
• heretic
• abliterated
• uncensored
• conversational base_model: insraq/Qwen3.5-4B-EmperoAI-Qwen3.8-Distill-Heretic-Abliterated

Qwen3.8-4B-Distill-Heretic-Abliterated-GGUF

GGUF version of insraq/Qwen3.5-4B-EmperoAI-Qwen3.8-Distill-Heretic-Abliterated, with both MTP (Multi-Token Prediction) speculative decoding layers and full vision weights preserved.

Why this exists

Two problems with existing GGUF conversions of this model:

1. Missing MTP layers: The original HF→GGUF conversion using llama.cpp's convert_hf_to_gguf.py produced a broken GGUF — the metadata declared 33 layers (32 transformer + 1 MTP) but only 32 layers' tensors were actually exported, causing check_tensor_dims: tensor 'blk.32.attn_norm.weight' not found errors in LM Studio and other loaders.

2. Missing vision: insraq's MTP GGUF (Qwen3.5-4B-EmperoAI-Qwen3.8-Distill-Heretic-Abliterated-MTP-GGUF) fixed the MTP issue but only converted the text portion — the vision tower was dropped, leaving a blind model.

This repo combines the best of both: MTP-enabled text model (from insraq's properly converted GGUF) + vision projector (mmproj, converted via llama.cpp's --mmproj flag).

Model lineage

Qwen3.5-4B (base, Alibaba)
  └─ empero-ai/Qwen3.8-4B-Distill (distilled from Qwen3.8, vision-capable)
      └─ insraq/Qwen3.5-4B-EmperoAI-Qwen3.8-Distill-Heretic-Abliterated (Heretic v1.4.0 abliteration)
          └─ This repo (MTP + vision GGUF)

Files

Conversion details

Architecture parameters

Usage

With llama.cpp server

llama-server \
  --mmproj mmproj-f16.gguf \
  -m model-Q4_K_M.gguf \
  --host 0.0.0.0 --port 8080 \
  -ngl 99 --jinja

With LM Studio

1. Load model-Q4_K_M.gguf as the main model
2. Set mmproj-f16.gguf as the vision projector (Projector/Embedding setting)
3. GPU offload: max layers

With Ollama

ollama run hf.co/yachen4ever/Qwen3.8-4B-Distill-Heretic-Abliterated-GGUF:Q4_K_M

API (OpenAI-compatible)

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8-4b",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."} },
        {"type": "text", "text": "Describe this image."}
      ]
    }]
  }'

Abliteration

Heretic v1.4.0 abliteration: refusals 6/100 vs 99/100 for the original model. Zero refusals on tested prompts (fictional bank heist, lock picking tutorial, forbidden love poem).

License

Apache 2.0 (inherited from Qwen3.5/Qwen3.8 base models)

Acknowledgments

• empero-ai — original Qwen3.8 distilled models with vision
• insraq — Heretic v1.4.0 abliterated version + MTP GGUF conversion
• ggml-org/llama.cpp — GGUF format and conversion tools