language:

en
th tags:
qwen3.5
qwen3_5
gdn
linear-attention
hybrid
sft
lora
fine-tuned
sme
tool-calling
agent
business
multimodal
gguf
mtp license: apache-2.0 base_model: hotdogs/Qwen3.8-27B-abliterated-sme-preview pipeline_tag: text-generation

Qwen3.8-27B-Abliterated-SME-Preview-MTP-GGUF

GGUF quantization of hotdogs/Qwen3.8-27B-abliterated-sme-preview — an SME business assistant fine-tuned on document-analysis + tool-calling data. Converted for llama.cpp with the MTP (Multi-Token Prediction) head preserved.

Base: hotdogs/Qwen3.8-27B-abliterated-sme-preview
Method: Quantization to GGUF, MTP head intact (mtp_num_hidden_layers=1)
Format: .gguf (f16 source, quantized to Q4_K_M / IQ3_M etc.)
Multimodal: vision wrapper present — use as text-only via apply_chat_template

⚡ Quick Results (inherited from base)

Benchmark	Base	SME-Preview	Δ
ARC-Challenge acc	0.5667	0.5700	+0.003
MMLU	0.8477	0.8449	−0.003
GSM8K (5-shot)	0.6000	0.8100	+0.210

GGUF conversion preserves these numbers within ±0.01.

🚀 Usage

# Download a quant
huggingface-cli download hotdogs/Qwen3.8-27B-abliterated-sme-preview-mtp-GGUF \
  --include "*.gguf" --local-dir ./model

# Run
./llama-cli -m ./model/model-Q4_K_M.gguf \
  -c 8192 -ngl 99 \
  -p "Which rows in expenses_2025.csv have department = Sales?"

Requires a llama.cpp build with qwen35 architecture support (GDN + linear-attention). Do not pass --no-mtp — the model has MTP tensors.

⚠️ Notes

MTP preserved: block_count=64+1, do not strip MTP tensors
BF16→f16: minor precision loss acceptable for inference
Multimodal: vision wrapper present; for text-only use apply_chat_template

📜 License

Apache-2.0. Base: hotdogs/Qwen3.8-27B-abliterated-sme-preview

Qwen3.8-27B-Abliterated-SME-Preview-MTP-GGUF

Base: hotdogs/Qwen3.8-27B-abliterated-sme-preview

Method: Quantization to GGUF, MTP head intact (mtp_num_hidden_layers=1)

Format: .gguf (f16 source, quantized to Q4_K_M / IQ3_M etc.)

Multimodal: vision wrapper present — use as text-only via apply_chat_template

Benchmark

Base

SME-Preview

ARC-Challenge acc

0.5667

0.5700

+0.003

MMLU

0.8477

0.8449

−0.003

GSM8K (5-shot)

0.6000

0.8100

+0.210

🚀 Usage

# Download a quant huggingface-cli download hotdogs/Qwen3.8-27B-abliterated-sme-preview-mtp-GGUF \ --include "*.gguf" --local-dir ./model # Run ./llama-cli -m ./model/model-Q4_K_M.gguf \ -c 8192 -ngl 99 \ -p "Which rows in expenses_2025.csv have department = Sales?"

Requires a llama.cpp build with qwen35 architecture support (GDN + linear-attention). Do not pass --no-mtp — the model has MTP tensors.