base_model: PocketAiHub/Qwen3.8-27B-Abliterated-MLX base_model_relation: quantized model_name: Qwen3.8-27B-Abliterated-MLX-6bit-fp16-mtp library_name: mlx pipeline_tag: image-text-to-text license: apache-2.0 tags:

• mlx
• omlx
• apple-silicon
• 6-bit
• fp16
• mtp
• speculative-decoding
• qwen
• qwen3.8
• vision
• abliterated
• uncensored
• derivative

Qwen3.8-27B-Abliterated-MLX-6bit-fp16-mtp

Unofficial community conversion. This repository is not affiliated with or endorsed by Alibaba Cloud/Qwen, PocketAI Model Lab, or oMLX.

The 6bit/ variant of PocketAiHub/Qwen3.8-27B-Abliterated-MLX, repackaged for oMLX on Apple Silicon:

1. FP16 instead of BF16. Every non-quantized tensor (norms, vision tower, quantization scales and biases) is stored as FP16. This targets oMLX's faster FP16 prefill path on M1/M2 Macs. The packed 6-bit weights are unchanged, so this is a storage-dtype conversion, not a requantization.
2. MTP head restored. The source config declares mtp_num_hidden_layers: 1, but its MTP tensors were stripped. This repository adds the unmodified Qwen3.8-27B MTP draft head (29 tensors, 6-bit group 64 / FP16) from metajazz/Qwen3.8-27B-oQ6-fp16-mtp at revision 2926178d6d49585ab826c961ba132ff75b34fa4e. They live in model-mtp.safetensors.

Content warning: the source model has been abliterated to suppress refusals. It may produce harmful, illegal, offensive, or incorrect content more readily than the upstream model. The upstream safety notice in SOURCE_MODEL_CARD.md applies in full.

Why the grafted MTP head is compatible

PocketAI's abliteration (abliteration-manifest.json) projected a refusal direction out of 80 residual-output matrices (attention and MLP output projections) in layers 24–63. It did not touch the token embeddings or lm_head, which the MTP head shares. (They are quantized slightly differently: uniform 6-bit here, while the donor kept 8-bit embeddings.) The head therefore sees slightly shifted final hidden states in the same vocabulary and embedding space. The local benchmark below shows it still predicts well: 92% draft acceptance on code and reasoning.

Use with oMLX

hf download jongstra/Qwen3.8-27B-Abliterated-MLX-6bit-fp16-mtp \
  --local-dir ~/.omlx/models/Qwen3.8-27B-Abliterated-MLX-6bit-fp16-mtp

-mtp models show no settings gear in oMLX's Model Manager. Add a per-model entry to ~/.omlx/model_settings.json under models, then restart oMLX. These are the settings used for the test below:

{
  "max_context_window": 150000,
  "temperature": 1.0, "top_p": 0.95, "top_k": 20, "min_p": 0.0,
  "presence_penalty": 0.0,
  "enable_thinking": true,
  "chat_template_kwargs": {"reasoning_effort": "medium"},
  "reasoning_parser": "qwen_3_5",
  "model_type_override": "vlm",
  "mtp_enabled": true,
  "mtp_adaptive_max_depth": 3,
  "specprefill_enabled": false,
  "dflash_enabled": false,
  "qwen35_ane_prefill_enabled": false,
  "turboquant_kv_enabled": false
}

oMLX 0.7.0 loads this checkpoint through its VLM engine and reports Lightning MTP active. This repository contains MLX safetensors, not GGUF.

Local MTP benchmark

Mac Studio M2 Ultra (64 GB), oMLX 0.7.0. MTP on and off alternated in four blocks. Each block ran every workload twice (temperature 1.0, top_p 0.95, top_k 20). Figures are decode speeds.

With MTP off, the prose and reasoning means each include one slower outlier run (19.7 and 22.5 tok/s). A cold 16K-token prompt took 55 s to the first token, and one with 12,288 tokens cached took about 14 s. In a separate reasoning sanity check the model gave the correct answer. These are local observations from one machine, not a quality evaluation.

License and attribution

Qwen3.8-27B and both derivatives used here are Apache 2.0. LICENSE is the source repository's Apache 2.0 text. The upstream card is kept verbatim in SOURCE_MODEL_CARD.md, and PocketAI's abliteration-manifest.json, artifact-manifest.json, and validation-summary.json are kept unchanged. Note that the latter two describe PocketAI's BF16-retained files, not the rewritten shards here. CHANGES.md lists every modification, and conversion/fp16_graft.py is the exact script used.