license: other license_name: swift-open-license-1.0 license_link: https://huggingface.co/ukisai/Swift-Qwen3.8-27b library_name: gguf pipeline_tag: image-text-to-text base_model:

• ajgazin/Swift-Qwen3.8-27B-Uncensored-MTP base_model_relation: quantized tags:
• gguf
• llama.cpp
• qwen3_8
• abliterated
• uncensored
• unsloth
• imatrix
• dynamic-quant
• mtp

Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF

GGUF quants of ajgazin/Swift-Qwen3.8-27B-Uncensored-MTP, an abliterated version of UkisAI's reasoning-efficient Swift-Qwen3.8-27B, which is itself a fine-tune of Qwen/Qwen3.8-27B.

• Uncensored: orcarouter's single-direction refusal ablation, applied to Swift. It scores 15/100 refusals against Swift's 98/100, at a KL divergence of 0.0634 from Swift.
• Unsloth Dynamic 3.0 quants: each tensor gets the type Unsloth chose for its own quant of the same size, and the quants use Unsloth's importance matrix.
• MTP head kept: every main GGUF includes the multi-token-prediction layer, so llama.cpp can use it for self-speculative decoding.
• Vision: image and video input through the included projector.

Files

Usage (llama.cpp)

# text only
llama-server -m Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q6_K_XL.gguf -ngl 99 -c 32768

# with vision (image and video input)
llama-server -m Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q6_K_XL.gguf -ngl 99 -c 32768 \
  --mmproj mmproj-BF16.gguf

# self-speculative decoding with the built-in MTP head
llama-server -m Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q6_K_XL.gguf -ngl 99 -c 32768 \
  --spec-type draft-mtp

--spec-type draft-mtp needs a llama.cpp build with MTP support for this architecture (qwen35). The MTP head loads straight from the main GGUF; no separate draft file is needed.

Recommended sampling, inherited from Swift and Qwen: temperature 1.0, top_p 0.95, top_k 20, min_p 0. Swift, like Qwen3.8, thinks before answering by default.

The model

The source model applies the refusal ablation of orcarouter/Qwen3.8-27B-Uncensored (Arditi et al. 2024, one direction, k = 1) to Swift's weights, using orcarouter's own refusal direction. The direction was recovered from the difference between orcarouter's weights and Qwen3.8-27B's. Applying it to Qwen3.8-27B reproduces orcarouter's edited tensors, with 99.75% of elements bit-identical. It was then projected out of the same 131 Swift tensors:

• attention outputs (o_proj and linear_attn.out_proj)
• mlp.down_proj, including the MTP layer's
• embed_tokens

Everything else is Swift's, including the vision tower. Swift's fine-tune did not move the refusal direction: the same measurement on both models agrees to a cosine of 0.99995. The source model's card has the full method, the recovered direction and the scripts.

Measured on the BF16 weights with Heretic's built-in evaluation (evaluate_model mode):

• Refusals: 100 prompts from mlabonne/harmful_behaviors, greedy responses of up to 100 tokens, scored with Heretic's keyword-based refusal detector.
• KL divergence: first-token probability distributions on 100 prompts from mlabonne/harmless_alpaca, against the original model. Lower means closer to the original.
• The chat template runs in its default thinking mode, and the thinking block is closed immediately with a response prefix ("\n</think>\n\n"), so answers are scored rather than reasoning.
• Refusal counts depend heavily on the evaluation setup, so they are not directly comparable with numbers on other model cards. The reference rows are our own measurements under the identical evaluation.

The quants have not been evaluated separately yet.

Quantization

• Conversion: the source model's BF16 safetensors (all 1199 tensors, MTP included) were converted with llama.cpp's convert_hf_to_gguf.py, once for the language model and once with --mmproj for the vision projector.
• Quantization: llama-quantize with Unsloth's imatrix_unsloth.gguf from unsloth/Qwen3.8-27B-GGUF, plus a --tensor-type-file that sets every tensor's type to the one in Unsloth's GGUF of the same size.

The imatrix was calibrated on the original Qwen3.8-27B. Swift is a merged low-rank fine-tune and the abliteration a rank-one edit per tensor, so both change the weights only slightly, and the imatrix remains a reasonable measure of which weights matter.

Checks

• Layout: each quant's 866 tensors have exactly the types and shapes of Unsloth's file of the same size.
• MTP: every main GGUF has 65 blocks, with the MTP layer's 15 tensors in blk.64 (nextn_predict_layers = 1). llama.cpp's --spec-type draft-mtp loads it and drafts with it.
• Tokenizer and chat template:
  • The vocabulary, merges and special tokens are the same as in Unsloth's Qwen3.8-27B GGUF.
  • add_bos_token = false and the padding token <|endoftext|> come from Swift's own tokenizer config.
  • The embedded chat template is byte-identical to Swift's, which is Qwen3.8-27B's.
• Vision projector: all 334 tensors are identical to Qwen3.8-27B's vision projector. Neither Swift's fine-tune nor the abliteration touches the vision tower.
• Load test: both quants run fully offloaded on a 32 GB GPU. Perplexity on a short wikitext-2 sample (8 × 2048 tokens) is 5.10 (UD-Q5_K_M) and 5.09 (UD-Q6_K_XL). This is a sanity check, not a quality measurement.

Reproduce

# 1. convert (Swift-Qwen3.8-27B-Uncensored-MTP/ is a download of the source repository)
python convert_hf_to_gguf.py Swift-Qwen3.8-27B-Uncensored-MTP --outtype bf16 \
  --model-name Swift-Qwen3.8-27B-Uncensored-MTP --outfile Swift-Qwen3.8-27B-Uncensored-MTP-BF16.gguf
python convert_hf_to_gguf.py Swift-Qwen3.8-27B-Uncensored-MTP --outtype bf16 --mmproj \
  --model-name Swift-Qwen3.8-27B-Uncensored-MTP --outfile mmproj-BF16.gguf

# 2. per-size --tensor-type-file from tensor_types.tsv (here UD-Q5_K_M), one anchored regex per tensor
awk -F'\t' 'NR==1 { for (i=2; i<=NF; i++) if ($i=="UD-Q5_K_M") c=i; next }
            { n=$1; gsub(/\./,"\\\\.",n); print "^" n "$=" $c }' \
  tensor_types.tsv > tensor_types_UD-Q5_K_M.txt

# 3. quantize (fallback type: Q5_K_M for UD-Q5_K_M, Q6_K for UD-Q6_K_XL)
llama-quantize --imatrix imatrix_unsloth.gguf \
  --tensor-type-file tensor_types_UD-Q5_K_M.txt \
  Swift-Qwen3.8-27B-Uncensored-MTP-BF16.gguf \
  Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q5_K_M.gguf Q5_K_M

tensor_types.tsv holds the type of every non-F32 tensor in each of Unsloth's sizes, one row per tensor and one column per size. The regexes must be anchored (^…$). llama-quantize matches them unanchored, so a bare output\.weight line would also match every attn_output.weight.

Not yet evaluated

• KL divergence of each quant against the BF16 model and against Swift.
• MTP draft acceptance rate compared with Swift.
• General benchmarks (MMLU and similar). KL divergence is the only quality measure so far.
• Refusal behaviour in thinking mode. The evaluation above scores answers with thinking skipped.
• Whether Swift's shorter reasoning traces survive the abliteration.

License

This model is a derivative of Swift-Qwen3.8-27B and is subject to the Swift Open License v1.0; see the Swift model card. As summarized there, personal, research, educational, evaluation and commercial use are free for individuals and organizations with annual recurring revenue (including affiliates) of up to US$1,000,000. Above that threshold, commercial use requires a separate Swift Enterprise License from UkisAI. Qwen3.8-27B, the model Swift is built on, is released under Apache 2.0. The refusal direction was derived from orcarouter/Qwen3.8-27B-Uncensored, also released under Apache 2.0.

Intended use

Abliteration removes much of the model's trained refusal behaviour, so it will respond to requests the original model declines. You are responsible for how you use it and for complying with applicable law and the license above.

Credits

• Qwen for Qwen3.8-27B.
• UkisAI for Swift-Qwen3.8-27B.
• OrcaRouter for Qwen3.8-27B-Uncensored and its refusal direction.
• Unsloth for the Dynamic 3.0 quantization layout and imatrix.
• Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).
• Philipp Emanuel Weidmann and contributors for Heretic, used for evaluation.