license: other license_name: swift-open-license-1.0 base_model: d0xin/Swift-Qwen3.8-27B-Uncensored-BF16 base_model_relation: quantized library_name: llama.cpp pipeline_tag: text-generation tags:
- gguf
- quantized
- imatrix
- qwen3.8
- swift
- uncensored
- abliterated
- rtx-3090
- long-context
- speculative-decoding
- mtp
Swift-Qwen3.8-27B-Uncensored ATX-IQ4_XS-M (GGUF)
The ATX-IQ4_XS-M recipe applied to d0xin/Swift-Qwen3.8-27B-Uncensored-BF16. That model is an uncensored (abliterated) derivative of UkisAI's Swift-Qwen3.8-27B, made by rank-1 directional residual-stream ablation. The architecture, the 27.32B parameters and the 866 text and MTP tensors all match the base Qwen3.8-27B, so the recipe carries over unchanged. This build adds an importance matrix computed on this model.
The recipe was designed for a single RTX 3090 / 3090 Ti (24 GB). The goal is to fit a populated 200K-token context with an 8-bit key cache and the model's own MTP speculative head, and to decode faster per speculative round than the stock Q4 mixes on that card. The original build and its measurements are at jakeatx/Qwen3.8-27B-ATX-IQ4_XS-M-GGUF. The same recipe applied to Jackrong's Qwopus fine-tune is at jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF.
This is an uncensored model. Its refusal behavior was deliberately removed upstream, so it may produce content that the original Swift model would decline to produce. Review its output before you rely on it, and make sure your use complies with the license and applicable law. The upstream card describes the ablation and its refusal and intelligence-preservation evaluations.
| file | size | notes |
|---|---|---|
ATX-Swift-Qwen3.8-27B-Uncensored-IQ4_XS-M.gguf | 14.5 GiB (15,588,550,816 bytes) | 4.56 bits per weight, MTP layer included, text only (no vision tower) |
imatrix_swift_unc.gguf | importance matrix computed from this model, see below | |
tensor_types_ATX-4-XS.txt | the per-tensor type map used by llama-quantize (identical to the base ATX-IQ4_XS-M build) |
On the name. ATX-IQ4_XS-M means IQ4_XS as the base format for the bulk tensors, with upgrade pattern M. In llama.cpp's naming, the XS in IQ4_XS belongs to the format's name, not to a mix size: it is the 4.25 bits-per-weight super-block layout of the IQ4 codebook. On the K-quants, by contrast, the S/M/L suffix says how many tensors are lifted above the base format. This file lifts the same tensors Q4_K_M does, so it is an M-pattern mix on an IQ4_XS base.
Recipe
The Swift uncensored safetensors were converted to a BF16 GGUF with llama.cpp's converter, keeping the MTP layer. That file was then quantized with the per-tensor type map from the base ATX-IQ4_XS-M build:
| tensors | format | share of weight bytes |
|---|---|---|
| everything not listed below | IQ4_XS | ~75% |
| attn_output, ssm_out, ffn_down in the layers Unsloth's tier ladder upgrades first | Q5_0 | ~17% |
| attention K/V projections, output head | Q6_K | |
| token embedding (host side) | Q4_K | |
| the eight attention K/V tensors Q4_K_M keeps at Q8_0 (V in layers 11, 27, 31, 51, 55, 59, 63; K in 31) | Q8_0 | |
| MTP draft layer (blk.64) | Q5_0 | |
| GDN alpha / beta vectors (96 tiny) | Q8_0 |
llama-quantize --imatrix imatrix_swift_unc.gguf --tensor-type-file tensor_types_ATX-4-XS.txt \
--token-embedding-type q4_K Swift-Qwen3.8-27B-Uncensored-BF16.gguf ATX-Swift-Qwen3.8-27B-Uncensored-IQ4_XS-M.gguf iq4_xs
After quantization, every tensor's name and type was checked against the base ATX-IQ4_XS-M file, and all of them were identical. The Qwopus build carries eleven extra early-layer Q5_0 tensors to match the protections in Jackrong's own Q4_K_S. No such reference mix exists for Swift, so this build uses the base map unchanged.
Importance matrix. This build does not reuse the base-model imatrix. It computes its own, from this uncensored Swift model at Q8_0 precision, over about 226K tokens of our calibration text:
- Bartowski's
calibration_datav3(mixed prose, code, multilingual text and chat). - 64K tokens of agentic and coding prompts, plus a 32K-token retrieval-analysis prompt, from the production prompt set the recipe was tuned on.
The calibration text is the same one used for the Qwopus build. Ablation and fine-tuning shift activation statistics, so measuring them on the model being quantized translates the recipe more faithfully.
Why this mix:
- On SM86, the fastest weight format per tensor at speculative verification widths 1-5 is IQ4_XS. The 2-3 bit codebook types (IQ3_S, IQ3_XXS, IQ2_S) are instruction-bound, so they run slower despite using fewer bytes.
- Q5_0 is about 16% cheaper than Q5_K.
- Q8_0 is the only format near the memory roof.
Extra bits go where Unsloth's tier ladder puts them: attention V/K, attention output, GDN output and FFN down.
sha256 of the GGUF: 51880ce0f15aebcad7abc8e4d6273b7bf270e15353a5ddcf63e8139c571b4425.
Run it
Runtime: llamAmpere, the Qwen3.8 / RTX 3090 fork with the SM86 kernel and memory work. It is the successor of https://github.com/JakeATX/llama-cpp-qwen-ampere. Build with -DGGML_CUDA=ON -DGGML_CUDA_FA=ON -DCMAKE_CUDA_ARCHITECTURES=86. The file also loads on stock TurboQuant+ and on mainline llama.cpp, with less context headroom; mainline has no turbo3 cache. The fastest llamAmpere v0.3 configuration:
GGML_Q8_TURBO3_MMA_FUSED=1 llama-server -m ATX-Swift-Qwen3.8-27B-Uncensored-IQ4_XS-M.gguf \
-c 245760 -b 4096 -ub 1024 -t 8 -tb 8 -ngl 99 -fa on -ctk q8_0 -ctv turbo3 \
--parallel 1 --jinja --fit off \
--cache-prompt --cache-ram 8192 --ctx-checkpoints 24 --checkpoint-min-step 10240 \
--spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0 \
--spec-draft-type-k q8_0 --spec-draft-type-v q8_0 \
--spec-draft-vocab-map docs/mtp-vocab/atx_65536.txt
This is a single-user configuration that serves one request at a time. The base model's card explains the cache flags: they keep long conversations from re-prefilling, and they let you edit or regenerate a turn on this partly recurrent architecture. Lower -c if you need VRAM for something else. The upstream recommended sampling settings for thinking mode are temperature 1.0, top-p 0.95, top-k 20 and min-p 0.
This GGUF holds only the text model and its MTP head; the vision tower is not included. Speed and quality were not re-measured for this build. The recipe's measurements are on the base model's card, and the upstream card documents the Swift model's own evaluations.
License
These weights are distributed under the Swift Open License v1.0, inherited from the upstream model. Personal, research, educational, evaluation and commercial use is free for individuals and organizations with annual recurring revenue of up to US$1,000,000, including affiliates. Above that threshold, commercial use requires a separate Swift Enterprise License. Contact UkisAI for terms.
Credits
- UkisAI, for the original Swift model,
ukisai/Swift-Qwen3.8-27b. - d0xin, for the uncensored BF16 derivative,
d0xin/Swift-Qwen3.8-27B-Uncensored-BF16. - The Qwen team, for Qwen3.8.
- Unsloth, for the dynamic-quant tier ladder this recipe follows.
- Bartowski, for the calibration set.
- TheTom, for TurboQuant+.