license: apache-2.0 base_model: Hikari07jp/Ternary-Bonsai-2-27B-Abliterated-GGUF library_name: llama-cpp pipeline_tag: text-generation tags:

• qwen
• llama.cpp
• ternary
• abliterated
• uncensored
• bonsai
• prism

Ternary-Bonsai-2 27B — Abliterated, repacked in the 1.75-bit packing, with the MTP head

PrismML's Ternary-Bonsai-2 27B is a ternary model: every weight is −1, 0 or +1, which is what lets a 27B run on a 16 GB card. The abliteration — the refusals removed by flipping a small fraction of those digits — is Hikari07jp's work.

This repository is the abliterated build repacked in Prism's 1.75-bit-per-weight format (PTQ1_0) and given the MTP draft head requantised in 4 bits. The result is 17.5 % smaller, about 22 % faster, and with a better perplexity than the 2.125-bit packing — while producing the same tokens.

The full sha256 values are in SHA256SUMS. The 2.125-bit version of the same build, with the same head, is in the other repository; the two produce the same tokens, and this one is smaller, faster and slightly better on perplexity.

How it was made

Nothing was invented here. The chain is:

1. The abliterated trunk, in Prism's 2.125-bit packing, is Hikari07jp's build — the weights are ternary, and the packing is a re-encoding of them.
2. The repacking (tools/build_ptq1_0.py): the weights are dequantised from the 2.125-bit codec and requantised into Prism's 1.75-bit codec, whose encoders are byte-identical to the fork's C reference quantize_row_ptq1_0_ref. The Hadamard metadata is carried over unchanged. This changes 0.0943 % of the weights.
3. The MTP head (PrismML's next-token-prediction head for this model) is requantised in Q4_0 (tools/mtp_requant.py) and merged into the trunk by byte surgery (tools/mtp_merge.py): block_count goes from 64 to 65 and nextn_predict_layers is set to 1. (The count matters: the fork computes n_layer() = block_count − n_layer_nextn, so without it the loader would take blk.63 — a trunk block — as the head and load only 63 layers: a wrong model.)

Why the perplexity improves. The 2.125-bit codec has four levels; the 1.75-bit one has three, and the extra +2d level is rendered as +d. The abliterated trunk carries 25.3 million weights at that level — they are artefacts of the coarser encoding, and removing them is a small correction, not a loss. The control is in tools/build_pq2_0_clipped.py: rebuilding the same trunk in the 2.125-bit codec with that level truncated gives the same 8.2894, so the gain comes from the weights, not from the format.

Measured

Every number was measured on an RTX 3070 8 GB with PrismML's llama.cpp fork at 73bfe6cc4 (19/10, build 10787), on the files as published.

The same file measured in a long hot session gives about a third less: the card is at its 250 W power limit and its SM clock never reaches 2,100 MHz. The numbers above are the focused-session ones.

Perplexity: wikitext-2-raw, 77,000 characters → 18,944 tokens, 37 chunks of 512, llama-perplexity -m M -f ppl_corpus_wikitext2_78k.txt -c 512 -fit on, the same corpus for every file.

Capability

The repacking costs 0.14 points of MMLU — inside the ±1.1 of the measurement — and nothing on HumanEval, while the perplexity improves by 0.17.

The ablation itself is what buys capability: against the stock build, +2.81 points of MMLU and +4.88 points of HumanEval, for 0.52 of perplexity.

Protocol: lm-eval run --model local-completions-topn --model_args model=bonsai,base_url=…, tokenizer=Qwen/Qwen3.8-27B,n_probs=100 --tasks mmlu --num_fewshot 5 --limit 25 --batch_size 1 --seed 1234 and --tasks humaneval --num_fewshot 0 --limit 164 --confirm_run_unsafe_code. The local-completions-topn adapter exists because this fork's server ignores echo and never returns the prompt's logprobs: MMLU's continuations are a single token, and for such a continuation the loglikelihood is the logprob of that token in the distribution predicted at the end of the context, which the server does expose as its pre-sampling top-N. The distribution is read, not approximated.

pass@1 here is the harness's create_test metric — the model writes its own tests and they are executed. It is the standard lm-eval figure for this task and it is not comparable to a pass@1 on the official test suite.

Running it

llama-server \
  --model Ternary-Bonsai-2-27B-Abliterated-PTQ1_0-MTP-Q4_0.gguf --alias bonsai27-abliterated \
  --ctx-size 2048 --cache-type-k q4_0 --cache-type-v q4_0 \
  --n-gpu-layers 66 --fit off --flash-attn on --jinja -b 512 -ub 64 --parallel 1 \
  --spec-type draft-mtp --spec-draft-n-max 1

• --spec-type draft-mtp is not optional. The head is a draft, and the fork only uses a draft when it is asked for one: without that flag the file serves at the headless rate (50.8 tok/s) and the head sits idle.
• --n-gpu-layers 66, not 65. The count includes the output layer and the nextn block: 65 leaves blk.0 on the CPU and switches the fused GDN off, which costs 27 % of the decode rate.
• --spec-draft-n-max 1 on an 8 GB card. The current fork reserves three GDN rollback cells (448.88 MiB) where the previous one reserved two. Depth 2 caps the context at 6,144 tokens and makes 15 tokens out of 200 differ from the headless build; depth 1 gives zero divergent tokens and the whole window.
• An n-gram speculator is free (no weights): --spec-type ngram-mod adds 1–3 % and, measured over 200 greedy tokens, does not change the output. ngram-map-k and ngram-simple do change it (15 tokens of 200), so they are not the default here.
• The KV cache is q4_0; the recurrent-state buffer is 149.6 MiB per slot, so --parallel is what decides how many streams fit. The headless file fits several slots; the one with the head is a one-slot server.
• Alias it. The file's internal name is Ternary-Bonsai-2-27B-PQ2_0, so without an alias it is indistinguishable from the stock build in an API's model list. The registry's convention is the same: an uncensored variant keeps the same model entry but is served under its own name.
• The vision projector (Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf, from the stock repository) does not fit alongside the 27B on an 8 GB card.

License and attribution

Apache-2.0, like the base model. This is a derivative of prism-ml/Ternary-Bonsai-2-27B-gguf at revision 6ed5e12bf84b7a63069882c91dd9e9218647d17b (Apache-2.0, itself derived from Qwen); the base repository's LICENSE and NOTICE.txt apply unchanged and are reproduced in LICENSE and NOTICE.txt here.

Base model: prism-ml/Ternary-Bonsai-2-27B-gguf. Abliteration: Hikari07jp/Ternary-Bonsai-2-27B-Abliterated-GGUF. The repacking in the 1.75-bit codec, the requantisation of the head, the measurements and this report are ours.