license: apache-2.0 library_name: mlx pipeline_tag: image-text-to-text base_model:

• OS-Software/Ternary-Bonsai-2-27B-Uncensored-Heretic-v2-GGUF tags:
• mlx
• mlx-vlm
• ternary
• 2-bit
• bonsai
• heretic
• uncensored
• conversational
• image-text-to-text
• tool-calling
• apple-silicon

Ternary Bonsai 2 27B Uncensored Heretic v2 — MLX 2-bit

Community MLX conversion of OS-Software/Ternary-Bonsai-2-27B-Uncensored-Heretic-v2-GGUF, with the unchanged vision tower from Prism ML's Bonsai 2 MLX pack. The Safetensors weights occupy 8,595,478,392 bytes, approximately 8.60 GB (8.01 GiB), including vision.

The language weights come from the Heretic v2 PQ2_0 model. They were repacked into MLX affine 2-bit/group-128 storage without additional quantization or training. This repository is a community conversion, with the original developers credited below.

Model and format

“2-bit” describes the MLX storage format. The underlying low-bit language weights remain ternary. Metadata, group scales, biases, higher-precision state tensors, and the FP16 vision tower also take space.

Runtime requirement

Use mlx-vlm 0.7.2 with native prism_hadamard_qwen35 support. This model stores its language weights in a rotated basis. The loader must apply the matching Hadamard activation transforms and embedding lookup. Generic MLX or Transformers loaders that omit these operations can produce incorrect output. Keep config.json, hadamard.json, tokenizer and processor files together with the weights.

The model files are intended for Apple Silicon Macs (M1 or later) with sufficient unified memory and a macOS/Python build supported by the pinned runtime. They are not specific to M1 hardware. This conversion was created and tested on an Apple M1 Max with 64 GiB unified memory; other Mac models have not been tested for this release. Merely being able to run MLX is insufficient: the loader must support this model's Hadamard-aware architecture. See the MLX installation requirements and the pinned versions below. Memory use depends on context length, image resolution and workload; the weight-file size alone is not a RAM requirement.

The tested environment used Python 3.11 on Apple Silicon with:

mlx==0.32.2
mlx-vlm==0.7.2
transformers==5.14.1

Newer versions may work, but were not validated for this release. The older runtime/ helper bundled with the original Prism pack is not required by this schema 2 native pack.

Run locally

Download this repository into a directory named bonsai-heretic-mlx, including the weights and all configuration, tokenizer, processor, and Hadamard files. Change into that folder and create a Python environment:

cd bonsai-heretic-mlx
python3.11 -m venv .venv-heretic
source .venv-heretic/bin/activate
python -m pip install -r requirements.txt
python -m mlx_vlm.generate \
  --model . \
  --prompt "What is the capital of France? Answer with only the city name." \
  --max-tokens 64 \
  --temperature 0

That short command uses the native CLI's default of thinking disabled. For serving:

python -m mlx_vlm.server \
  --model . \
  --host 127.0.0.1 \
  --port 8083 \
  --enable-thinking

The native server exposes an OpenAI-compatible API at http://127.0.0.1:8083/v1. Read /v1/models to obtain the model ID used by your server. Local paths may appear as IDs when running the native server directly.

The included optional serve.py uses the same native runtime and provides a stable model ID based on the folder name:

python serve.py --model . --host 127.0.0.1 --port 8083 --enable-thinking

Run either server command. The adapter only translates model identity at the HTTP boundary; generation remains native.

For a short text request, run this in another terminal with Python available:

import json
from urllib.request import Request, urlopen

base = "http://127.0.0.1:8083/v1"
with urlopen(base + "/models") as response:
    models = json.load(response)["data"]
model_id = next((m["id"] for m in models if m.get("loaded")), models[0]["id"])
payload = {
    "model": model_id,
    "messages": [{"role": "user", "content": "Explain ternary model weights briefly."}],
    "max_tokens": 256,
    "temperature": 0.7,
    "top_p": 0.8,
    "top_k": 20,
    "enable_thinking": False,
    "stream": False,
}
request = Request(
    base + "/chat/completions",
    data=json.dumps(payload).encode(),
    headers={"Content-Type": "application/json"},
)
with urlopen(request) as response:
    result = json.load(response)
print(result["choices"][0]["message"]["content"])

For image input, use an OpenAI image_url content part; data URLs are supported. For tool use, send an OpenAI tools array, execute the returned function in your application, and send its result back as a tool message with the matching tool_call_id. Thinking can be selected per request with enable_thinking; the sample disables it for a short answer. Model loading and image processing require additional memory beyond the weight-file size.

Conversion and provenance

The source is pinned to:

Repository: OS-Software/Ternary-Bonsai-2-27B-Uncensored-Heretic-v2-GGUF
Revision:   6db27547a8f25667cdb43a28d15cc5d33937b7dc
File:       Ternary-Bonsai-2-27B-Uncensored-Heretic-v2-PQ2_0.gguf
SHA256:     9ba04345402584cf7f581ff72c00ea8925a35f3d78ccd34ca3eb195fe552393c

The conversion preserves the PQ2_0 ternary codes and group scales and repacks them into MLX affine storage. Codes 0, 1, and 2 represent -s, 0, and +s through scale=s and bias=-s. Hadamard signs and transforms are preserved. No additional quantization was performed.

The SSM A_log tensors are reconstructed as FP32 log(-stored GGUF SSM A), following the Prism runtime convention. Thus “lossless repacking” refers to the ternary codes and scales; it does not claim that this logarithmic reconstruction is a bit-exact inverse of an unavailable pre-GGUF checkpoint.

The output contains 2,390 tensors, of which 2,057 are language tensors. Tokenizer IDs and BPE merge ranks matched the source. The converter was validated against an already installed original Bonsai 2 GGUF/MLX pair: all 2,057 reconstructed language tensors matched that official MLX pack bit for bit.

The Heretic and original GGUF projectors had identical SHA256 hashes, permitting reuse of the original MLX vision tower and processor files unchanged. The Prism base MLX template revision is fcba37d2117a7077eac6b613b2668d14d9779edd. Its Safetensors SHA256 was 130de5925082c168b7866b2e91b52e44abbafc99017e3ca352b77b5b55a269ed; the corresponding GGUF projector SHA256 was 6807ede61d570bb86ba34b756a0fa109edc33668604de867c6ea6d8f1d631903.

See conversion.json for the provenance manifest, reproduce.md for conversion instructions, and SHA256SUMS for publication-file checksums. The original-pair validation is preserved in evidence/mlx-base-conversion-validation.json.

Verification

Functional smoke checks completed on 2026-10-09, on an Apple M1 Max with 64 GiB unified memory, using the tested stack above and the Bonsai demo's model-ID adapter around the native server:

The weather response was supplied by the test harness; the model did not fetch live weather. These tests used temperature 0, thinking disabled, and a maximum of 192 output tokens. They verify basic text, image, and tool plumbing. They do not constitute a benchmark or measure OCR, broad vision accuracy, long-context reliability, reasoning quality, or refusal rates. Results published for the original Bonsai model or the source GGUF are not presented as measurements of this MLX conversion.

The requests, responses, and checks are retained in evidence/mlx-verification.json.

Intended use and limitations

OS-Software describes Heretic v2 as a decensored derivative with an OT-Ridge LoRA merged into the ternary language weights. The source publisher recommends research and experimentation, including alignment evaluation and red-teaming, and advises against public end-user service deployment.

This conversion carries the source model's reduced safety alignment. Outputs can be inaccurate, biased, offensive, or harmful and should be checked before use. The label “uncensored” does not establish universal refusal-free behavior. The publisher reports that vision was not evaluated for its release; this conversion adds only the simple image smoke check documented above. A comprehensive independent evaluation of the converted Heretic model has not been performed.

Credits and license

• OS-Software: Heretic v2 language-weight modification and source GGUF publication.
• Prism ML: Bonsai 2 ternary model, Hadamard-aware representation, original MLX vision pack, and compatible runtime work. Created using Bonsai by Prism ML.
• Qwen / Alibaba Cloud: the underlying Qwen model lineage, identified by the retained upstream notice as Qwen3.8-27B.
• p-e-w: Heretic, acknowledged by the source publisher.
• Bonsai demo: conversion and runtime reference project.

The model is distributed under Apache 2.0. The original LICENSE and attribution notice are retained. Changes made for this distribution are the PQ2_0-to-MLX repack, native schema 2 packaging, reconstructed SSM A_log tensors, and publication documentation and provenance metadata. The source-model use guidance above is attributed to its publisher; the governing redistribution terms are in LICENSE.