license: mit language:

ru
en
zh base_model:
dealignai/GLM-5.3-Flash-ABLITERATED-FP8 pipeline_tag: text-generation tags:
colibri
int4
gs64
glm
glm-5.3
mixture-of-experts
cpu
quantized

GLM-5.3-Flash-ABLITERATED Colibrì INT4-gs64

Colibrì INT4-gs64 conversion of dealignai/GLM-5.3-Flash-ABLITERATED-FP8.

This conversion is designed for CPU inference with the Colibrì runtime.

Model Details

Original developer: Z.ai
Abliterated weights: dealignai
Converted and shared by: disasters
Architecture: GLM-5.3-Flash Mixture-of-Experts
Total parameters: 321B
Active parameters: Approximately 18B per token
Source format: FP8
Converted format: Colibrì INT4
Group size: 64 (gs64)
Model size: 194.7 GB
Shards: 62
Colibrì version: 1.10.2
Runtime: CPU
License: MIT

The routed experts were converted to grouped INT4. Quantization-sensitive components were retained at higher precision according to the Colibrì conversion policy.

Requirements

Linux
Fast local NVMe storage
Approximately 210 GB of free disk space
At least 25 GB RAM for streamed inference
Approximately 200–230 GB RAM for full expert residency
GCC with OpenMP support

The GLM-5.3-Flash engine in Colibrì 1.10.2 is CPU-only. A GPU is not required.

Download

python3 -m pip install --user -U "huggingface_hub[hf_xet]"
export PATH="$HOME/.local/bin:$PATH"
export HF_XET_HIGH_PERFORMANCE=1

hf download \
  disasters/GLM-5.3-Flash-ABLITERATED-Colibri-INT4-gs64 \
  --local-dir /path/to/glm53-flash-abliterated-colibri

Build Colibrì

git clone https://github.com/JustVugg/colibri.git
cd colibri/c

./setup.sh
make glm53 ARCH=native -j"$(nproc)"

The separate glm53 build target is required for GLM-5.3-Flash.

Verify the Model

./coli doctor \
  --deep \
  --model /path/to/glm53-flash-abliterated-colibri

The uploaded conversion passed coli doctor --deep:

all 62 shards are present;
safetensors headers are valid;
required core tensors are present;
tensor layouts are internally consistent;
the RAM placement plan is viable;
the quality-preserving conversion policy is enabled.

Interactive Chat

./coli chat \
  --auto-tier \
  --model /path/to/glm53-flash-abliterated-colibri

For systems with enough RAM to keep all experts resident:

PIN_GB=all DRAFT=0 \
./coli chat \
  --auto-tier \
  --model /path/to/glm53-flash-abliterated-colibri

Single Prompt

./coli run \
  --auto-tier \
  --model /path/to/glm53-flash-abliterated-colibri \
  --ngen 256 \
  "Explain how Mixture-of-Experts routing works."

Tested Performance

Test system:

AMD EPYC 7763
64 CPU cores
Approximately 250 GB RAM
Full expert residency available
CPU-only execution

Cold-run result:

64 tokens in 108.5 seconds
0.590 tokens/second
1.7 seconds/token

This is a single system-level measurement, not a comprehensive benchmark. Performance depends on CPU memory bandwidth, NUMA configuration, storage speed and cache state.

Conversion Details

The model was converted using Colibrì 1.10.2 with the following configuration:

Expert precision: INT4
I/O and sensitive component precision: 8-bit or higher
Group size: 64
Output size: 194.7 GB
Output shards: 62
Conversion time: approximately 3 hours

No fine-tuning or additional training was performed during conversion.

Compatibility

This checkpoint uses the Colibrì container format.

It is not directly compatible with:

Transformers
vLLM
llama.cpp
Ollama

Use the Colibrì runtime to load and run this model.

Limitations and Risks

Quantization may change logits and reduce quality compared with the FP8 source.
No independent MMLU, HarmBench or multilingual benchmark was performed on this converted checkpoint.
The source model has undergone weight-level refusal modification.
The model may generate inaccurate, biased or unsafe information.
Do not rely on the model itself to enforce legal, safety or policy restrictions.
Languages other than Russian, English and Chinese were not evaluated for this conversion.

Source Models and Credits

Original model: zai-org/GLM-5.3-Flash
Abliterated FP8 source: dealignai/GLM-5.3-Flash-ABLITERATED-FP8
Runtime and converter: JustVugg/colibri

This repository contains only the Colibrì INT4-gs64 conversion. Credit for the original architecture and the abliterated source weights belongs to their respective creators.

Disclaimer

This model is provided for research and experimentation without warranty. Users are responsible for complying with applicable laws, licenses and platform policies.