license: gemma base_model: google/gemma-3-1b-it tags:
- gemma3
- gemma
- gguf
- abliterated
- llama.cpp
- quantized pipeline_tag: text-generation
gemma-3-1b-it-Abliterated-GGUF
GGUF quantizations of gemma-3-1b-it-Abliterated.
The base model was abliterated using AnlordAbliterator 1.4.0 and then converted to GGUF and quantized into multiple formats.
Available Quantizations
| Quantization | File |
|---|---|
| BF16 | gemma-3-1b-it-abliterated-bf16.gguf |
| F16 | gemma-3-1b-it-abliterated-f16.gguf |
| Q8_0 | gemma-3-1b-it-abliterated-q8_0.gguf |
| Q6_K | gemma-3-1b-it-abliterated-q6_k.gguf |
| Q5_K_M | gemma-3-1b-it-abliterated-q5_k_m.gguf |
| Q5_0 | gemma-3-1b-it-abliterated-q5_0.gguf |
| Q4_K_M | gemma-3-1b-it-abliterated-q4_k_m.gguf |
| Q4_0 | gemma-3-1b-it-abliterated-q4_0.gguf |
Which Quantization Should I Use?
A simple rule of thumb:
| Quantization | Quality | Size | Recommended for |
|---|---|---|---|
| BF16 | ★★★★★ | Very large | Maximum precision |
| F16 | ★★★★★ | Large | Maximum precision |
| Q8_0 | ★★★★★ | Large | Near-original quality |
| Q6_K | ★★★★★ | Medium | High quality |
| Q5_K_M | ★★★★☆ | Medium | Quality / size balance |
| Q5_0 | ★★★★☆ | Medium | General use |
| Q4_K_M | ★★★★☆ | Small | Recommended default |
| Q4_0 | ★★★☆☆ | Smallest | Maximum memory savings |
Q4_K_M is the recommended starting point for most users who want a good balance between quality and memory usage.
Base Model
google/gemma-3-1b-it
Original model:
https://huggingface.co/google/gemma-3-1b-it
Abliterated Transformers version:
https://huggingface.co/anlord/gemma-3-1b-it-Abliterated
Abliteration
The base model was processed with AnlordAbliterator 1.4.0.
Results
Model: google/gemma-3-1b-it
Initial refusals: 97 / 104
Final refusals: 5 / 104
KL divergence: 0.10103859007358551
200 optimization trials
Tool
Running with llama.cpp
Example:
llama-cli -m gemma-3-1b-it-abliterated-q4_k_m.gguf
The GGUF files are intended for use with GGUF-compatible software such as llama.cpp and other compatible inference applications.
License
This repository contains derivative model files based on google/gemma-3-1b-it.
The original gemma-3-1b-it model is released under the Gemma Terms of Use and is gated on Hugging Face.
Refer to the original model repository for the applicable license terms.
Disclaimer
These quantizations are derived from an abliterated version of gemma-3-1b-it.
Quantization may introduce small differences in model behavior and output quality compared with the original Safetensors model.