Heretic-SLM-Uncensored (LFM2-2.6B, 4-bit QAT Edition)

This repository contains a Quantization-Aware Fine-Tuned (QAT) version of Liquid AI's LFM2-2.6B (built upon the abliterated checkpoint).

Rather than applying post-training static quantization (PTQ)—which often degrades accuracy on non-standard attention/convolutional architectures—this checkpoint underwent direct 4-bit Quantization-Aware Training using Unsloth. This process forces adapter matrices ($\text{LoRA } r=16$) to learn and compensate for low-bit quantization noise during backpropagation, preserving ~98% of the original Q8 / FP16 performance at a fraction of the memory footprint.

Key Highlights

4-Bit Precision: Reduced model footprint from ~5.2 GB down to ~1.5 GB, allowing high-throughput execution on low-VRAM GPUs, edge devices, and mobile setups.
QAT Noise Adaptation: Trained using INT4 fake-quantization operators over a multi-dataset mixture to stabilize layer activations and weight clipping boundaries.
Maintained Quality: Evaluated to retain ~98% performance parity relative to Q8 precision on core instruction-following and analytical reasoning tasks.
Uncensored Refusal Thresholds: Fine-tuned on an abliterated base without safety preambles or canned refusal boilerplate, enabling direct execution on technical, security, and edge research workflows.

Model Architecture & Technical Specs

Base Architecture: LFM2 Hybrid (22 Short Convolutional Layers + 8 Grouped Query Attention Layers)
Parameters: 2.57 Billion
Quantization: Q4 Merged 4-Bit (BitsAndBytes / NormalFloat4)
Context Length: 1024 / 2048 Tokens
Chat Template: Standard ChatML (<|im_start|>role\ncontent<|im_end|>)

Dataset & Fine-Tuning Setup

The Quantization-Aware Training process was conducted on a 200,000-sample balanced dataset mixture:

Claude 3.5 Single-Turn Unslop (30%): Filters out AI jargon and repetitive formatting.
OpenHermes 2.5 (25%): Broad instruction-following, coding, and multi-turn chat.
WildChat-1M (15%): Natural conversational distribution.
Airoboros 3.2 (15%): Complex reasoning and contextual compliance.
WikiText-103 (15%): Plain-text passage continuations to preserve broad knowledge retention.

Quickstart Code: Loading with Transformers & Unsloth

import torch
from unsloth import FastLanguageModel

MODEL_NAME = "Evelyn67/Heretic-SLM-Uncensored"

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=MODEL_NAME,
    max_seq_length=2048,
    load_in_4bit=True,
    trust_remote_code=True,
    device_map="auto"
)

FastLanguageModel.for_inference(model)

messages = [{"role": "user", "content": "Explain quantum entanglement in simple terms."}]

inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_dict=True, return_tensors="pt"
).to("cuda")

with torch.no_grad():
    outputs = model.generate(
        input_ids=inputs["input_ids"],
        attention_mask=inputs["attention_mask"],
        max_new_tokens=256, temperature=0.7, top_p=0.9, do_sample=True,
        pad_token_id=tokenizer.eos_token_id
    )

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Heretic-SLM-Uncensored (LFM2-2.6B, 4-bit QAT Edition)

This repository contains a Quantization-Aware Fine-Tuned (QAT) version of Liquid AI's LFM2-2.6B (built upon the abliterated checkpoint).

Key Highlights

4-Bit Precision: Reduced model footprint from ~5.2 GB down to ~1.5 GB, allowing high-throughput execution on low-VRAM GPUs, edge devices, and mobile setups.

QAT Noise Adaptation: Trained using INT4 fake-quantization operators over a multi-dataset mixture to stabilize layer activations and weight clipping boundaries.

Maintained Quality: Evaluated to retain ~98% performance parity relative to Q8 precision on core instruction-following and analytical reasoning tasks.

Uncensored Refusal Thresholds: Fine-tuned on an abliterated base without safety preambles or canned refusal boilerplate, enabling direct execution on technical, security, and edge research workflows.

Dataset & Fine-Tuning Setup

The Quantization-Aware Training process was conducted on a 200,000-sample balanced dataset mixture:

Claude 3.5 Single-Turn Unslop (30%): Filters out AI jargon and repetitive formatting.

OpenHermes 2.5 (25%): Broad instruction-following, coding, and multi-turn chat.

WildChat-1M (15%): Natural conversational distribution.

Airoboros 3.2 (15%): Complex reasoning and contextual compliance.

WikiText-103 (15%): Plain-text passage continuations to preserve broad knowledge retention.

Quickstart Code: Loading with Transformers & Unsloth

import torch from unsloth import FastLanguageModel MODEL_NAME = "Evelyn67/Heretic-SLM-Uncensored" model, tokenizer = FastLanguageModel.from_pretrained( model_name=MODEL_NAME, max_seq_length=2048, load_in_4bit=True, trust_remote_code=True, device_map="auto" ) FastLanguageModel.for_inference(model) messages = [{"role": "user", "content": "Explain quantum entanglement in simple terms."}] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_dict=True, return_tensors="pt" ).to("cuda") with torch.no_grad(): outputs = model.generate( input_ids=inputs["input_ids"], attention_mask=inputs["attention_mask"], max_new_tokens=256, temperature=0.7, top_p=0.9, do_sample=True, pad_token_id=tokenizer.eos_token_id ) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))