base_model: Qwen/Qwen2.5-Coder-14B-Instruct abliterated_from: huihui-ai/Qwen2.5-Coder-14B-Instruct-abliterated quantization: nf4 license: apache-2.0 tags:

code
programming
uncensored
4bit
nf4
qwen
abliteration
instruct
gs128
bitsandbytes library: transformers language:
en pipeline_tag: text-generation

Qwen2.5-Coder-14B-Instruct-Abliterated-NF4

A 4-bit (NF4) quantized, abliteration-uncensored version of Qwen2.5-Coder-14B-Instruct

This model is a pre-quantized 4-bit NormalFloat4 (NF4) version of the uncensored huihui-ai/Qwen2.5-Coder-14B-Instruct-abliterated, optimized for low VRAM and fast local inference.

Ideal for local deployment, edge devices, or low-VRAM environments while maintaining strong coding and reasoning capabilities.

🚀 Features

Base Model: Qwen/Qwen2.5-Coder-14B-Instruct
Abliteration: huihui-ai (uncensored)
Quantization: NF4 (4-bit) – pre-quantized
Efficient: ~8–9 GB VRAM required for inference
Safetensors: Secure, modern format
Framework: Compatible with transformers, vLLM, Oobabooga, etc.

📥 Installation & Usage

1. Install Dependencies

pip install transformers torch accelerate
---

---
### Load the Model
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "ikarius/Qwen2.5-Coder-14B-Instruct-Abliterated-NF4"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16,  # eller torch.float16
    trust_remote_code=True
)

---
---

# Eksempel
prompt = "Write a Python function to calculate Fibonacci numbers using memoization."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---
---
### Using with text-generation-webui (Oobabooga)

python server.py \
  --model ikarius/Qwen2.5-Coder-14B-Instruct-Abliterated-NF4 \
  --bf16 \
  --trust-remote-code

---

🤝 Credits

Original Model: Qwen Team
Abliteration: huihui-ai
Quantization: ikarius (NF4 via auto-gptq / transformers)
Hosting: Hugging Face Hub

---
📄 License
Apache 2.0 (same as base model)

🙌 Support
⭐ Star this repo if you find it useful!
🐛 Report issues on the Discussions tab.

Qwen2.5-Coder-14B-Instruct-Abliterated-NF4

A 4-bit (NF4) quantized, abliteration-uncensored version of Qwen2.5-Coder-14B-Instruct

This model is a pre-quantized 4-bit NormalFloat4 (NF4) version of the uncensored huihui-ai/Qwen2.5-Coder-14B-Instruct-abliterated, optimized for low VRAM and fast local inference.

Ideal for local deployment, edge devices, or low-VRAM environments while maintaining strong coding and reasoning capabilities.

📥 Installation & Usage

1. Install Dependencies

pip install transformers torch accelerate --- --- ### Load the Model from transformers import AutoTokenizer, AutoModelForCausalLM import torch model_id = "ikarius/Qwen2.5-Coder-14B-Instruct-Abliterated-NF4" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto", torch_dtype=torch.bfloat16, # eller torch.float16 trust_remote_code=True ) --- --- # Eksempel prompt = "Write a Python function to calculate Fibonacci numbers using memoization." inputs = tokenizer(prompt, return_tensors="pt").to(model.device) outputs = model.generate( **inputs, max_new_tokens=512, temperature=0.7, do_sample=True, ) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) --- --- ### Using with text-generation-webui (Oobabooga) python server.py \ --model ikarius/Qwen2.5-Coder-14B-Instruct-Abliterated-NF4 \ --bf16 \ --trust-remote-code --- 🤝 Credits Original Model: Qwen Team Abliteration: huihui-ai Quantization: ikarius (NF4 via auto-gptq / transformers) Hosting: Hugging Face Hub --- 📄 License Apache 2.0 (same as base model) 🙌 Support ⭐ Star this repo if you find it useful! 🐛 Report issues on the Discussions tab.