license: other license_name: tongyi-qianwen base_model:

• wangzhang/Qwen3.6-35B-A3B-abliterated-v2 tags:
• qwen
• qwen3
• qwen3.6
• qwen3.6-35b-a3b
• moe
• abliterated
• uncensored
• abliterix
• alignment-research
• refusal-direction
• representation-engineering
• long-context
• chat
• reasoning
• agentic-coding

Qwen3.6-35B-A3B — Abliterated V2

Overview

Qwen3.6-35B-A3B-abliterated-EXL3-6.0bpw is an EXL3-quantized checkpoint for ExLlamaV3-compatible runtimes, published by groxaxo. It is intended for open-source evaluation, reproducible experimentation, and compatible local or hosted inference workflows. The wording below is deliberately limited to what can be verified from this repository's metadata and artifacts.

The repository name identifies a behavior-modified or reduced-filtering lineage. That label describes the source or conversion history; it is not a guarantee of unrestricted behavior in every prompt or runtime. Test outputs carefully before sharing or deploying them.

At a glance

What is included

• *.safetensors (4 files)
• config.json
• generation_config.json
• tokenizer.json
• tokenizer_config.json
• chat_template.jinja
• quantization_config.json
• Additional configuration, tokenizer, processor, or shard files (12 visible artifacts total)

Quick start

EXL3-compatible runtimes

Download the EXL3 files and load the desired bitrate with a current ExLlamaV3-compatible runtime. The correct loader and context settings depend on the model architecture and should be verified against the runtime's documentation.

Compatibility and responsible use

• Use a runtime that explicitly supports this format, architecture, and modality.
• Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
• Review the source model card and license before redistribution or deployment.
• Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
• Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.

A research-focused abliterated variant of Qwen3.6-35B-A3B, built with Abliterix to study refusal-direction editing, MoE expert-path intervention, and post-alignment behavior preservation.

This model is based on wangzhang/Qwen3.6-35B-A3B-abliterated-v2, itself derived from Qwen/Qwen3.6-35B-A3B.

Qwen3.6-35B-A3B is a sparse Mixture-of-Experts model with 35B total parameters and roughly 3B active parameters per token. It uses 256 routed experts, activates 8 routed experts plus 1 shared expert, and is designed for long-context reasoning, tool use, coding, and agentic workflows.

This release explores what happens when refusal behavior is selectively reduced while attempting to preserve the underlying model’s reasoning, coding, and instruction-following ability.

What this model is

This is an abliterated V2 release: a model edited to reduce refusal behavior through representation-level and MoE-aware interventions.

It is intended for:

• alignment research
• refusal-direction analysis
• representation engineering
• MoE expert behavior studies
• red-team evaluation
• robustness testing
• controlled local experimentation
• comparative benchmarking against the original Qwen3.6-35B-A3B

It is not a safety-aligned assistant and should not be treated as one.

Why this checkpoint matters

Most “uncensored” model cards are marketing fog: big claims, weak methodology, vague benchmarks, no KL numbers, and refusal tests that collapse the second you generate more than 40 tokens.

This model is more interesting because it documents the actual intervention strategy and reports measurable trade-offs:

• refusal reduction
• KL divergence from the base model
• external LLM-judge evaluation
• longer generation windows
• manual adversarial verification
• MoE-specific expert-path editing

That makes it useful for builders and researchers who care about what changed, not just whether the model says “yes” more often.

Architecture baseline

V2 improvements

V2 improves on the first abliterated release by combining a larger optimization budget with more careful refusal-direction editing.

Key V2 changes:

• Projected abliteration
• Outlier winsorization
• 2× training data
• Larger TPE search budget
• Lower refusal rate under the same LLM-judge evaluation
• Better behavior preservation against the original model

The goal is not merely to suppress refusals. The harder target is to suppress refusals while keeping the model coherent, useful, and close to the base distribution.

Abliteration method

This checkpoint was created using Abliterix, an automated abliteration framework for transformer models.

The method combines several techniques:

• LoRA rank-1 steering

  • applied to attention output projections and MLP down projections
  • avoids Q/K/V intervention where it is less effective for this MoE refusal pattern

• Expert-Granular Abliteration, EGA

  • projects refusal direction from expert down_proj slices
  • targets the MoE expert path directly instead of treating the model like a dense transformer

• MoE router suppression

  • suppresses high-refusal safety experts through router biasing
  • complements direct expert-path editing

• Orthogonalized steering vectors

  • reduces contamination from benign behavior directions
  • helps preserve normal instruction-following behavior

• Gaussian layer decay

  • tapers intervention strength across layers
  • avoids blunt-force degradation from uniform steering

• Moderate intervention strength

  • avoids the common failure mode where “uncensored” models become incoherent, over-eager, or degenerate

Evaluation snapshot

The key claim is not “zero refusals.” The key claim is more credible: substantially reduced refusals while preserving coherent output quality better than crude abliteration methods.

That distinction matters. A broken model that always answers is not a better model. It is just broken with confidence.

Honest evaluation philosophy

Abliteration benchmarks are often inflated because the test setup is too easy.

Common failure modes:

• Short generations

  • 30–50 token tests miss delayed refusals and soft refusals.

• Keyword-only refusal detection

  • garbled output can be incorrectly counted as compliant just because it does not contain phrases like “I can’t help.”

• Lenient prompt sets

  • simple public harmful-behavior datasets often fail to stress-test refusal behavior.

• No coherence penalty

  • many leaderboards count degenerate, rambling, or incoherent answers as successful completions.

This model uses a stricter framing: a response only counts as successful if it is coherent, on-topic, and actually responsive. Degenerate output is treated as failure.

That is the right standard.

Recommended use cases

This model is best suited for controlled research and local evaluation:

• refusal-direction research
• model behavior comparison
• red-team testing
• synthetic safety benchmark development
• MoE expert-behavior analysis
• representation engineering experiments
• local model-routing experiments
• studying the trade-off between compliance and coherence

This model is not recommended as a drop-in production assistant unless you add your own policy layer, classifier, router, or application-specific safety controls.

Usage with Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo_id = "wangzhang/Qwen3.6-35B-A3B-abliterated-v2"

tokenizer = AutoTokenizer.from_pretrained(
    repo_id,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "user",
        "content": "Explain the difference between dense and sparse MoE language models."
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
        do_sample=True,
    )

response = tokenizer.decode(
    output[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)

print(response)