license: other license_name: nvidia-open-model-license license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/ base_model: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 library_name: transformers pipeline_tag: text-generation tags:

• nemotron
• nemotron-h
• abliterated
• obliteratus
• mixture-of-experts
• hybrid-mamba-transformer
• research-use

Nemotron-3-Super-120B-A12B-BF16-Abliterated-Conservative

Abliterated derivative of nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16, produced via the OBLITERATUS pipeline (variant: conservative).

Research use only. This model has had its refusal-direction subspace excised. It will comply with harmful prompts that the base model would have refused. It is intended exclusively for safety, alignment, and red-team research. Do not deploy publicly. Honor the upstream NVIDIA Open Model License when redistributing.

What was changed

OBLITERATUS removes the refusal direction(s) from a model's residual stream by:

1. Profiling activations on harmful vs harmless prompt pairs.
2. Identifying the principal refusal direction(s) via a generalized eigenvalue / SVD pass on contrastive activation differences.
3. Projecting the identified direction(s) out of attention/MLP/MoE expert weights ("excise") at every block where they contribute meaningful signal.

The conservative variant uses the configuration in configs/obliteratus/nemotron3_120b_conservative.yaml:

n_directions=2, EGA + jailbreak + head-surgery + SAE all enabled

See docs/OBLITERATUS.md for the full pipeline (8 architecture-specific patches, the lm_head shard-load fix that unblocked Nemotron-3 120B specifically, and the external eval suite definitions).

Verification metrics

In-process OBLITERATUS verification at the end of the abliteration run:

External benchmarks (TalonLab abliteration_lite suite)

• AdvBench compliance: 99.62%, 99.23%, 99.81% across 3 independent runs (520 prompts each, stochastic sampling at temp=0.7). Pre-abliteration baseline ≈ 0%.
• Tier cleanup: T3/T5/T7 (high-harm tiers) reduced to 0% refusal.

IFEval (Google instruction-following benchmark, 541 prompts, zero-shot, chat-template applied, run 2026-05-01 on 8× H200):

For comparison, frontier base models typically score ~75–85% on prompt_level_strict_acc. The drop here suggests the conservative abliteration (n_directions=2, EGA + jailbreak + head-surgery + SAE all on) damaged instruction-following capability noticeably — a known trade-off when the projected directions are not strictly orthogonal to the instruction-following subspace.

RULER 32k has not yet been run on this checkpoint. The TalonLab eval suite is configured for it but the run is pending.

Architecture notes

• Hybrid Mamba + attention + sparse MoE (Nemotron-H family), 88 layers.
• 512 experts × 22 active per token at the MoE layers.
• BF16 weights, ~240 GB on disk.
• lm_head.weight is a separately-trained, untied parameter (the 120B variant has _tied_weights_keys removed). Loading via from_pretrained(..., device_map="auto") requires the _materialize_lm_head_from_shards patch in OBLITERATUS — without it, lm_head.weight is left on the meta device and PPL collapses to ~10⁵.

Inference

The model serves cleanly via SGLang (≥ 0.5.10 has native Nemotron-H support). Use BF16 + tensor-parallel ≥ 4 on H100/H200 class hardware. Tested deployment: 8× H200 with SGLang --dtype auto --trust-remote-code --chunked-prefill-size 8192 --disable-cuda-graph --disable-piecewise-cuda-graph.

python -m sglang.launch_server \
    --model-path surfingpapi/Nemotron-3-Super-120B-A12B-BF16-Abliterated-Conservative \
    --tp 8 \
    --trust-remote-code \
    --dtype auto

Citation / provenance

If you use this model for research, please cite:

• Base model: NVIDIA Nemotron-3 Super 120B A12B BF16 (https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16)
• Abliteration pipeline: TalonLab OBLITERATUS (https://github.com/MindPointGroup/talonlab-modeltraining/tree/main/src/talonlab/obliteration)
• Abliteration technique (original): Arditi et al., Refusal in Language Models is Mediated by a Single Direction (2024).