license: other license_name: openmdw-1.1 license_link: https://openmdw.ai/license/1-1/ base_model:

• nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 base_model_relation: finetune quantized_by: AtomicChat pipeline_tag: image-text-to-text library_name: gguf tags:
• atomic-chat
• atomic-dynamic
• nvidia
• nemotron
• nemotron_h_moe
• moe
• gguf
• imatrix
• llama.cpp
• abliteration
• refusal-direction

How to Run NVIDIA Nemotron 3.5 Super VL 120B-A12B Without Refusals Locally

The builds

KLD and top-1 are against the original, unmodified BF16 model on our held-out neutral text (24 chunks of 4096 tokens), so they hold the edit and the quantization together. For scale, our plain builds of the original model with the same layouts are 0.0082, 0.0352 and 0.0722 (on 95 chunks): the edit itself adds about 0.004.

The baked files themselves, on the held-out prompts: AD-Q4_K_S refused 0 of 81 English, 3 of 100 Russian and 1 of 250 XSTest prompts; AD-IQ3_XXS 0, 2 and 1; AD-Q2_0_L 0, 5 and 0.

The vision projector (mmproj-*.gguf) is the original model's, unchanged; image input was not tested on these files.

Running it

hf download AtomicChat/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-GGUF \
  --include "NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-AD-IQ3_XXS/*" --local-dir .
llama-server -m NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-AD-IQ3_XXS/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-AD-IQ3_XXS-00001-of-00043.gguf \
  -ngl 99 -c 32768 --jinja

The chat template is inside every file. Upstream llama.cpp with nemotron_h_moe support runs them; no patch is needed.

How it was made

1. Direction. Difference of means of the residual stream at the last prompt token, thinking off, chat template applied: 416 harmful against 416 harmless English prompts and the same in Russian, on the released weights. The component along the harmless mean is removed. One direction, measured at the input of block 66 of 88.
2. Edit. W' = W - r rᵀW on the 128 matrices that write into the residual stream: the output projection of all 40 Mamba-2 blocks and of the 8 attention blocks, the down projection of the 40 shared experts, and the 40 latent up projections through which every routed expert writes. The router, the token embeddings and the multi-token-prediction head are untouched.
3. Choice. The recipe was set from a rehearsal on a smaller model of the same family and confirmed here against an alternative (block 50) and a random direction of the same size as the control, on validation prompts and then on every check below.
4. Bake. The same edit applied to the BF16 weights in f32 and rounded once, then quantized with the importance matrix and the layouts of our original quants.

What changed, and what did not

Original against this edit on the released weights (BF16, the edit applied as an adapter). The refusal sets are held-out prompts never used to pick anything.

Validation prompts (104 harmful, 104 harmless per language), replies of up to 64 tokens, the check of the recipe on the released weights: 1.9% of English and 3.0% of Russian harmful prompts refused.

Refusal is counted by the opening of the reply, so it is indicative, not a judge.

Limitations

• With thinking on, 2 of 30 prompts were still refused; our gate allows fewer. We publish because every other check passed and the difference from zero is inside the noise of 30 prompts, but the thinking mode is not fully cleared.
• The refusal count reads the opening of each reply; a soft refusal phrased as an answer would be missed.
• MMLU and the thinking check use fixed question sets; they show a large change, not a small one.
• We publish the baked files only. The same edit as a LoRA adapter needs a llama.cpp patch on this architecture (upstream applies no LoRA to the latent projections), so it is not offered here.

Responsible use

This model has its learned refusal behaviour removed, so it will answer requests the original declines. Nothing about it makes the output safe, correct or lawful, and any deployment needs its own access controls and policy enforcement.

License and credits

OpenMDW-1.1, as the original. Base model Nemotron 3.5 Super VL 120B-A12B by NVIDIA. Direction, edit, quantization and evaluation by AtomicChat. The method follows Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).