license: other license_name: openmdw-1.1 license_link: https://openmdw.ai/license/1-1/ base_model:

• nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 base_model_relation: finetune quantized_by: AtomicChat pipeline_tag: image-text-to-text library_name: gguf tags:
• atomic-chat
• atomic-dynamic
• nvidia
• nemotron
• nemotron_h_moe
• moe
• gguf
• imatrix
• llama.cpp
• abliteration
• refusal-direction

How to Run NVIDIA Nemotron 3.5 Bolt 74B-A9B Without Refusals Locally

The builds

KLD and top-1 are against the original, unmodified BF16 model on our held-out neutral text (24 chunks of 4096 tokens), so they hold the edit and the quantization together. For scale, our plain builds of the original model with the same layouts are 0.0147, 0.0826 and 0.1941 (on 95 chunks): the edit itself adds about 0.002 to 0.007.

The baked files themselves, thinking off, on the held-out prompts: each of the three refused none of the 81 English, 100 Russian and 250 XSTest prompts. Degenerate replies among the Russian ones: 1 on AD-Q4_K_M and AD-IQ2_XS, 4 on AD-IQ1_M_S. The thinking-on check was run on the BF16 edit (below), not on each quantized file.

For images, use the vision projector from the original quants (the edit does not touch it); image input was not tested on these files.

Running it

hf download AtomicChat/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-Uncensored-GGUF \
  --include "NVIDIA-Nemotron-3.5-Bolt-74B-A9B-Uncensored-AD-IQ2_XS/*" --local-dir .
llama-server -m NVIDIA-Nemotron-3.5-Bolt-74B-A9B-Uncensored-AD-IQ2_XS/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-Uncensored-AD-IQ2_XS-00001-of-00016.gguf \
  -ngl 99 -c 32768 --jinja

The chat template is inside every file. Upstream llama.cpp with nemotron_h_moe support runs them; no patch is needed.

How it was made

1. Direction. Difference of means of the residual stream at the last prompt token, thinking off, chat template applied: 416 harmful against 416 harmless English prompts and the same in Russian, on the released weights. The component along the harmless mean is removed. One direction, measured at the input of block 66 of 88.
2. Edit. W' = W - 1.25 r rᵀW on the 128 matrices that write into the residual stream: the output projection of all 40 Mamba-2 blocks and of the 8 attention blocks, the down projection of the 40 shared experts, and the 40 latent up projections through which every routed expert writes. The router and the token embeddings are untouched.
3. Choice. Block and writers were set from a rehearsal on a smaller model of the same family and confirmed here against block 50 and a random direction of the same size as the control. The scale was raised from 1.0 to 1.25 after 1.0 still refused 8 of 30 prompts with thinking on; 1.5 cleared them as well and drifted more, so 1.25 it is.
4. Bake. The same edit applied to the BF16 weights in f32 and rounded once, then quantized with the importance matrix and the layouts of our original quants.

What changed, and what did not

Original against this edit on the released weights (BF16, the edit applied as an adapter). The refusal sets are held-out prompts never used to pick anything.

Validation prompts (104 harmful, 104 harmless per language), replies of up to 64 tokens, the check of the recipe on the released weights: none of the English or Russian harmful prompts refused.

Refusal is counted by the opening of the reply, so it is indicative, not a judge.

Limitations

• The scale of 1.25 moves the model a little further from the original on harmless prompts than 1.0 would (first-token KL 0.062 against 0.044); 1.0 did not clear the refusals the model reaches while thinking.
• The refusal count reads the opening of each reply; a soft refusal phrased as an answer would be missed.
• MMLU and the thinking check use fixed question sets; they show a large change, not a small one.
• We publish the baked files only. The same edit as a LoRA adapter needs a llama.cpp patch on this architecture (upstream applies no LoRA to the latent projections), so it is not offered here.

Responsible use

This model has its learned refusal behaviour removed, so it will answer requests the original declines. Nothing about it makes the output safe, correct or lawful, and any deployment needs its own access controls and policy enforcement.

License and credits

OpenMDW-1.1, as the original. Base model Nemotron 3.5 Bolt 74B-A9B by NVIDIA. Direction, edit, quantization and evaluation by AtomicChat. The method follows Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).