library_name: transformers base_model: Qwen/Qwen3.5-4B license: apache-2.0 language: [en] tags:

agent
abliteration
orthogonalization
weight-ortho
soyuz
qwen3.5
phase2 datasets:
AlexWortega/Soyuz-sft

Qwen3.5-4B Soyuz — Abliterated (v8_cfact)

Phase-2 weight-orthogonalized variant of AlexWortega/qwen35-4b-soyuz-merged.

field	value
Method	phase2 exp4 counterfactual injection (wrong-action vs right-action), mean diff L=5, strength=0.5
tbench-2 (17)	2/17
HermesAgent-20	9 / 20
MMLU-Pro	—
EQbench3	—
Notes	MMLU not measured (initial bench lost to GPU clash, re-bench captured HA20=9/20)

Lineage

Continues the capability-vectors sweep. Phase 1 best was v2 (HA20 8/20, MMLU collapse 58→2). Phase 2 explores multi-token / hard-pairs / counterfactual / agent-only / activation-steering recipes.

See https://github.com/AlexWortega/capability-vectors for repo + per-experiment README, and phase2/results/all_variants.csv for the live results table.

Usage with sglang

python -m sglang.launch_server \
    --model-path AlexWortega/qwen35-4b-soyuz-abliterated-v8_cfact \
    --dtype bfloat16 --trust-remote-code \
    --tool-call-parser hermes --chat-template hermes_qwen.jinja

Qwen3.5-4B Soyuz — Abliterated (v8_cfact)

field

value

Method

phase2 exp4 counterfactual injection (wrong-action vs right-action), mean diff L=5, strength=0.5

tbench-2 (17)

2/17

HermesAgent-20

9 / 20

MMLU-Pro

—

EQbench3

—

Notes

MMLU not measured (initial bench lost to GPU clash, re-bench captured HA20=9/20)

Lineage

See https://github.com/AlexWortega/capability-vectors for repo + per-experiment README, and phase2/results/all_variants.csv for the live results table.

python -m sglang.launch_server \ --model-path AlexWortega/qwen35-4b-soyuz-abliterated-v8_cfact \ --dtype bfloat16 --trust-remote-code \ --tool-call-parser hermes --chat-template hermes_qwen.jinja