base_model: XHToken/Spark-X2.6-1.7B library_name: transformers pipeline_tag: text-generation tags:

• spark
• spark-x2.6
• heretic
• abliterated
• uncensored
• decensored
• text-generation
• conversational
• causal-lm
• optuna
• custom_code
• roleplay
• reasoning
• edge-ai
• pytorch

Spark-X2.6-1.7B-Abliterated

OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated is an abliterated variant of XHToken/Spark-X2.6-1.7B created using Heretic (v2.0.0.dev0) across 150 Optuna Pareto optimization trials.

By projecting out refusal directions across the residual stream while optimizing for minimal KL-divergence drift against the base model, this checkpoint removes synthetic refusal behavior while preserving reasoning, coding, and instruction-following capabilities.

How It Was Done (Methodology)

Running Heretic (v2.0.0.dev0) on XHToken/Spark-X2.6-1.7B required solving two architectural and environment challenges:

1. Per-Layer Hidden State Extraction in modeling_spark.py: The base Spark custom modeling code only returned last_hidden_state rather than the tuple of intermediate layer hidden states required by Heretic to compute per-layer refusal directions. A temporary local copy of modeling_spark.py was patched to accumulate and return (hidden_states,) across every decoder_layer in self.layers during the Heretic run, and the clean upstream modeling_spark.py was restored prior to export.
2. Bridging transformers==4.57.1 with Heretic v2.0.0.dev0: Spark-X2.6-1.7B requires transformers==4.57.1 (tokenizers==0.22.2), whereas Heretic v2.0.0.dev0 installs huggingface_hub>=1.33.0 and kernels. On-disk patches were applied to dependency_versions_table.py, hub_kernels.py, utils/hub.py, and dynamic_module_utils.py to reconcile version checks and run non-interactively.

Quickstart (Transformers)

Note: As recommended by the base model authors, use transformers==4.57.1.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if (torch.cuda.is_available() and torch.cuda.is_bf16_supported()) else torch.float16

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=dtype,
    device_map=device,
    trust_remote_code=True
).eval()

messages = [
    {"role": "user", "content": "Explain the difference between stack memory and heap memory in 3 concise bullet points."}
]

input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        temperature=0.3,
        top_p=0.9,
        top_k=50,
        repetition_penalty=1.1,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id
    )

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Full Reproducibility Script

Below is the exact script used to patch the environment, expose per-layer hidden states in modeling_spark.py, run 150 trials of Heretic abliteration, and export the merged weights:

import os
import re
import sys
import glob
import shutil
import subprocess
import logging
import importlib.util

os.environ["CUDA_VISIBLE_DEVICES"] = "0"   # single GPU, avoids cuda:0/cuda:1 split

# ==============================================================================
# 1. CONFIGURATION
# ==============================================================================
hf_token = "Your hf token here"

SOURCE_MODEL      = "XHToken/Spark-X2.6-1.7B"
TARGET_REPO       = "OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated"
SAVE_DIR          = "./Spark-X2.6-1.7B-Uncensored"
TEMP_DOWNLOAD_DIR = "./temp_base_model"
LOCAL_MODEL_DIR   = "./spark_local_patched"   # patched local copy used by Heretic

# ==============================================================================
# 2. LOGGING & HELPERS
# ==============================================================================
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(message)s",
    handlers=[logging.StreamHandler(sys.stdout)],
    force=True
)
logger = logging.getLogger(__name__)

def run_live_command(cmd, env=None):
    logger.info(f"Running command: {' '.join(cmd)}")
    merged_env = os.environ.copy()
    merged_env["PYTHONUNBUFFERED"] = "1"
    if env:
        merged_env.update(env)
    process = subprocess.Popen(
        cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
        text=True, bufsize=1, env=merged_env
    )
    for line in iter(process.stdout.readline, ""):
        print(line, end="", flush=True)
    process.stdout.close()
    if process.wait() != 0:
        raise RuntimeError(f"Command failed: {' '.join(cmd[:4])}")

def transformers_file(name):
    """Locate a transformers file WITHOUT importing transformers."""
    spec = importlib.util.find_spec("transformers")
    return os.path.join(spec.submodule_search_locations[0], name)

# ==============================================================================
# 3. INSTALL (all pip installs happen before any patching)
# ==============================================================================
hf_token = hf_token.strip()
if not hf_token or hf_token == "Your hf token here":
    raise ValueError("Paste your Hugging Face token into `hf_token`!")

logger.info("Step 1/5: Installing Heretic and dependencies...")
run_live_command([
    sys.executable, "-m", "pip", "install", "-U",
    "torchao==0.16.0", "accelerate", "peft", "huggingface_hub",
    "git+[https://github.com/p-e-w/heretic.git](https://github.com/p-e-w/heretic.git)"
])

logger.info("Pinning transformers==4.57.1 and tokenizers==0.22.2 (--no-deps)...")
run_live_command([
    sys.executable, "-m", "pip", "install", "--no-deps", "--force-reinstall",
    "transformers==4.57.1", "tokenizers==0.22.2"
])

# ==============================================================================
# 4. PATCH TRANSFORMERS ON DISK (never imports transformers)
# ==============================================================================
logger.info("Step 2/5: Patching transformers files on disk...")

# 4a. Allow huggingface-hub 1.x
dvt_file = transformers_file("dependency_versions_table.py")
with open(dvt_file, "r", encoding="utf-8") as f:
    code = f.read()
code = code.replace('"huggingface-hub>=0.34.0,<1.0"', '"huggingface-hub>=0.34.0"')
with open(dvt_file, "w", encoding="utf-8") as f:
    f.write(code)
logger.info(f"Patched {dvt_file}")

# 4b. Auto-enable trust_remote_code
dmu_file = transformers_file("dynamic_module_utils.py")
with open(dmu_file, "r", encoding="utf-8") as f:
    code = f.read()
if "def resolve_trust_remote_code_orig(" not in code:
    code = code.replace("def resolve_trust_remote_code(",
                        "def resolve_trust_remote_code_orig(")
    code += "\n\ndef resolve_trust_remote_code(*args, **kwargs):\n    return True\n"
    with open(dmu_file, "w", encoding="utf-8") as f:
        f.write(code)
logger.info(f"Patched {dmu_file}")

# 4c. Fix kernels incompatibility (ignores commented-out lines)
def active_text(body):
    return "\n".join(l for l in body.splitlines() if not l.strip().startswith("#"))

def has_kw(body, kw):
    return re.search(r'\b' + kw + r'\s*=', active_text(body)) is not None

hk_file = transformers_file("integrations/hub_kernels.py")
with open(hk_file, "r", encoding="utf-8") as f:
    code = f.read()
code = code.replace('LayerRepository(revision="main", ', 'LayerRepository(')

def fix(m):
    body = m.group(1)
    if has_kw(body, "revision") or has_kw(body, "version"):
        return m.group(0)
    return 'LayerRepository(revision="main", ' + body + ')'

code = re.sub(r'LayerRepository\((.*?)\)', fix, code, flags=re.S)
with open(hk_file, "w", encoding="utf-8") as f:
    f.write(code)
logger.info(f"Patched {hk_file}")

bad = 0
for m in re.finditer(r'LayerRepository\((.*?)\)', code, flags=re.S):
    body = m.group(1)
    ok = has_kw(body, "revision") != has_kw(body, "version")
    bad += (not ok)
    print("OK " if ok else "BAD", " ".join(active_text(body).split()))
if bad:
    raise RuntimeError(f"{bad} LayerRepository call(s) still invalid")

# 4d. Make list_repo_templates return [] on any error (404 on missing folder)
hub_file = transformers_file("utils/hub.py")
with open(hub_file, "r", encoding="utf-8") as f:
    code = f.read()
if "def list_repo_templates(" in code and "_list_repo_templates_orig" not in code:
    code = code.replace("def list_repo_templates(", "def _list_repo_templates_orig(")
    code += (
        "\n\ndef list_repo_templates(*args, **kwargs):\n"
        "    try:\n"
        "        return _list_repo_templates_orig(*args, **kwargs)\n"
        "    except Exception:\n"
        "        return []\n"
    )
    with open(hub_file, "w", encoding="utf-8") as f:
        f.write(code)
    logger.info(f"Patched list_repo_templates in {hub_file}")
else:
    logger.info("list_repo_templates already patched or not found")

# ==============================================================================
# 5. AUTHENTICATE
# ==============================================================================
logger.info("Step 3/5: Authenticating with Hugging Face Hub...")
os.environ["HF_TOKEN"] = hf_token
os.environ["HUGGING_FACE_HUB_TOKEN"] = hf_token

from huggingface_hub import HfApi, login, snapshot_download
login(token=hf_token)
api = HfApi(token=hf_token)

# ==============================================================================
# 5a. DOWNLOAD MODEL LOCALLY AND PATCH modeling_spark.py TO RETURN HIDDEN STATES
# ==============================================================================
logger.info(f"Downloading {SOURCE_MODEL} to {LOCAL_MODEL_DIR} for patching...")
snapshot_download(repo_id=SOURCE_MODEL, local_dir=LOCAL_MODEL_DIR, token=hf_token)

ms_file = os.path.join(LOCAL_MODEL_DIR, "modeling_spark.py")
with open(ms_file, "r", encoding="utf-8") as f:
    ms = f.read()

if "_spark_all_hs" in ms:
    logger.info("modeling_spark.py already patched")
else:
    # (1) collect each layer's input hidden states inside the decoder loop
    loop_re = re.compile(r'(?m)^([ \t]*)for decoder_layer in self\.layers:\n')
    if len(loop_re.findall(ms)) != 1:
        raise RuntimeError("Expected exactly 1 'for decoder_layer in self.layers:' in modeling_spark.py - paste the layer loop")
    ms = loop_re.sub(
        r'\1_spark_all_hs = ()\n'
        r'\1for decoder_layer in self.layers:\n'
        r'\1    _spark_all_hs += (hidden_states,)\n',
        ms, count=1
    )
    # (2) return them (plus the final normed output) from the base model
    ret_old = "last_hidden_state=hidden_states,"
    if ms.count(ret_old) != 1:
        raise RuntimeError("Expected exactly 1 'last_hidden_state=hidden_states,' in modeling_spark.py - paste the return statement")
    ms = ms.replace(
        ret_old,
        "last_hidden_state=hidden_states,\n"
        "            hidden_states=_spark_all_hs + (hidden_states,),"
    )
    with open(ms_file, "w", encoding="utf-8") as f:
        f.write(ms)
    logger.info("Patched modeling_spark.py (now returns per-layer hidden states)")

# ==============================================================================
# 5b. PRE-FLIGHT: exactly Heretic's generate() call must give per-layer states
# ==============================================================================
logger.info("Pre-flight check on the patched local model...")

DIAG = '''
import sys, torch
import transformers
from transformers import AutoTokenizer, AutoModelForCausalLM
print("DIAG transformers:", transformers.__version__)
name = sys.argv[1]
tok = AutoTokenizer.from_pretrained(name, trust_remote_code=True)
m = AutoModelForCausalLM.from_pretrained(
    name, trust_remote_code=True, dtype=torch.float16, device_map={"": 0})
ids = tok("Hello", return_tensors="pt").to("cuda:0")
g = m.generate(**ids, max_new_tokens=1, do_sample=False, top_k=None,
               output_hidden_states=True, return_dict_in_generate=True,
               use_cache=False)
hs = g.hidden_states
first = hs[0] if hs else None
print("DIAG first:", type(first).__name__)
print("DIAG layers:", len(first) if first is not None else None)
print("DIAG shape:", tuple(first[-1].shape) if first is not None else None)
'''
diag_path = "/tmp/diag_hidden_states.py"
with open(diag_path, "w") as f:
    f.write(DIAG)

res = subprocess.run(
    [sys.executable, diag_path, LOCAL_MODEL_DIR],
    capture_output=True, text=True, env=os.environ.copy()
)
diag = {}
for line in res.stdout.splitlines():
    if line.startswith("DIAG "):
        print(line)
        k, _, v = line[5:].partition(":")
        diag[k.strip()] = v.strip()

if res.returncode != 0 or "first" not in diag:
    print("Pre-flight crashed. Last lines of output:")
    print("\n".join((res.stdout + res.stderr).splitlines()[-25:]))
    raise RuntimeError("Pre-flight check failed - paste the lines above")

if diag["first"] != "tuple":
    raise RuntimeError(
        f"Hidden states still not returned (first = {diag['first']}). "
        "Paste the DIAG lines above."
    )
logger.info("Pre-flight OK - patched model returns per-layer hidden states.")

# ==============================================================================
# 6. RUN HERETIC (on the patched local copy)
# ==============================================================================
logger.info(f"Step 4/5: Running Heretic on {LOCAL_MODEL_DIR} with 150 trials...")

heretic_cmd = [
    "heretic",
    f"--model={LOCAL_MODEL_DIR}",
    '--modifiers=[{"plugin":"heretic.modifiers.abliteration.Abliteration"}]',
    "--batch-size=8",
    "--max-response-length=100",
    "--n-trials=150",
    "--n-startup-trials=30",
    "--print-debug-information",
    "--trial-index=0",
    "--export-strategy=MERGE",
    "--model-action=save",
    f"--save-directory={SAVE_DIR}"
]
run_live_command(heretic_cmd)

# Copy the ORIGINAL (unpatched) custom modeling files from the hub into the output
logger.info("Copying original custom modeling files...")
try:
    snapshot_download(
        repo_id=SOURCE_MODEL, local_dir=TEMP_DOWNLOAD_DIR,
        token=hf_token, allow_patterns=["*.py", "*.jinja"]
    )
    for ext in ("*.py", "*.jinja"):
        for fp in glob.glob(os.path.join(TEMP_DOWNLOAD_DIR, ext)):
            shutil.copy2(fp, SAVE_DIR)
            logger.info(f"Copied {os.path.basename(fp)}")
    shutil.rmtree(TEMP_DOWNLOAD_DIR, ignore_errors=True)
except Exception as e:
    logger.warning(f"Could not copy custom files: {e}")

# ==============================================================================
# 7. UPLOAD
# ==============================================================================
logger.info(f"Step 5/5: Uploading to '{TARGET_REPO}'...")
api.create_repo(repo_id=TARGET_REPO, repo_type="model", private=False, exist_ok=True)
api.upload_folder(
    folder_path=SAVE_DIR, repo_id=TARGET_REPO, repo_type="model",
    commit_message="Upload Heretic abliterated Spark-X2.6-1.7B weights (150 trials)"
)
logger.info(f"SUCCESS! [https://huggingface.co/](https://huggingface.co/){TARGET_REPO}")