routangseng-qwen35-0.8b-abliterated-lora-onnx

ONNX export of bobber/routangseng-qwen35-0.8b-abliterated with identity-override LoRA merged.

Build Process

Base model: huihui-ai/Huihui-Qwen3.5-0.8B-abliterated
LoRA fine-tuning: Identity-override LoRA (rank 64, alpha 128, 251 training rows focused on identity responses)
Merge: LoRA adapter merged into base model
ONNX: Weight transplant into reference graph from onnx-community/Qwen3.5-0.8B-ONNX
Quantization: q8 (MatMul-only for decoder, full dynamic for embed/vision)

⚠️ Known Issues

Identity-override LoRA degrades analytical depth: The LoRA was trained heavily on short identity responses (1-2 sentences), which biases the model toward brevity. For deep structured analysis, use the base abliterated ONNX at bobber/routangseng-qwen35-0.8b-abliterated-onnx instead.

Usage with transformers.js

import { Qwen3_5ForConditionalGeneration, AutoProcessor } from '@huggingface/transformers';

const model = await Qwen3_5ForConditionalGeneration.from_pretrained(
  'bobber/routangseng-qwen35-0.8b-abliterated-lora-onnx',
  { dtype: { embed_tokens: 'q8', vision_encoder: 'q8', decoder_model_merged: 'q8' }, device: 'webgpu' }
);

Files

onnx/decoder_model_merged_quantized.onnx + .onnx_data — Decoder (q8, MatMul-only quantization)
onnx/embed_tokens_quantized.onnx + .onnx_data — Embeddings (q8)
onnx/vision_encoder_quantized.onnx + .onnx_data — Vision encoder (q8, from reference, not fine-tuned)

Build Process

Base model: huihui-ai/Huihui-Qwen3.5-0.8B-abliterated

LoRA fine-tuning: Identity-override LoRA (rank 64, alpha 128, 251 training rows focused on identity responses)

Merge: LoRA adapter merged into base model

ONNX: Weight transplant into reference graph from onnx-community/Qwen3.5-0.8B-ONNX

Quantization: q8 (MatMul-only for decoder, full dynamic for embed/vision)

Usage with transformers.js

import { Qwen3_5ForConditionalGeneration, AutoProcessor } from '@huggingface/transformers'; const model = await Qwen3_5ForConditionalGeneration.from_pretrained( 'bobber/routangseng-qwen35-0.8b-abliterated-lora-onnx', { dtype: { embed_tokens: 'q8', vision_encoder: 'q8', decoder_model_merged: 'q8' }, device: 'webgpu' } );