Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK

library_name: transformers license: mit license_link: https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B/blob/main/LICENSE language:

• en
• zh
• ja pipeline_tag: text-generation tags:
• heretic
• uncensored
• decensored
• abliterated
• mpoa
• ornith
• fp8
• fp8-block
• compressed-tensors
• moe
• qwen3.5
• mtp
• speculative-decoding
• multimodal
• vllm
• sglang
• agentic
• coding base_model:
• llmfan46/Ornith-1.0-35B-uncensored-heretic
• deepreinforce-ai/Ornith-1.0-35B

Overview

This is an FP8_BLOCK quantized version of llmfan46/Ornith-1.0-35B-uncensored-heretic with an MTP head grafted from Qwen3.6-35B-A3B for speculative decoding.

The uncensored heretic version achieves 90% fewer refusals (9/100 vs 89/100 original) with only 0.0019 KL divergence.

Key Features

Quick Start

vLLM (Recommended)

pip install vllm

vllm serve AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --served-model-name ornith-35b-uncensored-fp8 \
  --quantization compressed-tensors \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.93 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Performance-optimized configuration (validated on RTX 4090 48GB):

vllm serve AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --served-model-name ornith-35b-uncensored-fp8 \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 204800 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.93 \
  --dtype bfloat16 \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8_e4m3 \
  --mm-encoder-tp-mode data \
  --mm-processor-cache-type shm \
  --mm-shm-cache-max-object-size-mb 512 \
  --media-io-kwargs '{"video": {"num_frames": -1}}' \
  --attention-backend flashinfer \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --generation-config vllm \
  --max-cudagraph-capture-size 16 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --compilation_config.mode VLLM_COMPILE \
  --compilation_config.cudagraph_mode PIECEWISE \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_xml \
  --enable-auto-tool-choice \
  --trust-remote-code

GPU Requirements: ~40GB VRAM minimum (model is ~38GB FP8). The configuration above was validated on a modified RTX 4090 48GB. For lower VRAM GPUs, quantization to GGUF format is recommended instead.

SGLang

pip install sglang

python3 -m sglang.launch_server \
  --model-path AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK \
  --host 0.0.0.0 \
  --port 30000

Transformers

from transformers import AutoModelForMultimodalLM, AutoProcessor

model = AutoModelForMultimodalLM.from_pretrained(
    "AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(
    "AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK",
    trust_remote_code=True,
)

Performance

Note: This model has not been benchmarked. The data below is from the reference model shisa-ai/Ornith-1.0-35B-FP8-BLOCK-MTP and is provided for reference only.

Reference benchmarks (RTX PRO 6000 Blackwell, single GPU):

Source: shisa-ai/Ornith-1.0-35B-FP8-BLOCK-MTP

Quantization Details

• Tool: llmcompressor v0.12.0
• Method: Model-free PTQ (no calibration data needed)
• Scheme: FP8_BLOCK (compressed-tensors format)
• Weight: Static FP8, symmetric, 128×128 block strategy
• Activation: Dynamic FP8, symmetric, group size 128
• MTP Head: BF16 (not quantized)

Ignored layers: lm_head, embed_tokens, visual.*, mlp.gate, mlp.shared_expert_gate, linear_attn.*, mtp.*

Model Lineage

deepreinforce-ai/Ornith-1.0-35B (MIT, original)
    └── llmfan46/Ornith-1.0-35B-uncensored-heretic (Heretic v1.2.0 MPOA ablation)
        └── AxiangQAQ/Ornith-1.0-35B-uncensored-heretic-FP8-BLOCK (this model)
            ├── FP8_BLOCK quantization (llmcompressor)
            └── MTP head grafted from Qwen/Qwen3.6-35B-A3B (donor only)

Credits

License

MIT — same as the source Ornith-1.0-35B model.

Citation

@misc{ornith-35b,
    title = {{Ornith-1.0-35B}: Agentic Coding, Open to All},
    url = {https://deep-reinforce.com/ornith_1_0.html},
    author = {{DeepReinforce Team}},
    year = {2026}
}