library_name: openvino license: llama3.1 base_model:

• Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2 pipeline_tag: text-generation tags:
• openvino
• llama
• uncensored
• abliterated
• intel
• arc
• text-generation

Llama-3.1-8B-Lexi-Uncensored-V2 — OpenVINO IR (fp16)

OpenVINO IR conversion of Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2, an uncensored Llama-3.1-8B fine-tune.

Sibling builds: int4 · int8

Details

Benchmark

Single-stream, Intel Core Ultra 7 258V (Arc 130V/140V iGPU), 30 GB RAM, OpenVINO Model Server 2026.4.0 on GPU, greedy decoding, 128 max new tokens, mean of 3 runs after warmup.

Decode on this iGPU is memory-bandwidth bound, so throughput tracks model size almost exactly. int4 is both the smallest and the fastest.

Serving

This repository contains the IR only. Create an OVMS config alongside it:

cat > ovms_config.json <<'EOF'
{
  "model_config_list": [
    {
      "config": {
        "name": "lexi",
        "base_path": "/absolute/path/to/this/model",
        "target_device": "GPU",
        "nireq": 8,
        "plugin_config": { "PERFORMANCE_HINT": "THROUGHPUT", "NUM_STREAMS": "2" }
      }
    }
  ]
}
EOF

# graph.pbtxt must also exist in the model directory (copy from any OVMS LLM model).
PYTHONPATH=$OVMS_ROOT/lib/python ovms --config_path ./ovms_config.json --rest_port 11436

Conversion notes

• Source: BF16 safetensors (4 shards).
• Stage 1: optimum-cli export openvino --task text-generation-with-past --weight-format fp16.
• Stage 2: nncf.compress_weights on the IR (never a single-pass int4 export — that OOMs on a 30 GB machine for this size class).
• Exported with transformers 5.5.0 / optimum-intel 2.2.0.

Limitations

• Uncensored fine-tune: outputs are not filtered. Evaluate before deployment.
• Throughput depends on runtime kernels, hardware, and prompt distribution.

Attribution

Base model: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2, built on Meta's Llama 3.1 8B under the Llama 3.1 Community License. Conversion: OpenVINO IR via optimum-intel + NNCF.