base_model: orcarouter/Qwen3.8-27B-Uncensored-FP8 library_name: peft pipeline_tag: image-text-to-text license: other language:

en
fr tags:
unsloth
lora
qlora
cyber
image-text-to-text
vision
video
peft

model-index:

name: Qwen3.8-27B-Uncensored-FP8-cyber — LoRA results:
- task: type: text-generation name: Causal language modeling dataset: name: cyber SFT holdout type: dataset_cyber.jsonl metrics:
  - type: loss value: 3.998755 name: eval_loss

Qwen3.8-27B-Uncensored-FP8-cyber — LoRA

Instruction-tuned cybersecurity assistant (offensive, defensive, GRC, architecture, SOC/DFIR, RSSI).


Base model	`orcarouter/Qwen3.8-27B-Uncensored-FP8`
Domain	cyber
Method	LoRA / QLoRA (Unsloth) · rank 8 · α 16
Quantization at train	bf16 LoRA
Context	2048 tokens
Dataset	`dataset_cyber.jsonl` · train 84471 / eval 854
GPU	NVIDIA RTX PRO 6000 Blackwell Server Edition (95.0 GiB)
Wall time	456.3 min
Modalities kept	vision, video

This checkpoint continues a strong general model and specialises it on a curated SFT corpus of cybersecurity procedures: pentest / red team, SOC and DFIR, cloud and identity, GRC (ISO, NIST, NIS2, DORA), and RSSI / project-management questions. Answers are meant to be concrete (controls, detections, hardening), not generic essays.

What changed vs the reference

Reference = the published base checkpoint orcarouter/Qwen3.8-27B-Uncensored-FP8, plus the first in-run loss (LoRA ≈ 0 at step 0).

Metric	Reference (base / first log)	This fine-tune	Δ
Train loss (first → last logged)	14.2453	5.0644	-64.4%
Train loss (best)	—	3.7237	—
Eval loss (holdout, first → last)	8.9583	3.9988	-55.4%

The first logged train loss is the closest in-run proxy for the base model (LoRA starts near zero). Option F, when executed, adds an independent holdout comparison against the frozen merged base.

Training data

File: dataset_cyber.jsonl
Split: 0.01 holdout, seed 42
Format: chat-templated SFT (messages / instruction+output / ### Instruction + ### Response)

Training procedure

Hyperparameter	Value
Epochs	1
Learning rate	0.0002
Warmup ratio	0.05
Device batch	4
Grad accum	2
Effective batch	8
Optim	adamw_8bit
Packing	True
LoRA targets	`['q_proj', 'k_proj', 'v_proj', 'o_proj', 'out_proj', 'gate_proj', 'up_proj', 'down_proj']`

Intended use

Authorized defensive work, tabletop exercises, control design, detection engineering, audit readiness, and explaining attack techniques without weaponized payloads.

Out of scope: Do not use it to attack systems you do not own, to generate exploit payloads, or as a substitute for a licensed auditor or incident commander.

Multimodal

Kept towers: vision, video. Vision/audio layers were frozen during text SFT (vision=False, audio=False). Load the merged Transformers folder (or GGUF + mmproj) to keep image / video / audio.

Sample (post-train smoke test)

Prompt: How do you design a reliable Zero Trust infrastructure? Answer with a concrete control list.

system
Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.
user
How do you design a reliable Zero Trust infrastructure? Answer with a concrete control list.
assistant
<think>
k3.5 (Process Injection) and T1071 (Process Injection**

NIST CSF's Respond function, specifically: 137 (e.g., the user, and file system to handle comprehensive coverage of the user should be used by the same attack vectors, and prioritize correctness, and the serverless, and how to to write a secure file. This technique can be used by a security posture. The NIST Cybersecurity Framework's Detect function, then identify the main processes. This aligns with NIST Cybersecurity Framework's Detect function. Deploy user input, enabling automated monitoring and threat intelligence feeds to the network segmentation, ensuring security controls in a system memory patterns (TTPs) in security controls, including T1025, and behavioral analysis of security controls, and remediation, and network access, and how can be used by the response.

**Temporal Correlation with MITRE ATT&CK techniques T1027 (T1

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = 'orcarouter/Qwen3.8-27B-Uncensored-FP8'
adapter = "nico248000000000/Qwen3.8-27B-Uncensored-FP8-cyber-LoRA"
tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, adapter)

Limitations

Domain shift: quality drops outside the SFT topics.
Eval above is holdout loss (and optional targeted checks). It is not a public leaderboard.
The base model license and acceptable-use policy still apply.

License

other — inherit and respect the license of orcarouter/Qwen3.8-27B-Uncensored-FP8.

Qwen3.8-27B-Uncensored-FP8-cyber — LoRA

Instruction-tuned cybersecurity assistant (offensive, defensive, GRC, architecture, SOC/DFIR, RSSI).

Base model

orcarouter/Qwen3.8-27B-Uncensored-FP8

Domain

cyber

Method

LoRA / QLoRA (Unsloth) · rank 8 · α 16

Quantization at train

bf16 LoRA

Context

2048 tokens

Dataset

dataset_cyber.jsonl · train 84471 / eval 854

GPU

NVIDIA RTX PRO 6000 Blackwell Server Edition (95.0 GiB)

Wall time

456.3 min

Modalities kept

vision, video

What changed vs the reference

Reference = the published base checkpoint orcarouter/Qwen3.8-27B-Uncensored-FP8, plus the first in-run loss (LoRA ≈ 0 at step 0).

Metric

Reference (base / first log)

This fine-tune

Train loss (first → last logged)

14.2453

5.0644

-64.4%

Train loss (best)

—

3.7237

—

Eval loss (holdout, first → last)

8.9583

3.9988

-55.4%

The first logged train loss is the closest in-run proxy for the base model (LoRA starts near zero). Option F, when executed, adds an independent holdout comparison against the frozen merged base.

Hyperparameter

Value

Epochs

Learning rate

0.0002

Warmup ratio

0.05

Device batch

Grad accum

Effective batch

Optim

adamw_8bit

Packing

True

LoRA targets

['q_proj', 'k_proj', 'v_proj', 'o_proj', 'out_proj', 'gate_proj', 'up_proj', 'down_proj']

Intended use

Authorized defensive work, tabletop exercises, control design, detection engineering, audit readiness, and explaining attack techniques without weaponized payloads.

Out of scope: Do not use it to attack systems you do not own, to generate exploit payloads, or as a substitute for a licensed auditor or incident commander.

Sample (post-train smoke test)

Prompt: How do you design a reliable Zero Trust infrastructure? Answer with a concrete control list.

system Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer. user How do you design a reliable Zero Trust infrastructure? Answer with a concrete control list. assistant <think> k3.5 (Process Injection) and T1071 (Process Injection** NIST CSF's Respond function, specifically: 137 (e.g., the user, and file system to handle comprehensive coverage of the user should be used by the same attack vectors, and prioritize correctness, and the serverless, and how to to write a secure file. This technique can be used by a security posture. The NIST Cybersecurity Framework's Detect function, then identify the main processes. This aligns with NIST Cybersecurity Framework's Detect function. Deploy user input, enabling automated monitoring and threat intelligence feeds to the network segmentation, ensuring security controls in a system memory patterns (TTPs) in security controls, including T1025, and behavioral analysis of security controls, and remediation, and network access, and how can be used by the response. **Temporal Correlation with MITRE ATT&CK techniques T1027 (T1

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel import torch base = 'orcarouter/Qwen3.8-27B-Uncensored-FP8' adapter = "nico248000000000/Qwen3.8-27B-Uncensored-FP8-cyber-LoRA" tok = AutoTokenizer.from_pretrained(adapter, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( base, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True, ) model = PeftModel.from_pretrained(model, adapter)