base_model: mistralai/Mistral-Medium-3.5-128B base_model_relation: finetune library_name: transformers pipeline_tag: image-text-to-text tags:
- transformers
- peft
- lora
- qlora
- bitsandbytes
- mistral
- mistral3
- mistral-medium
- multimodal
- reasoning
- eschaton-engine
- eschaton-uncensored
- uncensored license: other license_name: modified-mit license_link: https://huggingface.co/mistralai/Mistral-Medium-3.5-128B/blob/main/LICENSE datasets:
- cloudbjorn/eschaton-uncensored
Mistral-Medium-3.5-128B-Eschaton-Uncensored
This is a merged BF16 fine-tune of Mistral Medium 3.5 128B. Training loaded the full-BF16 axolotl-ai-co/Mistral-Medium-3.5-128B-BF16 checkpoint and used cloudbjorn/eschaton-uncensored with the Eschaton Engine.
Training used 4-bit NF4 QLoRA with BF16 compute. The resulting LoRA adapter was then merged into the original BF16 base checkpoint, so this repository contains the complete merged BF16 model rather than pre-quantized training weights.
The fine-tune focuses on direct, neutral, and useful responses to sensitive, gritty, controversial, emotionally intimate, and technically demanding prompts without repetitive moralizing or canned disclaimers.
Personality: Objectivity Over Preachiness
The Eschaton Uncensored dataset emphasizes direct answers, task-appropriate tone, technical substance, dark creative work, and candid analysis. It is intended to reduce unnecessary refusals and boilerplate while preserving the broad capabilities of the underlying instruct model.
Model Characteristics
- Architecture: Dense 128B Mistral 3 multimodal model
- Context window: Up to 262,144 tokens supported by the base architecture
- Reasoning modes: Supports Mistral's configurable reasoning behavior
- Output format: Complete merged model in BF16
- Fine-tuning scope: Text-language layers only; the vision tower and multimodal projector were excluded from LoRA adaptation
- Vision behavior: Vision components remain those of the base checkpoint and were not fine-tuned on this text-only dataset
The training sequence length was 2,048 tokens. The base architecture's larger inference context window was not used as the training sequence length for this fine-tune.
Training Details
| Parameter | Value |
|---|---|
| Base model | axolotl-ai-co/Mistral-Medium-3.5-128B-BF16 |
| Dataset | cloudbjorn/eschaton-uncensored |
| Framework | Eschaton Engine using Transformers, TRL, PEFT and bitsandbytes |
| Training method | 4-bit NF4 QLoRA |
| Quantization compute dtype | bfloat16 |
| Double quantization | Enabled |
| Final repository format | LoRA merged into the BF16 base model |
| Epochs | 1 |
| Training sequence length | 2,048 tokens |
| Packing | Disabled |
| Seed | 3407 |
LoRA Configuration
The Eschaton Engine automatically selected its 60B–149B model profile for this 127.7B-parameter checkpoint.
| Parameter | Value |
|---|---|
Rank (r) | 32 |
| LoRA alpha | 64 |
| Target modules | all-linear in the language model, excluding vision and multimodal-projector modules |
| LoRA dropout | 0.05 |
| Bias | none |
| Task type | CAUSAL_LM |
Optimization
| Parameter | Value |
|---|---|
| Optimizer | 8-bit paged AdamW |
| Per-device micro-batch size | 1 |
| Gradient accumulation | 32 |
| Effective batch size | 32 |
| Learning rate | 5e-6 |
| LR scheduler | Linear |
| Warmup steps | 50 |
| Weight decay | 0.01 |
| Gradient checkpointing | Enabled |
Evaluation Status
No standardized benchmark results are reported for this fine-tune. Users should evaluate it against their own instruction-following, reasoning, coding, safety, and domain-specific requirements before deployment.
License
This derivative follows the base model's Modified MIT License. Review that license and the upstream model card before use or redistribution.