library_name: gguf license: mit pipeline_tag: text-generation tags:
- gguf
- moe
- hybrid
- reasoning
- abliterated
- quantization
- bailingmoe3
- inclusionAI base_model:
- inclusionAI/Ling-3.0-flash
- SC117/Ling-3.0-flash-abliterated-APEX-GGUF base_model_relation: quantized
Ling-3.0-flash-abliterated — GGUF Quants
Standard static quants (no imatrix) + APEX mixed-precision quants of SC117's abliterix v6 trial 21 merge.
Original model: SC117/Ling-3.0-flash-abliterated-APEX-GGUF Base model: inclusionAI/Ling-3.0-flash Abliteration by: abliterix v6 trial 21 Architecture: bailingmoe3 (124B total, 5.1B active) License: MIT
Files
Standard Static Quants (llama-quantize, no imatrix)
| File | Type | Approx Size |
|---|---|---|
*-Q2_K.gguf | Q2_K | ~40 GB |
*-Q3_K_S.gguf | Q3_K_S | ~50 GB |
*-Q3_K_M.gguf | Q3_K_M | ~55 GB |
*-Q3_K_L.gguf | Q3_K_L | ~60 GB |
*-Q4_K_M.gguf | Q4_K_M | ~70 GB |
*-Q5_K_M.gguf | Q5_K_M | ~90 GB |
*-Q6_K.gguf | Q6_K | ~108 GB |
*-Q8_0.gguf | Q8_0 | ~140 GB |
APEX Mixed-Precision Quants
| File | Base | Experts | Shared | Attn |
|---|---|---|---|---|
*-APEX-Quality.gguf | Q6_K | edge Q6_K / mid iq4_xs | Q8_0 | Q6_K |
*-APEX-Balanced.gguf | Q6_K | edge Q6_K / mid Q5_K | Q8_0 | Q6_K |
*-APEX-Compact.gguf | Q4_K_M | edge Q4_K / mid Q3_K | Q6_K | Q4_K |
All APEX configs: routers Q8_0, MTP (blk.42) Q8_0. No imatrix required. APEX Mini (iq2_s mid) excluded — requires imatrix for quantization.
Usage
Requires llama.cpp with PR #26608 (bailingmoe3 support, merged b10472+).
llama-server -m .gguf --flash-attn on --fit on --ctx-size 32768 --reasoning off --jinja
Sampling: temperature 0.6, top_p 0.95, top_k 20
Credits
- Model: inclusionAI/Ling-3.0-flash
- Abliteration: SC117 using abliterix v6 trial 21
- APEX quantization method: mudler/apex-quant
- GGUF quants: This repo (quantization only, no model modifications)