base_model:

• ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF library_name: gguf pipeline_tag: image-text-to-text tags:
• gguf
• llama.cpp
• qwen3.8-flash-next
• abliterated
• uncensored
• coding
• agentic
• moe
• gsq
• rco

Qwen3.8-Flash-Next-Orca-Abliterated-GSQ-RCO-Coder-GGUF (58.6 GB)

A standalone, weight-baked uncensored & abliterated GGUF of ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF. This build is created by precisely transplanting 144 orthogonalized residual-writing tensors from orcarouter/Qwen3.8-Flash-Next-Uncensored (via SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF) directly into the optimized 256-expert pruned Coder architecture.

Model Overview & Architecture

ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF compresses Qwen3.8-Flash-Next from 354 GB (BF16, 512 experts per layer) down to 58.4 GB by combining:

1. Structured Expert Pruning ($512 \rightarrow 256$ experts per layer): Selects the 256 experts specialized for software engineering, coding, agentic tool calling, and vision.
2. GSQ (Gumbel-Softmax Quantization) + RCO (Riemannian Co-design Search): Assigns optimal mixed-precision GGUF quantization types across individual tensors while storing the retained weights at ~3.5 bits (IQ3_S-equivalent fidelity, labeled IQ1_M for its 1.89 effective bits per parameter relative to the unpruned 512-expert model).

Verified Benchmark Comparison (Original vs. Orca-Abliterated Build)

Repository Contents

• IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf (29.85 GiB) — Transformer/GDN layers 0..47 with all 144 orcarouter-abliterated residual tensors baked in.
• IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00002-of-00002.gguf (28.80 GiB) — Untouched Per-Layer Embedding (PLE) N-gram lookup tables.
• mmproj-Qwen3.8-Flash-Next-BF16.gguf (0.90 GiB) — Untouched BF16 multimodal vision projector.
• tensor-allocation/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.rco-allocation.txt — Upstream per-tensor RCO quantization map.
• SHA256SUMS — SHA-256 checksums for all GGUF files.

Recommended llama.cpp Usage & Sampling Parameters

Because this is an ultra-low-bit MoE model (IQ1_M effective / IQ3_S retained weights), avoid pure greedy decoding (--temp 0.0 with no repetition penalty) on repetitive C/C++ struct definitions. Use standard Qwen sampling parameters with a light repetition/presence penalty:

llama-server \
  -m IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf \
  --mmproj mmproj-Qwen3.8-Flash-Next-BF16.gguf \
  -c 32768 \
  --temp 0.6 \
  --top-p 0.95 \
  --min-p 0.05 \
  --repeat-penalty 1.05 \
  --presence-penalty 0.5

SHA256 Checksums

f19b9ed5871c2f92aaf2358f936bceb9c48570b1f55cbe69753a53dae2712a7f  IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf
316b46f3a2dbd68c900f43136ab9449f9dcc3725dfd8c794847c204bc161e113  IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00002-of-00002.gguf
b1a82259702816a5330d7bd7607cd9676b11780e79ff7348c21103ff3ce49bd0  mmproj-Qwen3.8-Flash-Next-BF16.gguf

Acknowledgements & Credits

This model build integrates the exceptional contributions of several open-source AI engineering teams:

• ISTA-DASLab for engineering the structural expert pruning, GSQ quantization, RCO co-design search, and GGUF packaging of the core Coder architecture (Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF).
• orcarouter releasing the orthogonalized residual-writing tensors that successfully remove model refusals while preserving logical capability (Qwen3.8-Flash-Next-Uncensored).
• SC117 GSQ-RCO abliterated GGUF bridges.
• The llama.cpp team (ggerganov & contributors) for maintaining the inference framework enabling ultra-low-bit MoE execution.