license: other license_name: qwen-community-1.0 license_link: LICENSE base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF base_model_relation: quantized pipeline_tag: image-text-to-text library_name: gguf tags:
- gguf
- splash
- apple-silicon
- qwen3.8
- flash-next
- abliterated
- uncensored
Qwen3.8-Flash-Next-Uncensored — Q5_K_S for Splash
A lossless repack of the Q5_K_S quant from orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF that Splash can load. The encoding of 96 small tensors changes and nothing else: every other tensor is byte-identical, and every stored value is preserved exactly.
Safety alignment removed. The upstream build is abliterated (refusal-removed). This repack inherits that behaviour unchanged.
Why this exists
Splash 1.4.0 refuses orcarouter's 4- and 5-bit quants (revision d366e279) from the header alone:
error: this GGUF stores tensors Splash cannot load: hc_attn_inject.weight Q5_K (48 tensors), hc_ffn_inject.weight Q5_K (48 tensors); choose another variant
Splash accepts only F32, BF16 or Q8_0 for the hyper-connection inject tensors (FLOAT_ROW_TYPES in its install/gguf.py), and its loader widens them to F32 regardless. orcarouter's Q8_0 builds pass the check, but at 175–178 GiB they do not fit a 128 GB Mac. This repack stores those 96 tensors as F32 and keeps the rest of Q5_K_S.
What changed
| orcarouter Q5_K_S | This repo | |
|---|---|---|
hc_attn_inject.weight ×48, hc_ffn_inject.weight ×48, each [10240, 4] | Q5_K | F32 |
| The other 1,128 tensors | — | Byte-identical |
| GGUF metadata (incl. split and imatrix keys) | — | Identical |
| Size | 121,801.13 MiB | 121,813.55 MiB (+12.42 MiB) |
Verification
| Check | Result |
|---|---|
| Source files vs. orcarouter's published SHA-256 | All 4 match |
Splash 1.4.0 require_loadable() | This repo: accepted · orcarouter Q5_K_S: rejected |
| Inject tensor values | 96/96 bit-exact against an independent Q5_K dequantization of the originals (3,932,160 values, max abs diff 0) |
| Other tensors | 1,128/1,128 byte-identical (118.94 GiB compared) |
| Metadata keys | 0 differences |
| Tensor count / shapes | 1,224 / unchanged |
Tested
Tested on a MacBook Pro (M4 Max, 128 GB) running oMLX 0.7.2 with Splash 1.4.0 and the incoai/Qwen3.8-Flash-Next-DFlash2 draft.
- Loads at the full 262,144-token context.
- The PLE table streams from SSD (36,621 MiB stays on disk, 511 MiB row cache). All 81,602 MiB of routed experts stay in memory.
- With the mmproj present, Splash loads the vision tower; the projector passes Splash's
vision_config()andprocessor_config()checks. Image inference has not been tested. - Smoke test: correct output at 98.4 tok/s (31 generated tokens).
Not tested with llama.cpp or other runtimes.
Files
| File | SHA-256 |
|---|---|
Qwen3.8-Flash-Next-Uncensored-Q5_K_S-Splash-00001-of-00003.gguf | b0a9fde0d03f042b891a71fe15770babc1db20232ad6fcf2603f8ade1c6ed3ff |
Qwen3.8-Flash-Next-Uncensored-Q5_K_S-Splash-00002-of-00003.gguf | f964a5e985119f0bc4bc9ebaffdb0ded24d3272132f40d66204bac9ea2bd987e |
Qwen3.8-Flash-Next-Uncensored-Q5_K_S-Splash-00003-of-00003.gguf | 96f09ce436f85b41fa39ac978a3e0b9c5e7a85498a946359c54893c57d76889b |
mmproj-Qwen3.8-Flash-Next-Uncensored-F16.gguf (unchanged from orcarouter) | f0f352a97a62a057f3aecdb597cac664762cea2ca23f7b16ec92eee28c5572d9 |
Source files, orcarouter revision d366e279688c1f2fd6e75e2f98475d38db534f1d:
| File | SHA-256 |
|---|---|
Qwen3.8-Flash-Next-Uncensored-Q5_K_S-00001-of-00003.gguf | c789c1e7445e6fe90707ec13461dd1e0cf5343d374b5ac7ee1c8eeef604dd19c |
Qwen3.8-Flash-Next-Uncensored-Q5_K_S-00002-of-00003.gguf | d43111ec60a5868cdb60bdcffed89454ac8d51b911d5a847f7d2c2f6f20c6b7a |
Qwen3.8-Flash-Next-Uncensored-Q5_K_S-00003-of-00003.gguf | 1cf59e7ae710a43b5dfc3fe03f16fa74ae752f18ea1c555dc87a1c2678401366 |
Reproduce it
From orcarouter's Q5_K_S shards, with llama.cpp's llama-quantize:
llama-quantize --allow-requantize --keep-split \
--tensor-type "hc_attn_inject=f32" --tensor-type "hc_ffn_inject=f32" \
Qwen3.8-Flash-Next-Uncensored-Q5_K_S-00001-of-00003.gguf \
out/Qwen3.8-Flash-Next-Uncensored-Q5_K_S-Splash.gguf \
Q5_K_S
- The target type must be
Q5_K_S. WithCOPY,--tensor-typeis silently ignored and the inject tensors stay Q5_K. - Same-type tensors pass through without requantization; the log shows a conversion only on the 96 inject tensors.
--keep-splitappends-0000N-of-00003to the output name, so give the output name without a shard suffix.- It took 1 min 40 s on an M4 Max.
The same override with the matching target type should make orcarouter's other 4- and 5-bit variants loadable too. Only Q5_K_S was converted and tested here.
Credits and license
Quantization, abliteration and the imatrix (whose metadata is preserved) are by orcarouter. The base model is Qwen/Qwen3.8-Flash-Next.
The base model is released under the Qwen Community License 1.0, which governs this derivative. Abliteration and requantization do not change it. The license is included verbatim in LICENSE.