license: other license_name: qwen-community-license-1.0 license_link: LICENSE base_model:

• orcarouter/Qwen3.8-Flash-Next-Uncensored base_model_relation: quantized library_name: mlx-vlm pipeline_tag: image-text-to-text tags:
• mlx
• qwen4-exp
• quantized
• vision
• apple-silicon
• mixed-precision
• experimental

Qwen3.8-Flash-Next-Uncensored-mlx-optimized-2-8bit

Tested on an M1 Max with 64 GiB unified memory, the included helper's native 16,384-token profile recorded a 39.72 GiB process physical-footprint peak. The complete download is 72.843 GB; that storage size is not RAM use. This Apple Silicon MLX pack provides native qwen4_exp text and vision, full-detail images and optional thinking through the included helper, which applies the required FP16 PLE adapter. Measured memory evidence.

Plan for a 64 GiB-class Mac and a compatible existing Unsloth MLX environment. This release's source-build instructions are an unverified clean-install outline; the separate DFlash2 edition has its own tested installer.

Choose your download

The DFlash2 edition includes the same optimized target payload plus a drafter. Its speed gains were measured against a matched plain baseline in that edition's own tests. This plain helper uses ordinary decoding.

Quick facts

RAM requirements

The PLE table is already included in the download. Rows are mapped and read on demand, so its full 32.0 GB is not one resident MLX weight allocation. Mapped pages, context, image activations, buffers and other applications still consume memory.

A native run at a 16,384-token context budget recorded a 39.72 GiB process physical-footprint peak. A separate historical 8,192-context run observed 41.20 GiB under different conditions. These observations do not establish a 48 GB minimum or a simple relationship between context and peak memory. RSS, allocator memory and physical footprint overlap; do not add them. Native resource evidence · Earlier observations

Unsloth Studio

The target has been reported by the user to work in Unsloth Studio. This is user feedback, not an independently measured Studio benchmark or a promise about every Studio version.

The instructions below use the bundled Python helper. The DFlash2 speed measurements belong to its separate run_dflash2.py entrypoint; loading this target in an application does not establish speculative decoding.

Why does Hugging Face warn that this is too large for 64 GB?

The hardware panel displays the 72.843 GB checkpoint size and an 8-bit label. These do not describe the measured memory of the documented runtime: the 8-bit value is the configuration's default, while experts use 2-bit, PLE uses 4-bit storage, and sensitive components use FP16. The download includes the 32.0 GB disk-backed PLE table, which the included compatible loader maps and reads on demand. Checkpoint storage bytes are not a resident-memory measurement.

On the tested 64 GiB M1 Max, the documented profile recorded a 39.72 GiB process peak (evidence). Use that runtime and the profile limits when interpreting the measurement; other loaders and memory capacities remain unverified. This card cannot override the Hub's automatic hardware badge.

Why can an Unsloth Hub card show a much larger VRAM figure?

Some Unsloth Studio Hub cards use a generic 4-bit QLoRA estimate. For example, a card showing 177.4B parameters can display 0.9 × 177.4 + 1.4 ≈ 161.1 GB. That is an estimator output, not a measured requirement for this MLX pack. The parameter count is a logical model size, not a RAM measurement. The estimate does not account for this pack's mixed 2-bit/4-bit/8-bit and FP16 storage or disk-mapped PLE table. The estimate is not this pack's download size, a measurement of resident GPU memory, or a general minimum-RAM claim. The scoped measurements above describe the tested M1 Max runs only.

Run with Python

Keep the complete repository together, including its PLE files, tokenizer, processor and helpers. Use Python 3.13 in a compatible local MLX/Unsloth environment. Read INSTALL.md before changing dependencies: it documents the tested source builds and their three declared Zoo dependency-bound conflicts. A fresh plain-helper installation has not been verified.

From the downloaded model folder:

python run_qwen38_unsloth.py --model . --check-only
python run_qwen38_unsloth.py --model . \
  --prompt 'Compute 19 + 23. Reply with only the integer.'

This is a one-request helper; a new invocation loads the model again. Its defaults are context 16,384, output cap 4,096, greedy decoding and thinking disabled. Model loading and prompt processing add latency beyond decode tokens/second. Generic loaders that omit the included FP16 PLE adapter are unverified.

--patch-mode native is the default: it uses the unpatched loader and native MLX-VLM generation. --patch-mode patched opts into Unsloth inference scopes; image requests retain the native fused vision loader.

Images and small text

python run_qwen38_unsloth.py --model . \
  --image screenshot.png --image-max-tokens 0 \
  --prompt 'Read the small text in this screenshot.'

Cap 0 preserves the processor's full-detail image path. Large images take longer and use temporary memory; this setting adds no model-weight disk storage. A positive --image-max-tokens value reduces large-image detail. The helper uses the native fused vision loader for image requests to avoid the earlier dense-attention allocation failure.

A 2048×2048 color fixture passed at the combined 16,384-context/4,096-output profile. A separate synthetic OCR fixture returned its six-digit control code exactly. These are targeted checks, not general photo, screenshot or OCR accuracy benchmarks. Full-image check · OCR check

Thinking and tools

Enable bounded thinking explicitly when useful; its tokens share the output allowance:

python run_qwen38_unsloth.py --model . --thinking \
  --thinking-budget 512 --max-tokens 1024 \
  --prompt 'Explain your calculation briefly, then give the answer.'

The model/template retains tool-call capability. A native tool call and mock-result round trip passed in a separate API harness. This folder does not bundle an HTTP server or an application tool-execution loop. The included CLI is not a tool agent. API checks

What was tested

The 50/50 result used the same fixed public GSM8K selection, greedy decoding, thinking off and a 4,096-token output cap, with zero API errors, parse failures or truncated generations. The questions were inspected during budget/truncation development. This is a development subset, not a blind full-benchmark result, general 100% accuracy or an original-BF16 retention measurement. Report

Separate API probes measured medians of 23.47 tokens/s for explanation and 23.49 tokens/s for Python, three requests each. Cache states were mixed. These are decode observations, not cold-start latency or a universal speed claim. Timing evidence

• The earlier fixed-subset scores were 45/50 (90%) at 384 output tokens, 49/50 (98%) at 1,024, and 50/50 at 4,096. Parse failures and truncations remain in the denominator; the budgets are separate measurements. 384 · 1024 · 4096
• The native API smoke suite passed 15/18 arithmetic/fact checks. Its structured-output, tool-round-trip and vision checks passed. Smoke evidence
• Longer-prompt/image integration completed three responses with zero errors, but passed 2/3 strict checks: the lookup answer was correct but its JSON had Markdown fences. The image and follow-up checks passed. Integration evidence
• The first patched-loader 2048×2048 image attempt failed with a Metal out-of-memory error. The helper subsequently used the native fused vision loader and passed the combined full-image profile. The failed attempt is retained. Failure · Completed check
• Six malformed client-generated PNGs were rejected before inference. Corrected PIL-verified color fixtures completed without API errors. Flat-color requests do not measure OCR accuracy. Separate API resolution/follow-up timings also depend on cache reuse, which the one-shot CLI does not provide. Resolution evidence · Warm/follow-up evidence
• One matched native/patched Unsloth text prompt produced identical output. Its reported rates were 20.38 and 26.68 tokens/s respectively; a single prompt does not establish general parity or a general speed improvement. Comparison

Format, source and limitations

Experts use affine 2-bit/group-128 reconstruction fitting; other projections use 8-bit/group-64. PLE uses mapped 4-bit/group-32 storage, while sensitive components and vision weights are FP16. MTP weights are omitted. Quantization is lossy; reconstruction error and these small task results do not establish broad accuracy retention. Conversion record

The source is the approximately 360 GB BF16 orcarouter/Qwen3.8-Flash-Next-Uncensored, revision d9dbbb7f49cad93a0e656ea69200f6b58600714a. Local preparation converted and fitted source weights; it did not train on personal prompts, computer files or call logs. Public GSM8K questions were used for evaluation only. Undisclosed upstream training data is outside that claim. Source attribution

License and file verification

The target is governed by Qwen Community License 1.0. Read its terms before use or redistribution. Runtime dependencies retain their respective licenses. The public GSM8K evaluation license is included separately.

shasum -a 256 -c SHA256SUMS

Keep local prompts, result reports, caches and credentials outside any folder you intend to upload. The prepared publication contains portable inference files, public evaluation evidence and provenance; this does not control the logs created by your own application.