玩模型开放模型 · 清晰来源 · 自主选择
返回模型目录
AEON-7 /

Gemma-4-26B-A4B-it-Uncensored-NVFP4

HF 源站

AEON-7/Gemma-4-26B-A4B-it-Uncensored-NVFP4 is an uncensored open-source model hosted on Hugging Face.

Text GenerationvLLMsafetensorsEnglishLicense: gemma26Btransformersgemma4image-text-to-text4-bit4b-activeaarch64abliteratedaeonaeon-7agenticarm64blackwellchatchunked-prefillcodingcompressed-tensorsconversationaldflashdgx-sparkedgeflashinfer-cutlassfp4fp8-kv-cachefunction-callinggb10gemma-4googlegpugrace-blackwellinstructllmcompressorlong-contextmixture-of-expertsmoemultimodalnative-fp4nvfp4nvidiaon-deviceopenai-apiopenai-compatibleprefix-cachingproduction-readyquantizedreasoningrefusal-removedsliding-window-attentionsm_121asparse-moespeculative-decodingthinkingtool-callingtransformeruncensoredunfilteredvisionvision-languageweight-quantization8-bit
README.md说明 / 使用方法

license: gemma library_name: transformers pipeline_tag: text-generation base_model: TrevorJS/gemma-4-26B-A4B-it-uncensored tags:

  • 26b
  • 4-bit
  • 4b-active
  • aarch64
  • abliterated
  • aeon
  • aeon-7
  • agentic
  • arm64
  • blackwell
  • chat
  • chunked-prefill
  • coding
  • compressed-tensors
  • conversational
  • dflash
  • dgx-spark
  • edge
  • english
  • flashinfer-cutlass
  • fp4
  • fp8-kv-cache
  • function-calling
  • gb10
  • gemma
  • gemma-4
  • gemma4
  • google
  • gpu
  • grace-blackwell
  • image-text-to-text
  • instruct
  • llmcompressor
  • long-context
  • mixture-of-experts
  • moe
  • multimodal
  • native-fp4
  • nvfp4
  • nvidia
  • on-device
  • openai-api
  • openai-compatible
  • prefix-caching
  • production-ready
  • quantized
  • reasoning
  • refusal-removed
  • safetensors
  • sliding-window-attention
  • sm_121a
  • sparse-moe
  • speculative-decoding
  • text-generation
  • thinking
  • tool-calling
  • transformer
  • uncensored
  • unfiltered
  • vision
  • vision-language
  • vllm
  • weight-quantization model_type: gemma4 quantization: nvfp4 language:
  • en
  • zh

Tips

快速链接

资源链接
模型权重 + 完整文档HuggingFace 上的 AEON-7/Gemma-4-26B-A4B-it-Uncensored-NVFP4
DFlash vLLM 容器 (DGX Spark)ghcr.io/aeon-7/aeon-vllm-ultimate:latest
DFlash 草稿模型 (Drafter)z-lab/gemma-4-26B-A4B-it-DFlash

快速开始

完整的开箱即用复制粘贴配方 —— 拉取容器、拉取模型（全新下载）、拉取 DFlash 草稿模型（全新下载），然后使用经过严格验证的 DGX Spark 配置配方进行部署服务（DFlash n=10，草稿模型 flash_attn，主干模型 triton_attn）。下方的配方说明涵盖了针对特定镜像标签的草稿模型后端以及生产级 GPU 利用率 / 上下文分级配置；另请参阅容器镜像详情。

# 1. 拉取 AEON vLLM Ultimate 镜像（vLLM 0.24.0 sm_121a 源码编译 + PR#44389 NVFP4-KV +
#    PR#40898/#41703 DFlash 草稿模型修复 + DFlash 高并发修复）。
#    :latest = :2026-07-01-v0.24.0；回滚版本 :2026-06-18-v0.23.0-dflashfix。
docker pull ghcr.io/aeon-7/aeon-vllm-ultimate:latest

# 2. 下载目标模型（全新下载）。
huggingface-cli download AEON-7/Gemma-4-26B-A4B-it-Uncensored-NVFP4 \
  --local-dir ./models/gemma4

# 3. 下载 DFlash 草稿模型（全新下载 —— 拉取最新的 z-lab 构建）。
huggingface-cli download z-lab/gemma-4-26B-A4B-it-DFlash \
  --local-dir ./models/gemma4-dflash

# 4. 部署服务 —— 适用于 :latest (2026-06-11+) 的配置方案：DFlash n=10，草稿模型 flash_attn，主干模型 triton_attn。
#    （该镜像的 ENTRYPOINT 默认为 bash，因此使用 --entrypoint vllm 覆盖。）
docker run --gpus all --ipc host --network host \
  -e VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
  -e TORCH_MATMUL_PRECISION=high \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  -e VLLM_TEST_FORCE_FP8_MARLIN=0 \
  -e VLLM_USE_FLASHINFER_SAMPLER=1 \
  -v "$PWD/models/gemma4:/models/gemma4:ro" \
  -v "$PWD/models/gemma4-dflash:/models/gemma4-dflash:ro" \
  --entrypoint vllm \
  ghcr.io/aeon-7/aeon-vllm-ultimate:latest \
  serve /models/gemma4 \
    --served-model-name gemma4-aeon-uncensored gemma4-fast gemma4-deep \
    --host 0.0.0.0 \
    --port 8000 \
    --tensor-parallel-size 1 \
    --dtype auto \
    --quantization compressed-tensors \
    --linear-backend flashinfer_cutlass \
    --moe-backend cutlass \
    --attention-backend triton_attn \
    --max-model-len 262144 \
    --max-num-seqs 64 \
    --max-num-batched-tokens 32768 \
    --gpu-memory-utilization 0.65 \
    --enable-chunked-prefill \
    --enable-prefix-caching \
    --trust-remote-code \
    --enable-auto-tool-choice \
    --tool-call-parser gemma4 \
    --reasoning-parser gemma4 \
    --speculative-config '{"method":"dflash","model":"/models/gemma4-dflash","num_speculative_tokens":10,"attention_backend":"flash_attn"}'

配方说明 —— 草稿模型后端与镜像标签强绑定：

  • 在 :latest / :2026-07-01-v0.24.0（当前最新；与 :2026-06-18-v0.23.0-dflashfix 修复分支行为相同）上： 草稿模型必须使用 attention_backend: flash_attn（通过 PR #41703 的 use_mm_prefix=False 为多模态 Gemma 目标启用）。在此镜像上，flex_attention 会在收到第一个请求时崩溃（新 KV 共享路径中的非连续 KV 视图）。--enable-prefix-caching 是安全的且推荐开启 —— 已在 DFlash 上经过浸泡测试验证（参见下方的修复部分）。
  • 在回滚标签 :2026-06-04-pr44389（修复前）上： 情况正好相反 —— 仅支持加载 flex_attention（flash_attn 会失败并报错 partial multimodal token full attention not supported），草稿模型会静默运行且不带滑动窗口支持（在超过 2k 上下文时接受率接近 0%），且 --enable-prefix-caching 会导致接受率缓慢崩溃（见下文）。不建议使用。
  • 无论哪种情况，主干模型均保持使用 triton_attn（针对 Gemma 的异构注意力头维度），且切勿设置 --kv-cache-dtype fp8 —— DFlash 的非因果草稿模型需要 BF16 KV。--reasoning-parser gemma4 可以干净地将 enable_thinking 的输出拆分到 reasoning_content 中。num_speculative_tokens=10 是我们生产集群当前运行的值；草稿模型的训练块大小为 16，而 z-lab 发布的默认值为 15 —— 针对修复后的草稿模型进行 n=10 与 15 的重新基准测试仍在进行中（此前“10 胜过 15”的结论是在修复前的草稿模型上测得的，不应视为定论）。该模型的 config.json 已经自带视觉忽略列表修复（re:.*embed_vision.*，re:.*vision_tower.*），因此 vLLM 会保持视觉塔为 BF16 精度。

此默认配置文件适用于智能体网关（agentic gateways）—— 一个大型全上下文对话加上受 --max-num-seqs 64 上限约束的多个短生命周期子智能体。对于短上下文吞吐量基准测试，请使用 --max-model-len 32768 --max-num-seqs 256。

性能表现 —— DGX Spark：原生默认 vs 极致优化

这是本模型的核心亮点。 在 DGX Spark / GB10 上，AEON DFlash 容器将默认“能跑，但很慢”的原生基线转变为一个真正可用的长上下文本地智能体模型 —— 并且 AEON 源码构建版本现在可以平稳扩展至 64 个并发请求，而在先前的镜像中，并发推测解码会导致崩溃。

测试基于 AEON 源码构建版本（vLLM 0.23.0 sm_121a + DFlash num_speculative_tokens: 10，草稿模型 flash_attn，主干模型 triton_attn；当前 :latest 现为 vLLM 0.24.0，吞吐量持平），单流（c=1），按任务类别划分：

类别🟢 解码 tok/sTTFT p50TPOT p50预填充 (PP)DFlash 接受率对比原生
代码编写 (Coding)155.883 ms6.4 ms601 tok/s58.9%3.2×
数学计算 (Math)127.8145 ms7.8 ms420 tok/s48.7%2.6×
逻辑推理 (Reasoning)118.9105 ms8.4 ms439 tok/s43.9%2.4×
散文写作 (Prose)49.8105 ms20.1 ms324 tok/s11.1%1.0×
自然语言 (Natural language)67.397 ms14.9 ms393 tok/s20.0%1.4×
信息提取 / JSON (Extraction / JSON)202.485 ms4.9 ms602 tok/s77.5%4.3×

对比 原生未经优化的 vllm/vllm-openai 0.20.1 基线（约 47–49 tok/s，无 DFlash，无 sm_121a 优化）→ 解码速度最高提升 4.3 倍（平均约 2.5 倍），且现在可平稳扩展至 c=64 并发（代码类别总计 1,937 tok/s）且无崩溃。参见我们为 DGX Spark 所做的修复。

关于原生基线的说明： 原生/未优化数据采用官方原生 vLLM（vllm/vllm-openai 0.20.1，无 DFlash，无 AEON/sm_121a 优化）测得，属于临时数据 —— 待更新，后续将在当前版本上进行全新的全原生重新基准测试。优化数据测自当前的 aeon-vllm-ultimate:latest（vLLM 0.24.0）构建版本。（完整的原生扫描数据及来源保留在下文。）

长上下文层级（当前构建版本）

随着上下文增长，DFlash 接受率依然保持稳定 —— DFlash 滑动窗口注意力修复（PR #40898）使草稿模型在远超 2k 标记后依然发挥效用。单流（c=1），以测得的 prompt_tokens_p50 作为上下文大小：

上下文类别解码 tok/sTTFT p50预填充 (PP)DFlash 接受率
~16k代码编写 (Coding)128.23.8 s4,270 tok/s58.7%
~16k逻辑推理 (Reasoning)102.85.2 s3,915 tok/s46.4%
~16k信息提取 / JSON (Extraction / JSON)169.85.2 s3,890 tok/s77.5%
~32k代码编写 (Coding)92.810.2 s3,188 tok/s46.7%
~32k逻辑推理 (Reasoning)83.014.1 s2,899 tok/s40.8%
~32k信息提取 / JSON (Extraction / JSON)156.314.1 s2,888 tok/s77.5%

模型规格

属性规格值
架构Gemma 4 混合专家架构 (Mixture of Experts)
总参数量 / 激活参数量26B / 每个 token 约 4B（128 个专家中激活 top-8）
层数30 层（25 层滑动窗口 + 5 层全局全注意力）
最大上下文262,144 tokens
量化方式NVFP4 (compressed-tensors)
磁盘模型大小15.3 GB
显存占用16.25 GB
视觉模块27 层 ViT (BF16)
工具调用原生 Gemma 4 格式

2026-06-11 — DFlash 草稿模型修复 (vLLM PR #40898 + #41703)

该模型先前发布的所有 DFlash 数据均是在存在缺陷的草稿模型执行路径上测量的。我们在生产环境中定位了 vLLM 端的三个根本缺陷，并将上游修复（目前两个 PR 在上游仍处于 open 状态）合并到了 aeon-vllm-ultimate:latest 中：

缺陷（修复前镜像）症状表现修复方式
被拒绝的 token 上下文 KV 写入损坏了草稿模型的分页 KV 缓存；--enable-prefix-caching 使这种损坏变得持久且自加速草稿接受率在数分钟到数小时的流量中从 34–56% 衰减至 0.0% → 解码速度放慢约 6 倍（从 144 降至 24 tok/s），唯有重启才能恢复#41703 屏蔽被拒绝/无效的上下文槽位
z-lab 草稿模型在 5 层中有 4 层为 滑动窗口 2048 (sliding-window-2048)，但先前按全注意力运行任何上下文大于 2k token 的请求接受率均接近 0%（长对话、智能体历史记录、超长系统提示词）#40898 为 DFlash 添加滑动窗口注意力 (SWA) 支持
草稿模型路径中缺失 Gemma-4 sqrt(hidden) 嵌入归一化器 + 最终 logit softcap压低了接受率上限 —— 平均接受长度 (MAL) 为 4.4–6.6，而 z-lab 公布的基准为 6.1–8.6#41703

修复后镜像的验证结果（生产配置：--gpu-memory-utilization 0.68 --max-model-len 184320 --max-num-seqs 32 --enable-prefix-caching，草稿模型 flash_attn，n=10）：

测试关卡修复前2026-06-11-pr41703
长上下文（约 9k 系统提示词）草稿接受率~0–7%43.3% / MAL 5.3
开启前缀缓存 + 集群并发峰值 + 10 分钟生产浸泡测试在约 25 分钟内崩溃至 0%52.0% / MAL 6.20 —— 负载下持续向好
单流代码生成 (c=1, greedy)刚启动 144 tok/s，逐步衰减至 ~24149–150 tok/s，持久稳定
长上下文（约 9k）吞吐量~46 tok/s（APC 无法使用）78 tok/s（APC 加速已缓存的前缀）
线上生产探测（30 种人设语音集群）—60% 接受率 / MAL 7.0

在此配置下的 KV 缓存容量：在 180k 上下文下提供 726k token / 3.94× 并发度。平均接受长度首次达到 z-lab 公布的指标范围。在修复后镜像上的完整按类别并发重新测试正在进行中；下一节为历史修复前的基准测试数据。

性能表现 —— 历史短上下文基准测试（修复前镜像，2026-06-09）

⚠️ 测试于修复前镜像（:2026-06-04-pr44389），采用在当前 :latest 上会崩溃的 flex-drafter 配方 —— 且草稿模型存在缺陷（无 SWA、无嵌入归一化器），仅限短上下文提示词。保留此项数据作为真实的历史底线参考；修复后的镜像实测性能更高（见上文）。在此镜像上的长上下文表现远差于表中所示（超过 2k token 的上下文接受率接近 0%）。

按类别单流 + 并发扫描（c=1…128，每个类别全新启动服务，0 请求错误），DFlash n=10，BF16 KV：

类别c=1 tok/s峰值聚合 tok/s对比先前的 DFlash v2 (n=15)
代码编写 (Coding)144.21,724 @ c=128+53% c1 / +51% 峰值
数学计算 (Math)99.61,335 @ c=128+35% / +34%
逻辑推理 (Reasoning)73.61,087 @ c=128+21% / +24%
信息提取 / JSON (Extraction / JSON)158.41,315 @ c=32+2% c1
自然语言 (Natural language)53.8684 @ c=128+5% 峰值
散文写作 (Prose)36.5486 @ c=128—

这次修复前测试显示，在偏向推理的类别中，n=10 比 z-lab 默认的 15 高出 20–50% —— 但当时两组测试运行的都是有缺陷的草稿模型，因此在修复后镜像上完成重新基准测试前，关于 n=10 与 15 的对比结论应视为未定论（草稿模型的块大小为 16）。

若追求极致的多请求聚合吞吐量（极高并发，c≥128），在修复前镜像上测得不带草稿模型的效果最好（通过下方原生路径在 c=256 时达到约 3,000–3,700 tok/s）。请注意，当时观察到的“草稿模型在 c=256 时失稳”的行为，很可能正是如今已修复的 KV 损坏缺陷所致 —— 在将其作为当前指导之前，高并发交叉点需要在修复后镜像上重新测量。

原生社区版 vLLM 基线（无 DFlash）

使用 2026-05-06 拉取的官方社区镜像 vllm/vllm-openai:latest 进行基准测试（vLLM 0.20.1，PyTorch 2.11.0+cu130，transformers 5.7.0，镜像摘要 sha256:9eff9734a30b6713a8566217d36f8277630fd2d31cec7f0a0292835901a23aa4）。本次运行使用相同的模型权重、32K 上下文、--max-num-batched-tokens 32768 以及 --max-num-seqs 256，但未配置 DFlash 草稿模型，也没有设置 AEON 容器环境变量覆盖项。上游 vLLM 目前已支持在 GB10 上通过 FlashInfer CUTLASS NVFP4 线性核函数和 VLLM CUTLASS MoE 启动此模型。

全量基准扫描：6 个自然提示词类别 × 8 个并发级别（1, 4, 8, 16, 32, 64, 128, 256）= 48 个基准测试点，0 次请求错误。

类别c=1 tok/sc=1 TTFT p50峰值聚合 tok/sc=256 聚合 tok/sc=256 TTFT p50
代码编写 (Coding)49.12130.7 ms3,356.61 @ c=2563,356.61542 ms
数学计算 (Math)48.79134.0 ms3,006.60 @ c=2563,006.601,078 ms
逻辑推理 (Reasoning)48.90113.8 ms3,241.42 @ c=2563,241.42274 ms
散文写作 (Prose)48.86115.9 ms3,222.85 @ c=2563,222.85662 ms
自然语言 (Natural language)49.3872.4 ms3,418.94 @ c=2563,418.94650 ms
信息提取 / JSON (Extraction / JSON)47.34120.6 ms3,674.70 @ c=2563,674.70385 ms

当纯粹的多请求聚合吞吐量比推测单流速度更重要时，请使用原生社区路径。当您需要更低的交互式 TPOT 并享受集成的 Gemma 4 DFlash 服务方案时，请使用 AEON DFlash 镜像。

为什么这项任务极其艰难：DGX Spark 上的 Gemma 4

在 DGX Spark 上运行 Gemma 4 NVFP4 过去需要完全从源码构建的软件栈。截至 2026-05-06 的社区 vllm/vllm-openai:latest 镜像，上游 vLLM 已经可以在 GB10 上启动该模型，而 AEON 的 aeon-vllm-ultimate 镜像则将优化（且现已修正 —— 见上文修复部分）后的 DFlash 路径打包成了一键拉取即用的容器。从底层硬件芯片到推理服务框架，再到模型权重本身，该技术栈的每一层都存在过值得深入了解的兼容性鸿沟。

DGX Spark 带来的挑战

NVIDIA DGX Spark 搭载了 GB10 Grace Blackwell 芯片：基于 ARM64 (aarch64) 的 SM 12.1 架构。这是绝大多数机器学习生态仍在努力追赶适配的前沿硬件：

  • SM 12.1 上的 Python Wheel 仍存在较高风险。 官方 PyPI 发布的二进制轮子历史上一贯面向 SM 8.0/8.9/9.0（Ampere/Ada/Hopper）。直接运行 pip install vllm 可能会安装针对错误 GPU 架构编译的 CUDA 核函数；请务必使用经过测试的 Docker 镜像或从源码编译。
  • SM 12.1 缺少预构建的 FlashInfer wheel。 FlashInfer 提供了融合 MoE 分发核函数，这是实现高速专家路由的关键。如果没有针对该架构进行编译，MoE 模型将无法使用经过优化的 CUTLASS/Triton 后端。
  • ARM64 架构 意味着许多仅限 x86 的预构建二进制文件根本无法运行。即便有些软件包宣称支持 CUDA，其主机端（host-side）代码通常也仅针对 x86 编译。
  • 273 GB/s 内存带宽：对于桌面级设备而言非常出色，但与数据中心级 GPU 相比（H100: 3.35 TB/s, A100: 2 TB/s）仍有较大差距。这使得模型架构的选择至关重要：在每个 token 生成过程中都需要读取全部参数的稠密模型在此类硬件上会遭受严重的带宽瓶颈。

实际结果是：当前的原生 vLLM 虽然可以部署该模型，但若要构建高可靠性的生产环境，仍需严格锁定镜像版本、模型格式、注意力后端、KV 数据类型以及并发设置，而不是盲目假定任何 vLLM 标签都能表现一致。

Gemma 4 带来的挑战

Gemma 4 不仅仅是一个新模型，它的架构非常独特，打破了现有工具链中的诸多惯性假设：

1. 必须依赖 transformers v5+（目前尚无其他模型强制要求）

Gemma 4 是首个要求将 transformers 大版本升级到 v5 的主流模型。较旧的原生 vLLM 镜像内置的是 v4.x，无法解析 Gemma 4 的配置。当前的社区镜像可能已包含 transformers v5，但仍需固定版本，因为 v4/v5 之间的 API 差异仍可能导致模型加载失败。

2. 异构注意力头维度

大多数模型在所有层上都具有统一的注意力头维度。而 Gemma 4 的滑动窗口层为 head_dim=256，全局全注意力层则为 global_head_dim=512。这直接破坏了假设所有层头维度统一的注意力后端。vLLM 专为 Gemma 4 强制启用了 TRITON_ATTN 后端以处理这种情况 —— 其他后端（FlashAttention、FlashInfer attention）会导致数值异常发散或程序崩溃。

3. 滑动窗口与全注意力层混合架构

在全部 30 层中，有 25 层使用 1024 token 的滑动窗口注意力，另外 5 层使用全局全注意力；且所有 30 层均挂载 MoE 块（128 个专家，激活 top-8 —— 经权重校验：每一层都包含路由器和专家张量）。这两种注意力类型具有不同的计算模式和 KV 缓存需求 —— 交织贯穿于整个模型层级中。

4. 庞大的 MoE 专家数量

每层拥有 128 个专家，且全 30 层均执行 top-8 路由。这意味着共有 128 × 30 = 3,840 个专家权重块，每个权重块包含 4 个 NVFP4 张量（weight_packed、weight_scale、weight_global_scale、input_global_scale）。该模型中的张量总数达到了 47,648 个。要正确加载和路由这些张量，需要能够处理堆叠专家格式的 FusedMoE 核函数，且 compressed-tensors 的命名规范与 vLLM 预期的规范不匹配（见下文）。

NVFP4 量化带来的挑战

NVFP4（NVIDIA 4 位浮点 —— 具有块缩放因子的 E2M1 元素）是将一个 26B 参数模型压缩进 15.3 GB 的关键。但在开源生态中存在两种完全不同的 NVFP4 格式，彼此之间互不兼容：

ModelOpt NVFP4（NVIDIA TensorRT-LLM 工具链）：将权重存储为 weight、weight_scale_inverse、input_scale。这是 NVIDIA 自家工具导出的格式，也是大多数 vLLM NVFP4 代码路径所预期的格式。

Compressed-tensors NVFP4（llmcompressor/vLLM 社区）：将权重存储为 weight_packed、weight_scale、weight_global_scale、input_global_scale。张量名称不同、缩放比例约定不同、打包格式也不同。

本项目模型使用的是 compressed-tensors 格式（在 H200 上使用 llmcompressor 进行量化）。vLLM 的 Gemma 4 权重加载器中硬编码了不匹配的张量命名假设。具体表现为：

  • 专家路径不匹配：Compressed-tensors 将 MoE 专家命名为 layers.X.experts.{id}.{proj}.weight_packed。而 vLLM 的 FusedMoE 预期为 layers.X.moe.experts.{id}.{proj}.weight_packed —— 请注意其中的 .moe. 节点。若不打补丁，每一个专家张量都会加载失败并抛出 KeyError。
  • 后缀格式不匹配：权重加载器构造的名称类似于 w2_weight.weight_packed，而实际应该为 w2_weight_packed。需要将 _weight. 简化为 _。
  • 维度断言失败：原生代码对权重张量断言 dim == 2，但 NVFP4 打包张量由于 4-bit 打包具有不同的张量维度。

本仓库附带的 gemma4_patched.py 通过对权重加载管线进行针对性修补，一次性解决了上述所有三个问题。

误量化问题

在使用 llmcompressor 进行量化时，通常需要指定忽略模式以保留需要保持 BF16（全精度）的层。最初的量化使用了类似 re:.*visual.* 和 re:.*gate.* 的正则模式来跳过视觉和路由层。但 Gemma 4 的命名规范与此不符：

层类型预期正则匹配Gemma 4 中的实际名称结果
视觉塔re:.*visual.*model.vision_tower.*被误量化（错误）
视觉嵌入re:.*visual.*model.embed_vision.*被误量化（错误）
MoE 路由器re:.*gate.*model.*.router.proj.*被误量化（错误）

量化这些层会导致模型彻底损坏：

  • NVFP4 格式的视觉塔会直接导致崩溃，因为 vLLM 会为其分配标准的 Linear 层（预期为 .weight 张量，却收到了 weight_packed/weight_scale 等）
  • NVFP4 格式的 MoE 路由器 会损坏专家路由机制 —— 路由器负责决定为每个 token 激活哪些专家，对路由 logit 进行 4 位量化会导致退化的异常专家选择
  • 视觉嵌入投影层 连接 ViT 输出与语言模型 —— 此处的量化误差会级联传导到后续的每一层

我们通过从基础模型（TrevorJS/gemma-4-26B-A4B-it-uncensored）中提取原始 BF16 权重，替换 safetensors 文件中被错误量化的张量修复了此问题：

  • 从视觉塔中移除了 760 个 NVFP4 张量，替换为 190 个原始 BF16 权重（包含偏置和 layernorm 在内共 355 个视觉张量）
  • 从 router.proj 层移除了 120 个 NVFP4 张量，替换为 30 个 BF16 权重
  • 从 embed_vision 中移除了 4 个 NVFP4 张量，替换为 1 个 BF16 权重

控制 Token 泄漏问题

Gemma 4 使用内部控制 token 进行多通道生成（思考思维链、工具调用、最终输出）。这些 token 在词表中具有特定的 ID：

Token IDToken作用说明
100<|channel>开始内部通道（例如：思考）
101<channel|>结束内部通道
98<|think|>进入思考模式
48<|tool_call>开始工具调用
49<tool_call|>结束工具调用

如果没有正确的 EOS 配置，模型可能会在生成过程中误入其“思考”通道，导致这些内部控制 token 作为纯文本直接输出到 API 响应中。更严重的是，它可能会陷入无限重复循环 —— 没完没了地生成 <|channel>thought<channel|>call:process{...} 作为可见文本。表现为模型在对话中不断“刷屏”乱码。

修复方法是在 generation_config.json 的 eos_token_id 列表中加入 token 98、100 和 101，确保 vLLM 在任何内部通道 token 泄露至最终输出前干净利落地终止生成。

容器中包含的核心技术（秘密配方）

当前的容器镜像是 ghcr.io/aeon-7/aeon-vllm-ultimate:latest（= :2026-07-01-v0.24.0）。用户只需拉取一个镜像，无需任何本地打补丁或从源码编译操作。

组件属性与功能核心价值
vLLM 0.24.0 (sm_121a 源码编译) + PR #44389专为 GB10 构建的 v0.24.0 + Triton NVFP4 KV 缓存可选开启最高达 3 倍的 KV 缓存容量（--kv-cache-dtype nvfp4；仅限因果推测解码）。
PR #40898 + #41703 补丁叠加领先上游提前合并的 DFlash 草稿模型修复屏蔽被拒绝槽位的 KV 写入（在前缀缓存下杜绝接受率崩溃）、草稿模型滑动窗口支持、Gemma-4 嵌入归一化器 + logit softcap、多模态 Gemma 目标上的 flash_attn 草稿模型。
AEON sm_121a 专属补丁3 项幂等运行时补丁确保 GB10 正确性（可选导入延迟绑定、混合块大小 None 保护、推测解码 CUDA 图捕获大小对齐）。
PyTorch 2.11.0 + CUDA 13底层框架与运行时环境支持 SM 12.1 (GB10) 硬件架构。
transformers 5.10 dev模型配置与分词器加载支持识别 gemma4_unified 等前沿模型类。
DFlash 草稿模型z-lab/gemma-4-26B-A4B-it-DFlash（单独挂载）5 层 SWA-2048 块扩散草稿模型 —— 现已真正以滑动窗口运行。
原生 FP4 CUTLASS 核函数线性层使用 FlashInfer CUTLASS，MoE 使用 VLLM CUTLASS切勿在此镜像上强行启用 Marlin；在 GB10 上原生 FP4 路径性能更优。
主干 TRITON_ATTN / 草稿 FLASH_ATTN注意力计算引擎Triton 妥善处理 Gemma 4 的异构头维度 (256/512)；草稿模型的非因果注意力运行于 FlashAttention 之上（本镜像强制要求）。
torch.compile + CUDA 图图捕获与算子融合为所配置的批大小捕获解码计算图，削减解码热点路径上的 Python 调用开销。

为什么 MoE 使这一切成为可能

DGX Spark 的根本瓶颈在于内存带宽：273 GB/s。在自回归解码期间，GPU 必须为生成的每一个 token 完整读取一遍模型权重。这直接决定了 tok/s 的理论上限：

tok/s = 内存带宽 / 每 token 读取的字节数

对于一个 NVFP4 精度的稠密 27B 模型（权重约 13.5 GB）：

273 GB/s / 13.5 GB = ~20 tok/s（理论极限，尚未计入 KV 缓存与开销）

而对于本 MoE 模型（128 个专家中激活 top-8，每个 token 仅激活约 2.8 GB）：

273 GB/s / 2.8 GB = ~97 tok/s（理论极限）

我们实现了单流约 37–158 tok/s（视任务类别而定 —— 代码编写在修复后镜像上持久保持 149–150 tok/s；散文约 37 tok/s，信息提取/JSON 约 158 tok/s 来自修复前的短上下文测试，属保守下限，修复后镜像重新测试进行中）—— 在代码任务上聚合吞吐量突破 1,700 tok/s（修复前测试），在信息提取上通过原生无草稿路径达到约 3,700 tok/s。与理论极限的差距主要来自于 KV 缓存读取、注意力计算、路由器开销、草稿模型验证以及内存访问模式。但核心结论在于：MoE 将一个在带宽上面临不可能完成的任务（稠密 27B）转变成了一个带宽充裕的从容任务。

模型类型每 Token 读取参数量GB10 上理论最高 tok/s实际可用 tok/s
稠密 27B BF16~54 GB5无法实用
稠密 27B NVFP4~13.5 GB20~15
MoE 26B top-8/128 NVFP4 + DFlash~2.8 GB + 草稿模型97代码编写 149–150（修复后镜像）；自然对话约 54 / 提取约 158（修复前测试）；聚合 1,700+

这就是为什么在带宽受限的硬件上，架构选择远比盲目的参数总量更为重要。在相同硬件上，采用 NVFP4 的 26B MoE 模型的推理速度远超 BF16 的稠密 7B 模型。

我们为 DGX Spark 所做的修复

所有 AEON 模型均运行在同一个统一容器镜像上 —— ghcr.io/aeon-7/aeon-vllm-ultimate:latest（= :2026-07-01-v0.24.0；回滚版本 :2026-06-18-v0.23.0-dflashfix）—— 该镜像是由针对 GB10 / sm_121a 从源码构建的 vLLM v0.24.0 与 AEON 推测解码优化技术栈深度融合而成。

修复项目功能说明为什么对 GB10 至关重要
DFlash 高并发修复 (新)将推测草稿模型的 KV 块表切片至未填充的实际批次（block_table[:num_reqs]）修复前草稿模型在并发请求数 ≥32 时崩溃（FlashAttention 中填充与未填充块表形状不匹配）。现已平稳扩展至 c=64。此改动将上游 PR #43982（先前仅针对 MTP 修复，从未应用于 DFlash）移植至 DFlash —— 甚至在先前的镜像中该问题也依然存在。
Triton NVFP4 KV 缓存 (PR #44389)软件实现的 NVFP4 KV 缓存路径sm_121a 架构上唯一的 4-bit KV 路径（上游官方实现强行限定必须为 B200）→ 获得约 3 倍 KV 缓存容量 / 每 GB 统一内存支持更长的上下文。
DFlash 滑动窗口注意力 (PR #40898)将草稿模型的 SWA 层作为真正的滑动窗口执行随着智能体对话历史的延长，长上下文草稿接受率得以维持（该模型在 ~16k 时约为 47%，在 ~32k 时约为 41%），不再像先前一样在超过约 2k token 后断崖式崩溃。
sm_121a 原生构建TORCH_CUDA_ARCH_LIST=12.1a, ENABLE_NVFP4_SM100=0编译 GB10 实际调度执行的 SM120 家族 CUTLASS NVFP4/FP8 核函数 —— 释放真正的 4 位 Tensor Core 吞吐量，去除无用的 B200 专属无用核函数。
sm_121a 启动与 CUDA 图补丁RTLD-lazy 延迟加载 _C_stable_libtorch；对齐推测解码 CUDA 图捕获尺寸绕过 GB10 上缺失的 MXFP4（仅限 SM100）符号依赖顺利启动；防止在推测解码的部分接受解码步骤中触发 cudaErrorIllegalAddress 错误。
统一内存调优--gpu-memory-utilization 0.6-0.7，全量 CUDA 图，异步调度，z-lab DFlash 草稿模型GB10 在 CPU 和 GPU 之间共享同一片 LPDDR5X 统一内存池；请保持 gpu-util 在 0.6-0.7（若存在共存服务 / 高并发 / fp16 KV / DFlash 则调得更低）—— 高于约 0.8 时共享内存池会发生频繁换页颠簸并使整机卡死，即便 0.85 也会停滞 —— 在此设置下可同时兼顾全图捕获与推测解码吞吐。

针对该模型的实际成效：

  • 平稳扩展至 64 个并发请求且绝无崩溃（先前的镜像在推测解码下 c≥32 即崩溃）—— 在代码任务上 c=64 时聚合吞吐量高达 1,937 tok/s。
  • 在 Blackwell Tensor Core 上实现原生 NVFP4 4-bit 计算 —— 兼具 4 位的极致速度与接近 16 位的运算精度。
  • 推测解码 (DFlash) 从短提示词（c=1 时代码编写 58.9% / 信息提取 77.5%）到超长智能体历史（16k 时约 47%，32k 时约 41%）均能保持高接受率。
  • 相比原生未优化的官方 vLLM 基线，单流解码速度最高提升 4.3 倍（各类别平均提升约 2.5 倍）。

关于原生基线的补充说明： “原生 / 未优化”对比数据来自原生官方 vLLM（vllm/vllm-openai 0.20.1，默认配置，无推测解码，无 DGX-Spark / sm_121a 专属优化）。这些属于临时数据，将在当前版本上完成全新的全原生重新基准测试后予以更新。优化后的数据测自最新的 aeon-vllm-ultimate:latest（vLLM 0.24.0）构建版本。

容器镜像详情

AEON vLLM Ultimate（强烈推荐）

ghcr.io/aeon-7/aeon-vllm-ultimate:latest（= :2026-07-01-v0.24.0）— 代码仓库与完整模型卡片

组件版本
vLLM0.24.0 sm_121a 源码编译 + PR #44389 NVFP4-KV + PR #40898/#41703 DFlash 修复 + DFlash 高并发修复（PR #43982 移植至 DFlash）
PyTorch2.11.0+cu130
transformers5.10.0.dev (HEAD)
flashinfer0.6.12
DFlash 草稿模型挂载 z-lab/gemma-4-26B-A4B-it-DFlash，草稿模型后端使用 flash_attn
目标 GPUNVIDIA GB10 (DGX Spark, sm_121a)；亦支持 RTX 50 系列 (sm_120)

固定版本标签：:2026-07-01-v0.24.0（当前 :latest），:2026-06-18-v0.23.0-dflashfix（回滚版本）。较旧的 :2026-06-11-pr41703（vLLM 0.22.1 分支，DFlash 在 c≥32 时崩溃）、:2026-06-04-pr44389（修复前版本，仅支持 flex 草稿模型，前缀缓存与 DFlash 搭配使用不安全）以及 aeon-gemma-4-26b-a4b-dflash:v2（vLLM 0.20.1）分支均已被废弃替换，仅供历史对比参考。

原生社区基准镜像

vllm/vllm-openai:latest@sha256:9eff9734a30b6713a8566217d36f8277630fd2d31cec7f0a0292835901a23aa4

组件版本
vLLM0.20.1
PyTorch2.11.0+cu130
transformers5.7.0
推测解码无

该镜像可用作当前上游官方的参照点。它并非 AEON DFlash 定制包，不包含 Gemma 4 DFlash 草稿模型支持路径。

包含的全部修复项

该模型在量化后需要进行多项关键修复才能在 vLLM 中正常运行。所有修复均已直接固化集成至本 HuggingFace 发布版本中 —— 无需用户进行额外的调试：

  • 反量化了 760 个视觉塔张量（27 个 ViT 层）、120 个路由器张量（30 个 MoE 层）以及 4 个嵌入投影张量 —— 全部从原始 BF16 权重中完整恢复
  • 修补了 vLLM 针对 compressed-tensors NVFP4 MoE 格式的权重加载器（gemma4_patched.py —— 针对 _weight_iterator 和 load_weights 的 3 处专门补丁）
  • 在 config.json 中补齐了 audio_config 与 num_experts_per_tok（满足 vLLM 配置解析器的严格要求）
  • 创建了 preprocessor_config.json 与 processor_config.json 以支持多模态输入
  • 配置了 EOS token ID 列表 [1, 106, 50, 98, 100, 101]，杜绝思考/内部通道 token 泄露

完整技术细节请参阅：HuggingFace 模型卡片

相关模型

模型类型体积tok/s (DGX Spark)相关链接
本项目模型 (Gemma 4 26B MoE + DFlash)MoE NVFP415.3 GB代码单流持续 149–150 (c=1, 修复后镜像) / 代码聚合 1,724 (修复前测试；重新测试中) / 信息提取聚合约 3,700 (原生无草稿路径)HuggingFace
Gemma 4 31B DECKARD AWQ_FULL稠密 NVFP420.5 GB~12-14HuggingFace | GitHub
Gemma 4 31B DECKARD SVDQuant稠密 NVFP420.9 GB~10-13HuggingFace
Qwen3.5-27B Uncensored稠密 NVFP4~15 GB~15-18HuggingFace

MoE vs 稠密模型：MoE 模型的运行速度比稠密模型快 3-4 倍，因为其每个 token 仅需读取约 4B 激活参数（128 个专家中激活 top-8），而稠密模型则需完整读取 27-31B 参数。追求高吞吐量与高并发时请选择 MoE，追求极致综合质量时可选择稠密模型。

免责声明、责任豁免与风险自负协议

这是一个未审查 (Uncensored) 的模型。 下载、访问或使用该模型、关联的容器镜像（ghcr.io/aeon-7/aeon-vllm-ultimate）或其任何衍生作品，即表明您明确知悉并同意以下条款：

风险承担

相较于经过安全对齐的模型，未审查语言模型具有显著升高的风险，包括但不限于：生成有害、误导性、非法或令人反感的内容；易受对抗性滥用；可能协助违反适用法律法规的活动；以及在输出可能未经人工审核即被执行的自动化或智能体管线中带来被放大的风险。

这些工具功能极其强大，可用于大量合法且至关重要的用途 —— 包括安全研究、红队对抗演练、内容分析、创意工作以及安全过滤器干扰正常业务流程的特定场景。然而，安全防护栏的缺位要求运营者承担相应更高的审慎注意义务。您必须根据您的具体使用场景和所在司法管辖区，自行实施适当的安全防护、内容过滤、访问控制和监控机制。

责任限制

本模型和容器镜像的作者、贡献者及分发者（“提供方”）对于因使用或滥用这些材料而导致的任何行为、生成的内容、遭受的损害或法律后果，概不承担任何直接或间接的责任。这包括但不限于：

  • 模型生成的任何有害、非法、不道德或令人反感的内容
  • 基于模型输出做出的任何决策或采取的任何行动
  • 因使用该模型或容器而产生的任何损害 —— 无论直接、间接、附带、衍生、特殊或惩罚性损害，即便提供方已被告知发生此类损害的可能性
  • 用户对任何当地、州、国家或国际法律法规的违反

用户责任

您（用户）对以下各项承担全部且唯一的责任和法律后果：

  • 在您的操作下模型生成的所有输出内容
  • 确保您的使用符合您所在司法管辖区的所有适用法律、法规及道德标准
  • 部署适当的访问控制、内容审查过滤和人工监督机制
  • 将该模型部署至生产环境、自动化系统或面向公众的系统所产生的任何后果
  • 审慎评估未审查模型是否适用于您的特定业务场景

条款接受

下载或使用本发布版本的任何组件 —— 包括模型权重、容器镜像、配置文件或补丁代码 —— 即表示您接受这些条款，并自愿承担所有相关的风险与责任。 如果您不同意这些条款，请勿下载或使用这些材料。

许可证

本模型继承了来自 Google 的 Gemma 许可证。

支持这项工作

如果本发布版本对您有所帮助，非常感谢您的打赏赞助。所有打赏都将直接投入到更多的算力开销、更多的模型开发和更多的开源发布中。

以太坊 L2 网络（Base、Arbitrum、Optimism、Polygon 等）及 EVM 兼容代币可直接转账至相同的以太坊地址。

源站原文

license: gemma library_name: transformers pipeline_tag: text-generation base_model: TrevorJS/gemma-4-26B-A4B-it-uncensored tags:

  • 26b
  • 4-bit
  • 4b-active
  • aarch64
  • abliterated
  • aeon
  • aeon-7
  • agentic
  • arm64
  • blackwell
  • chat
  • chunked-prefill
  • coding
  • compressed-tensors
  • conversational
  • dflash
  • dgx-spark
  • edge
  • english
  • flashinfer-cutlass
  • fp4
  • fp8-kv-cache
  • function-calling
  • gb10
  • gemma
  • gemma-4
  • gemma4
  • google
  • gpu
  • grace-blackwell
  • image-text-to-text
  • instruct
  • llmcompressor
  • long-context
  • mixture-of-experts
  • moe
  • multimodal
  • native-fp4
  • nvfp4
  • nvidia
  • on-device
  • openai-api
  • openai-compatible
  • prefix-caching
  • production-ready
  • quantized
  • reasoning
  • refusal-removed
  • safetensors
  • sliding-window-attention
  • sm_121a
  • sparse-moe
  • speculative-decoding
  • text-generation
  • thinking
  • tool-calling
  • transformer
  • uncensored
  • unfiltered
  • vision
  • vision-language
  • vllm
  • weight-quantization model_type: gemma4 quantization: nvfp4 language:
  • en

Tips

Quick Links

ResourceLink
Model Weights + Full DocumentationAEON-7/Gemma-4-26B-A4B-it-Uncensored-NVFP4 on HuggingFace
DFlash vLLM Container (DGX Spark)ghcr.io/aeon-7/aeon-vllm-ultimate:latest
DFlash Drafterz-lab/gemma-4-26B-A4B-it-DFlash

Quick Start

Complete copy-paste recipe — pull the container, pull the model (fresh), pull the DFlash drafter (fresh), then serve with the vetted DGX Spark recipe (DFlash n=10, drafter flash_attn, body triton_attn). The recipe notes below cover tag-specific drafter backends and the production GPU-util / context-tier profile; see also Container Image Details.

# 1. Pull the AEON vLLM Ultimate image (vLLM 0.24.0 sm_121a from-source + PR#44389 NVFP4-KV +
#    PR#40898/#41703 DFlash drafter fixes + DFlash high-concurrency fix).
#    :latest = :2026-07-01-v0.24.0; rollback :2026-06-18-v0.23.0-dflashfix.
docker pull ghcr.io/aeon-7/aeon-vllm-ultimate:latest

# 2. Download the target model (fresh).
huggingface-cli download AEON-7/Gemma-4-26B-A4B-it-Uncensored-NVFP4 \
  --local-dir ./models/gemma4

# 3. Download the DFlash drafter (fresh — pull the latest z-lab build).
huggingface-cli download z-lab/gemma-4-26B-A4B-it-DFlash \
  --local-dir ./models/gemma4-dflash

# 4. Serve — recipe for :latest (2026-06-11+): DFlash n=10, drafter flash_attn, body triton_attn.
#    (The image ENTRYPOINT is bash, so override it with --entrypoint vllm.)
docker run --gpus all --ipc host --network host \
  -e VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
  -e TORCH_MATMUL_PRECISION=high \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  -e VLLM_TEST_FORCE_FP8_MARLIN=0 \
  -e VLLM_USE_FLASHINFER_SAMPLER=1 \
  -v "$PWD/models/gemma4:/models/gemma4:ro" \
  -v "$PWD/models/gemma4-dflash:/models/gemma4-dflash:ro" \
  --entrypoint vllm \
  ghcr.io/aeon-7/aeon-vllm-ultimate:latest \
  serve /models/gemma4 \
    --served-model-name gemma4-aeon-uncensored gemma4-fast gemma4-deep \
    --host 0.0.0.0 \
    --port 8000 \
    --tensor-parallel-size 1 \
    --dtype auto \
    --quantization compressed-tensors \
    --linear-backend flashinfer_cutlass \
    --moe-backend cutlass \
    --attention-backend triton_attn \
    --max-model-len 262144 \
    --max-num-seqs 64 \
    --max-num-batched-tokens 32768 \
    --gpu-memory-utilization 0.65 \
    --enable-chunked-prefill \
    --enable-prefix-caching \
    --trust-remote-code \
    --enable-auto-tool-choice \
    --tool-call-parser gemma4 \
    --reasoning-parser gemma4 \
    --speculative-config '{"method":"dflash","model":"/models/gemma4-dflash","num_speculative_tokens":10,"attention_backend":"flash_attn"}'

Recipe notes — the drafter backend is tied to the image tag:

  • On :latest / :2026-07-01-v0.24.0 (current; same behavior as the :2026-06-18-v0.23.0-dflashfix fix lineage): the drafter must use attention_backend: flash_attn (enabled for multimodal Gemma targets by PR #41703's use_mm_prefix=False). flex_attention crashes at the first request on this image (non-contiguous KV view in the new KV-sharing path). --enable-prefix-caching is safe and recommended — soak-validated with DFlash (see the fixes section below).
  • On the rollback tag :2026-06-04-pr44389 (pre-fix): the reverse — only flex_attention loads (flash_attn fails with partial multimodal token full attention not supported), the drafter silently runs without sliding-window support (~0% acceptance beyond 2k-token contexts), and --enable-prefix-caching triggers a slow acceptance collapse (see below). Not recommended.
  • Either way the body stays on triton_attn (Gemma's heterogeneous head dims), and do not set --kv-cache-dtype fp8 — DFlash's non-causal drafter requires BF16 KV. --reasoning-parser gemma4 cleanly splits enable_thinking output into reasoning_content. num_speculative_tokens=10 is the value our production fleet runs; the drafter's trained block size is 16 and z-lab ships 15 — an n=10-vs-15 re-bench on the fixed drafter is pending (the earlier "10 beats 15" result was measured on the pre-fix drafter and shouldn't be treated as settled). The model's config.json already ships the vision ignore-list fix (re:.*embed_vision.*, re:.*vision_tower.*) so vLLM keeps the vision tower BF16.

This default profile suits agentic gateways — one large full-context chat plus many short-lived subagents under the --max-num-seqs 64 cap. For short-context throughput benchmarking, use --max-model-len 32768 --max-num-seqs 256.

Performance — DGX Spark: stock vanilla vs maximally optimized

This is the headline. On DGX Spark / GB10 the AEON DFlash container turns the default "it runs, but it's slow" stock baseline into a usable long-context local agent model — and the AEON from-source build now scales cleanly to 64 concurrent requests where the prior image crashed under concurrent speculative decoding.

Measured on the AEON from-source build (vLLM 0.23.0 sm_121a + DFlash num_speculative_tokens: 10, drafter flash_attn, body triton_attn; the current :latest is now vLLM 0.24.0 at throughput parity), single-stream (c=1), by category:

Category🟢 Decode tok/sTTFT p50TPOT p50Prefill (PP)DFlash acceptvs stock
Coding155.883 ms6.4 ms601 tok/s58.9%3.2×
Math127.8145 ms7.8 ms420 tok/s48.7%2.6×
Reasoning118.9105 ms8.4 ms439 tok/s43.9%2.4×
Prose49.8105 ms20.1 ms324 tok/s11.1%1.0×
Natural language67.397 ms14.9 ms393 tok/s20.0%1.4×
Extraction / JSON202.485 ms4.9 ms602 tok/s77.5%4.3×

vs a stock vanilla vllm/vllm-openai 0.20.1 baseline of ~47–49 tok/s (no DFlash, no sm_121a optimizations) → up to 4.3× faster decode (≈2.5× average), and it now scales cleanly to c=64 concurrent (1,937 tok/s aggregate on coding) with no crash. See What we fixed for the DGX Spark.

Stock baseline note: the stock/un-optimized figures are vanilla vLLM (vllm/vllm-openai 0.20.1, no DFlash, no AEON/sm_121a optimizations) and are provisional — to be refreshed with a fresh fully-vanilla re-benchmark on the current version. The optimized figures are measured on the current aeon-vllm-ultimate:latest (vLLM 0.24.0) build. (The full stock sweep with provenance is preserved below.)

Long-context tiers (current build)

DFlash acceptance holds as context grows — the DFlash sliding-window-attention fix (PR #40898) keeps the drafter useful well past 2k tokens. Single-stream (c=1), measured prompt_tokens_p50 as the context size:

ContextCategoryDecode tok/sTTFT p50Prefill (PP)DFlash accept
~16kCoding128.23.8 s4,270 tok/s58.7%
~16kReasoning102.85.2 s3,915 tok/s46.4%
~16kExtraction / JSON169.85.2 s3,890 tok/s77.5%
~32kCoding92.810.2 s3,188 tok/s46.7%
~32kReasoning83.014.1 s2,899 tok/s40.8%
~32kExtraction / JSON156.314.1 s2,888 tok/s77.5%

Model Specs

PropertyValue
ArchitectureGemma 4 Mixture of Experts
Total / Active Parameters26B / ~4B per token (top-8 of 128 experts)
Layers30 (25 sliding-window + 5 full-attention)
Max Context262,144 tokens
QuantizationNVFP4 (compressed-tensors)
Model Size on Disk15.3 GB
VRAM Loaded16.25 GB
Vision27-layer ViT (BF16)
Tool CallingNative Gemma 4 format

2026-06-11 — DFlash drafter fixes (vLLM PR #40898 + #41703)

All previously published DFlash numbers for this model were measured on a defective drafter path. We root-caused three vLLM-side defects in production and merged the upstream fixes (both PRs still open upstream) into aeon-vllm-ultimate:latest:

Defect (pre-fix images)SymptomFixed by
Rejected-token context-KV writes corrupted the drafter's paged KV cache; --enable-prefix-caching made the corruption persistent and self-acceleratingDraft acceptance decayed 34–56% → 0.0% over minutes-to-hours of traffic → ~6× decode slowdown (144 → 24 tok/s), healed only by restart#41703 masks rejected/invalid context slots
The z-lab drafter is 4-of-5-layers sliding-window-2048, but ran as full attentionAny request with >2k tokens of context got ~0% acceptance (long chats, agent histories, big system prompts)#40898 adds DFlash SWA support
Missing Gemma-4 sqrt(hidden) embed normalizer + final-logit softcap in the draft pathDepressed acceptance ceiling — mean acceptance length 4.4–6.6 vs z-lab's published 6.1–8.6#41703

Validated results on the fixed image (production profile: --gpu-memory-utilization 0.68 --max-model-len 184320 --max-num-seqs 32 --enable-prefix-caching, drafter flash_attn, n=10):

Gatepre-fix2026-06-11-pr41703
Long-context (~9k system prompt) draft acceptance~0–7%43.3% / MAL 5.3
Prefix-caching ON + fleet-burst + 10-min production soakcollapse to 0% in ~25 min52.0% / MAL 6.20 — improves under load
Single-stream coding (c=1, greedy)144 tok/s fresh-boot, decaying to ~24149–150 tok/s, sustained
Long-context (~9k) throughput~46 tok/s (APC unusable)78 tok/s (APC accelerates the cached prefix)
Live production probe (30-persona voice fleet)—60% acceptance / MAL 7.0

KV at this profile: 726k tokens / 3.94× concurrency at 180k ctx. Mean acceptance length now lands in z-lab's published range for the first time. A full per-category concurrency re-sweep on the fixed image is pending; the section below is the historical pre-fix sweep.

Performance — historical short-context sweep (pre-fix image, 2026-06-09)

⚠️ Measured on the pre-fix image (:2026-06-04-pr44389) with the flex-drafter recipe that crashes on the current :latest — and with the broken drafter (no SWA, no embed normalizer), short-context prompts only. Kept as an honest historical floor; the fixed image measures higher (see above). Long-context performance on this image was far worse than the table suggests (~0% acceptance beyond 2k-token contexts).

Per-category single-stream + concurrency sweep (c=1…128, fresh server per category, 0 request errors), DFlash n=10, BF16 KV:

Categoryc=1 tok/sPeak aggregate tok/svs prior DFlash v2 (n=15)
Coding144.21,724 @ c=128+53% c1 / +51% peak
Math99.61,335 @ c=128+35% / +34%
Reasoning73.61,087 @ c=128+21% / +24%
Extraction / JSON158.41,315 @ c=32+2% c1
Natural language53.8684 @ c=128+5% peak
Prose36.5486 @ c=128—

This pre-fix sweep showed n=10 beating z-lab's default 15 by 20–50% on the reasoning-heavy categories — but both arms ran the broken drafter, so treat the n=10-vs-15 conclusion as unsettled pending the re-bench on the fixed image (drafter block size is 16).

For maximum many-request aggregate throughput (very high concurrency, c≥128), serving without the drafter measured best on the pre-fix image (~3,000–3,700 tok/s at c=256 via the stock path below). Note the "drafter destabilizes at c=256" behavior observed then is plausibly the now-fixed KV-corruption defect — the high-concurrency crossover needs re-measuring on the fixed image before treating it as current guidance.

Stock Community vLLM Baseline (No DFlash)

Benchmarked with the official community image vllm/vllm-openai:latest pulled on 2026-05-06 (vLLM 0.20.1, PyTorch 2.11.0+cu130, transformers 5.7.0, image digest sha256:9eff9734a30b6713a8566217d36f8277630fd2d31cec7f0a0292835901a23aa4). This run used the same model weights, 32K context, --max-num-batched-tokens 32768, and --max-num-seqs 256, but no DFlash drafter and no AEON container env overrides. Upstream vLLM now boots this model on GB10 with FlashInfer CUTLASS NVFP4 linear kernels and VLLM CUTLASS MoE.

Full sweep: 6 natural prompt categories x 8 concurrency levels (1, 4, 8, 16, 32, 64, 128, 256) = 48 benchmark points, 0 request errors.

Categoryc=1 tok/sc=1 TTFT p50Peak aggregate tok/sc=256 aggregate tok/sc=256 TTFT p50
Coding49.12130.7 ms3,356.61 @ c=2563,356.61542 ms
Math48.79134.0 ms3,006.60 @ c=2563,006.601,078 ms
Reasoning48.90113.8 ms3,241.42 @ c=2563,241.42274 ms
Prose48.86115.9 ms3,222.85 @ c=2563,222.85662 ms
Natural language49.3872.4 ms3,418.94 @ c=2563,418.94650 ms
Extraction / JSON47.34120.6 ms3,674.70 @ c=2563,674.70385 ms

Use the stock community path when raw many-request aggregate throughput matters more than speculative single-stream speed. Use the DFlash image when you want the lower interactive TPOT and the integrated Gemma 4 DFlash serving recipe.

Why This Is Hard: Gemma 4 on DGX Spark

Running Gemma 4 NVFP4 on a DGX Spark used to require a source-built stack. As of the 2026-05-06 community vllm/vllm-openai:latest image, upstream vLLM can boot this model on GB10, and AEON's aeon-vllm-ultimate image packages the optimized (and now corrected — see the fixes section above) DFlash path as a single pull-and-run container. Every layer of the stack, from the silicon to the serving framework to the model weights themselves, has had compatibility gaps worth understanding.

The DGX Spark Problem

The NVIDIA DGX Spark ships with a GB10 Grace Blackwell chip: SM 12.1 on ARM64 (aarch64). This is bleeding-edge silicon that much of the ML ecosystem is still catching up to:

  • Python wheels remain risky on SM 12.1. Official PyPI releases have historically targeted SM 8.0/8.9/9.0 (Ampere/Ada/Hopper). Installing pip install vllm can give you CUDA kernels compiled for the wrong GPU; use a tested Docker image or build from source.
  • No pre-built FlashInfer wheels for SM 12.1. FlashInfer provides the fused MoE dispatch kernels that make expert routing fast. Without it compiled for your architecture, MoE models can't use the optimized CUTLASS/Triton backends.
  • ARM64 architecture means many x86-only prebuilt binaries don't run at all. Even when packages claim CUDA support, the host-side code is often x86-compiled.
  • 273 GB/s memory bandwidth: fast for a desktop-class device, but a fraction of what data center GPUs offer (H100: 3.35 TB/s, A100: 2 TB/s). This makes model architecture choice critical: dense models that need to read all parameters every token are bandwidth-starved here.

The practical result: current stock vLLM can serve this model, but high-confidence production recipes still need to pin image versions, model format, attention backend, KV dtype, and concurrency settings instead of assuming any vLLM tag will behave the same way.

The Gemma 4 Problem

Gemma 4 is not just a new model. It is architecturally unusual in ways that break assumptions in existing tooling:

1. Requires transformers v5+ (nothing else does yet)

Gemma 4 was the first major model to require the transformers v5 major version bump. Older stock vLLM images shipped with v4.x and failed to parse the Gemma 4 config. Current community images may include transformers v5, but pin the version because v4/v5 API differences can still break model loading.

2. Heterogeneous attention head dimensions

Most models have uniform head dimensions across all layers. Gemma 4 has head_dim=256 for sliding-window layers and global_head_dim=512 for full-attention layers. This breaks attention backends that assume a single head dimension. vLLM forces the TRITON_ATTN backend specifically for Gemma 4 to handle this — other backends (FlashAttention, FlashInfer attention) produce numerical divergence or crash.

3. Hybrid sliding-window + full-attention layers

Of the 30 layers, 25 use a sliding window of 1024 tokens and 5 use full global attention; all 30 carry MoE blocks (128 experts, top-8 — checkpoint-verified: router + expert tensors in every layer). The two attention types have different compute patterns and KV cache requirements — interleaved through the stack.

4. Massive MoE expert count

128 experts per layer with top-8 routing across all 30 layers. That's 128 x 30 = 3,840 expert weight blocks, each with 4 NVFP4 tensors (weight_packed, weight_scale, weight_global_scale, input_global_scale). The total tensor count in this model is 47,648. Loading and routing these correctly requires FusedMoE kernels that can handle the stacked expert format, and the compressed-tensors naming convention doesn't match what vLLM expects (see below).

The NVFP4 Quantization Problem

NVFP4 (NVIDIA 4-bit floating point — E2M1 elements with block scales) is how we get a 26B-parameter model into 15.3 GB. But there are two completely different NVFP4 formats in the ecosystem, and they are not compatible:

ModelOpt NVFP4 (NVIDIA's TensorRT-LLM toolchain): Stores weights as weight, weight_scale_inverse, input_scale. This is what NVIDIA's own tools produce and what most vLLM NVFP4 code paths expect.

Compressed-tensors NVFP4 (llmcompressor/vLLM community): Stores weights as weight_packed, weight_scale, weight_global_scale, input_global_scale. Different tensor names, different scale conventions, different packing format.

This model uses compressed-tensors format (quantized with llmcompressor on an H200). vLLM's Gemma 4 weight loader has hard-coded assumptions about tensor naming that don't match. Specifically:

  • Expert path mismatch: Compressed-tensors names MoE experts as layers.X.experts.{id}.{proj}.weight_packed. vLLM's FusedMoE expects layers.X.moe.experts.{id}.{proj}.weight_packed — note the .moe. segment. Without patching, every single expert tensor fails to load with a KeyError.
  • Suffix format mismatch: The weight loader constructs names like w2_weight.weight_packed when it should be w2_weight_packed. The _weight. needs to be collapsed to _.
  • Dimension assertion failure: The original code asserts dim == 2 for weight tensors, but NVFP4 packed tensors have different dimensionality due to the 4-bit packing.

The included gemma4_patched.py fixes all three issues with targeted patches to the weight loading pipeline.

The Accidental Quantization Problem

When quantizing with llmcompressor, you specify ignore patterns for layers that should stay in BF16 (full precision). The original quantization used patterns like re:.*visual.* and re:.*gate.* to skip vision and routing layers. But Gemma 4's naming conventions didn't match:

LayerExpected PatternActual Name in Gemma 4Result
Vision towerre:.*visual.*model.vision_tower.*Quantized (wrong)
Vision embeddingre:.*visual.*model.embed_vision.*Quantized (wrong)
MoE routersre:.*gate.*model.*.router.proj.*Quantized (wrong)

Quantizing these layers breaks the model:

  • Vision tower in NVFP4 crashes because vLLM allocates standard Linear layers (expects .weight tensor, gets weight_packed/weight_scale/etc.)
  • MoE routers in NVFP4 corrupts expert routing — the router decides which experts to activate for each token, and 4-bit precision on routing logits causes degenerate expert selection
  • Vision embedding projection bridges the ViT output to the language model — quantization here cascades errors through every subsequent layer

We fixed this by extracting the original BF16 weights from the base model (TrevorJS/gemma-4-26B-A4B-it-uncensored) and replacing the incorrectly quantized tensors in the safetensors file:

  • 760 NVFP4 tensors removed from the vision tower, replaced with 190 original BF16 weights (355 total vision tensors including biases and layernorms)
  • 120 NVFP4 tensors removed from router.proj layers, replaced with 30 BF16 weights
  • 4 NVFP4 tensors removed from embed_vision, replaced with 1 BF16 weight

The Token Leakage Problem

Gemma 4 uses internal control tokens for multi-channel generation (thinking, tool calls, output). These tokens have specific IDs in the vocabulary:

Token IDTokenPurpose
100<|channel>Start internal channel (e.g., thinking)
101<channel|>End internal channel
98<|think|>Enter thinking mode
48<|tool_call>Start tool call
49<tool_call|>End tool call

Without proper EOS configuration, the model can enter its "thinking" channel mid-generation, and those internal tokens stream through as plaintext in the API response. Worse, it can get stuck in a repetition loop — endlessly generating <|channel>thought<channel|>call:process{...} as visible text. This manifests as the model appearing to "spam" garbage in the chat.

The fix is adding tokens 98, 100, and 101 to the eos_token_id list in generation_config.json, so vLLM terminates generation cleanly before any internal channel tokens leak into the output.

What's In The Container (The Special Sauce)

The current image is ghcr.io/aeon-7/aeon-vllm-ultimate:latest (= :2026-07-01-v0.24.0). Users pull one image; no local patching or source build is required.

ComponentWhat It IsWhy It Matters
vLLM 0.24.0 (sm_121a from-source) + PR #44389v0.24.0 built for GB10 + Triton NVFP4 KV cacheUp to 3× KV capacity opt-in (--kv-cache-dtype nvfp4; causal speculators only).
PR #40898 + #41703 overlayDFlash drafter fixes merged ahead of upstreamRejected-slot KV-write masking (no more acceptance collapse under prefix caching), drafter sliding-window support, Gemma-4 embed normalizer + logit softcap, flash_attn drafter on multimodal Gemma targets.
AEON sm_121a patches3 idempotent runtime patchesGB10 correctness (optional-import lazy binding, hybrid block-size None guards, CUDA-graph capture-size alignment for spec decode).
PyTorch 2.11.0 + CUDA 13Framework + runtimeSM 12.1 (GB10) support.
transformers 5.10 devModel config/tokenizer loadingRecognizes gemma4_unified and other bleeding-edge classes.
DFlash drafterz-lab/gemma-4-26B-A4B-it-DFlash (mount separately)5-layer SWA-2048 block-diffusion drafter — now actually run with sliding windows.
Native FP4 CUTLASS kernelsFlashInfer CUTLASS for linear layers, VLLM CUTLASS for MoEDo not force Marlin on this image; the native FP4 path is faster on GB10.
TRITON_ATTN body / FLASH_ATTN drafterAttention computationTriton handles Gemma 4's heterogeneous head dims (256/512); the drafter's non-causal attention runs on FlashAttention (required on this image).
torch.compile + CUDA graphsGraph capture and kernel fusionCaptures decode graphs for the configured batch sizes, reducing Python overhead on the decode hot path.

Why MoE Makes This Possible

The fundamental constraint on DGX Spark is memory bandwidth: 273 GB/s. During autoregressive decode, the GPU must read the model weights for every single token generated. This is what determines tok/s:

tok/s = memory_bandwidth / bytes_read_per_token

For a dense 27B model at NVFP4 (~13.5 GB weights):

273 GB/s / 13.5 GB = ~20 tok/s (theoretical max, before KV cache and overhead)

For this MoE model (top-8 of 128 experts, ~2.8 GB active per token):

273 GB/s / 2.8 GB = ~97 tok/s (theoretical max)

We achieve ~37–158 tok/s single-stream depending on category — coding sustains 149–150 tok/s measured on the fixed image; prose ~37 and extraction/JSON ~158 from the pre-fix short-context sweep (a floor; fixed-image re-sweep pending) — with aggregate throughput passing 1,700 tok/s on coding (pre-fix sweep) and ~3,700 tok/s on extraction via the stock no-drafter path. The gap from the theoretical limit comes from KV cache reads, attention computation, router overhead, drafter verification, and memory access patterns. But the key insight is that MoE turns a bandwidth-impossible problem (dense 27B) into a bandwidth-comfortable one.

Model TypeParams Read/TokenMax tok/s on GB10Practical tok/s
Dense 27B BF16~54 GB5Not viable
Dense 27B NVFP4~13.5 GB20~15
MoE 26B top-8/128 NVFP4 + DFlash~2.8 GB + drafter97149–150 coding (fixed image); ~54 natural chat / ~158 extraction (pre-fix sweep); 1.7K+ aggregate

This is why architecture choice matters more than raw parameter count on bandwidth-limited hardware. A 26B MoE model at NVFP4 is faster than a dense 7B at BF16 on the same hardware.

What we fixed for the DGX Spark

All AEON models run on one unified container — ghcr.io/aeon-7/aeon-vllm-ultimate:latest (= :2026-07-01-v0.24.0; rollback :2026-06-18-v0.23.0-dflashfix) — vLLM v0.24.0 built from source for GB10 / sm_121a and merged with the AEON speculative-decoding stack.

FixWhat it doesWhy it matters on GB10
DFlash high-concurrency fix (new)Slices the speculative drafter's KV block-table to the unpadded batch (block_table[:num_reqs])The drafter previously crashed at ≥32 concurrent requests (padded-vs-unpadded block-table shape mismatch in FlashAttention). Now scales cleanly to c=64. A port of upstream PR #43982 (fixed for MTP, never applied to DFlash) — present and unfixed even in the prior image.
Triton NVFP4 KV cache (PR #44389)Software NVFP4 KV-cache pathThe only 4-bit KV path on sm_121a (upstream's is hard-gated to B200) → ~3× KV capacity / longer context per GB of unified memory.
DFlash sliding-window attention (PR #40898)Runs the drafter's SWA layers as true sliding-windowLong-context draft acceptance holds as agent histories grow (≈47% at ~16k, ~41% at ~32k for this model) instead of collapsing past ~2k tokens.
sm_121a-native buildTORCH_CUDA_ARCH_LIST=12.1a, ENABLE_NVFP4_SM100=0Compiles the SM120-family CUTLASS NVFP4/FP8 kernels GB10 actually dispatches to — true 4-bit tensor-core throughput, no dead B200-only kernels.
sm_121a boot + CUDA-graph patchesRTLD-lazy _C_stable_libtorch load; spec-decode CUDA-graph capture-size alignmentBoots past MXFP4 (SM100-only) symbols absent on GB10; prevents cudaErrorIllegalAddress on partial-acceptance decode steps under speculative decoding.
Unified-memory tuning--gpu-memory-utilization 0.6-0.7, FULL CUDA graphs, async scheduling, z-lab DFlash drafterGB10 shares one LPDDR5X pool across CPU + GPU; keep gpu-util at 0.6-0.7 (go lower with co-located services / high concurrency / fp16 KV / DFlash) — above ~0.8 the shared pool page-thrashes and stalls the box, and even 0.85 stalls — while keeping FULL-graph + speculative-decode throughput.

The result for this model:

  • Scales to 64 concurrent requests with no crash (the prior image crashed at c≥32 under speculative decoding) — up to 1,937 tok/s aggregate at c=64 on coding.
  • Native NVFP4 4-bit compute on Blackwell tensor cores — the speed of 4-bit with near-16-bit accuracy.
  • Speculative decoding (DFlash) holds high draft acceptance from short prompts (58.9% coding / 77.5% extraction at c=1) through long agent histories (~47% at 16k, ~41% at 32k).
  • Up to 4.3× faster single-stream decode (≈2.5× average across categories) vs. the stock un-optimized vanilla vLLM baseline.

About the stock baseline: the "stock / un-optimized" comparison numbers are from stock vanilla vLLM (vllm/vllm-openai 0.20.1, default settings, no speculative decoding, no DGX-Spark / sm_121a optimizations). They are provisional and will be refreshed once a fresh fully-vanilla benchmark completes on the current version. The optimized figures are measured on the new aeon-vllm-ultimate:latest (vLLM 0.24.0) build.

Container Image Details

AEON vLLM Ultimate (recommended)

ghcr.io/aeon-7/aeon-vllm-ultimate:latest (= :2026-07-01-v0.24.0) — repo + full card

ComponentVersion
vLLM0.24.0 sm_121a from-source + PR #44389 NVFP4-KV + PR #40898/#41703 DFlash fixes + DFlash high-concurrency fix (PR #43982 ported to DFlash)
PyTorch2.11.0+cu130
transformers5.10.0.dev (HEAD)
flashinfer0.6.12
DFlash draftermount z-lab/gemma-4-26B-A4B-it-DFlash, drafter backend flash_attn
Target GPUNVIDIA GB10 (DGX Spark, sm_121a); also RTX 50-series (sm_120)

Pinned tags: :2026-07-01-v0.24.0 (current :latest), :2026-06-18-v0.23.0-dflashfix (rollback). The older :2026-06-11-pr41703 (vLLM 0.22.1-lineage, DFlash crashes at c≥32), :2026-06-04-pr44389 (pre-fix, flex drafter only, prefix caching NOT safe with DFlash) and the aeon-gemma-4-26b-a4b-dflash:v2 (vLLM 0.20.1) lineage are superseded and kept only for historical comparison.

Stock Community Baseline Image

vllm/vllm-openai:latest@sha256:9eff9734a30b6713a8566217d36f8277630fd2d31cec7f0a0292835901a23aa4

ComponentVersion
vLLM0.20.1
PyTorch2.11.0+cu130
transformers5.7.0
Speculative decodingNone

This image is useful as a current upstream reference point. It is not the AEON DFlash package and does not include the Gemma 4 DFlash drafter path.

All Fixes Included

This model required several post-quantization fixes to work correctly with vLLM. All fixes are baked into the HuggingFace release — no additional debugging needed:

  • De-quantized 760 vision tower tensors (27 ViT layers), 120 router tensors (30 MoE layers), and 4 embedding projection tensors — all restored from original BF16 weights
  • Patched vLLM weight loader for compressed-tensors NVFP4 MoE format (gemma4_patched.py — 3 targeted patches to _weight_iterator and load_weights)
  • Added audio_config and num_experts_per_tok to config.json (vLLM config parser requirements)
  • Created preprocessor_config.json and processor_config.json for multimodal support
  • Configured EOS token IDs [1, 106, 50, 98, 100, 101] to prevent thinking/channel token leakage

Full technical details: HuggingFace Model Card

Related Models

ModelTypeSizetok/s (DGX Spark)Links
This model (Gemma 4 26B MoE + DFlash)MoE NVFP415.3 GB149–150 c=1 coding sustained (fixed image) / 1,724 aggregate coding (pre-fix sweep; re-sweep pending) / ~3,700 aggregate extraction (stock, no-drafter)HuggingFace
Gemma 4 31B DECKARD AWQ_FULLDense NVFP420.5 GB~12-14HuggingFace | GitHub
Gemma 4 31B DECKARD SVDQuantDense NVFP420.9 GB~10-13HuggingFace
Qwen3.5-27B UncensoredDense NVFP4~15 GB~15-18HuggingFace

MoE vs Dense: The MoE model is 3-4x faster than dense models because it only reads ~4B parameters per token (top-8 of 128 experts) vs 27-31B for dense models. Choose MoE for speed and concurrency, dense for maximum quality.

Disclaimer, Liability Waiver, and Assumption of Risk

THIS IS AN UNCENSORED MODEL. By downloading, accessing, or using this model, the associated container image (ghcr.io/aeon-7/aeon-vllm-ultimate), or any derivative works thereof, you expressly acknowledge and agree to the following:

Assumption of Risk

Uncensored language models present materially elevated risks compared to safety-aligned models, including but not limited to: generation of harmful, misleading, illegal, or objectionable content; susceptibility to adversarial misuse; potential for facilitating activities that violate applicable laws or regulations; and amplified risk in automated or agentic pipelines where outputs may be executed without human review.

These tools are powerful and serve a multitude of legitimate and essential purposes — including security research, red-teaming, content analysis, creative work, and applications where safety filters interfere with valid use cases. However, the absence of safety guardrails demands a correspondingly higher standard of care from the operator. You must implement your own safeguards, content filtering, access controls, and monitoring appropriate to your use case and jurisdiction.

Limitation of Liability

The authors, contributors, and distributors of this model and container image ("Providers") are not responsible or liable, directly or indirectly, for any actions taken, content generated, damages incurred, or legal consequences arising from the use or misuse of these materials. This includes, without limitation:

  • Any harmful, illegal, unethical, or objectionable outputs produced by the model
  • Any decisions made or actions taken based on model outputs
  • Any damages — direct, indirect, incidental, consequential, special, or exemplary — arising from the use of the model or container, regardless of whether the Providers were advised of the possibility of such damages
  • Any violation of local, state, national, or international laws or regulations by the user

User Responsibility

You, the user, assume full and sole responsibility and liability for:

  • All outputs generated by the model under your operation
  • Ensuring your use complies with all applicable laws, regulations, and ethical standards in your jurisdiction
  • Implementing appropriate access controls, content filtering, and human oversight
  • Any consequences of deploying this model in production, automated, or public-facing systems
  • Evaluating whether an uncensored model is appropriate for your specific use case

Acceptance

By downloading or using any component of this release — including the model weights, container image, configuration files, or patched code — you indicate your acceptance of these terms and your assumption of all associated risks and liabilities. If you do not agree to these terms, do not download or use these materials.

License

This model inherits the Gemma license from Google.

Support the work

If this release has been useful, tips are deeply appreciated. They go directly toward more compute, more models, and more open releases.

Ethereum L2s (Base, Arbitrum, Optimism, Polygon, etc.) and EVM-compatible tokens can be sent to the same Ethereum address.

HUGGING FACE

上游数据

HF 下载量3.8万
HF 点赞量24

HF 下载量为源站最近 30 天统计，与本站点击分开。

SITE STATS

本站统计

下载点击—
请求数—
匿名去重—最近 7 天—

本站下载量记录按钮点击，不代表下载完成；短时连点会去重。

来源与版本

作者
AEON-7
最近更新
2026/07/15
最新创建
2026/04/09
HF 趋势
未知
数据快照
2026/09/16
版本
未知
许可证
gemma