license: mit base_model: cantina-security/apex-flash-1-abliterated base_model_relation: quantized library_name: transformers pipeline_tag: image-text-to-text tags:

• glm5_next
• abliterated
• security-research
• nvfp4
• compressed-tensors
• llm-compressor
• quantized

apex-flash-1-abliterated-NVFP4

Preliminary model card. Quality evaluation is pending (see Evaluation).

This is an NVFP4 (W4A4) quantization of cantina-security/apex-flash-1-abliterated, in the compressed-tensors format. The checkpoint is about 205 GB, compared with about 643 GB for the BF16 source.

Base model

apex-flash-1-abliterated is an experimental derivative of apex-flash-1, Cantina Security's open-weights security model developed in partnership with Yeta (@yetalabs on X). apex-flash-1 is a reinforcement learning post-train of GLM-5.3-Flash (zai-org/GLM-5.3-Flash) for focused investigations: reading code, using tools, pursuing an exploit, and verifying its effect in a running target. The abliterated variant modifies refusal behavior broadly; the changes are not limited to security tasks.

Further reading from the base authors: Apex Flash release post, Explore Apex.

All credit for the model itself belongs to Cantina Security and Yeta. This repository only contains a quantized copy produced by a third party; it is not affiliated with or endorsed by them.

Intended use

As stated on the base card: authorized security research in environments the researcher owns or has permission to test.

Architecture

Quantization

NVFP4 is W4A4. Weights are FP4 (E2M1) with group size 16, FP8 (E4M3) block scales and an FP32 per-tensor global scale. Activations use dynamic local FP8 block scales per 16 elements plus a static per-tensor input_global_scale calibrated from data.

What is quantized

Only the routed-expert gate/up/down projections in layers 3-44 are quantized (36,288 projections, the vast majority of parameters). Everything else stays BF16.

Calibration data

512 samples, 501,470 tokens, every sample rendered through the model's own chat template.

Context length used for calibration: at most 2048 tokens per sample (mean ~980). Calibration did not use long contexts. The impact is expected to be limited but is untested: weights are quantized independently of context length; activation block scales are computed dynamically at runtime; only the per-tensor activation global scale is static, and it was calibrated on sequences of 2048 tokens or fewer; attention and the KV path are BF16 and not quantized here. Long-context (>4K) quality has not been evaluated.

Fixes relative to the upstream llm-compressor GLM-5.3-Flash example

1. MTP layer dropped: llm-compressor 0.14.0 only copies MTP tensors for configs with num_mtp_layers/mtp_num_hidden_layers and an mtp. prefix, whereas this model uses num_nextn_predict_layers and layers.45.*. The layer is spliced back from the source (splice_mtp.py).
2. Target regex: .*mlp\.experts\..* also matches layer 45, which would make loaders expect NVFP4 weights there. Targets are restricted to layers 3-44 with an explicit ignore for layer 45.
3. Missing dependencies in the upstream environment: torchvision and flash-linear-attention.

Structural verification

Result of verify_ckpt.py against the BF16 source: VERIFY OK. This confirms checkpoint structure only, not model quality.

Evaluation

Pending. A BF16-vs-NVFP4 parity evaluation is in progress: a held-out set disjoint from the calibration data across the same five domains, plus 4096-token long samples and WikiText-2, measuring top-1 agreement, approximate KL divergence and perplexity. Results will be added to this card. No parity, accuracy-retention or benchmark claims are made at this time. The base model's reported results apply to the original apex-flash-1 BF16 checkpoint only.

Usage (untested on this checkpoint)

The compressed-tensors NVFP4 format is auto-detected by vLLM:

vllm serve Code4me2/apex-flash-1-abliterated-NVFP4 --tensor-parallel-size 2

MTP speculative decoding is available via:

--speculative-config '{"method":"mtp","num_speculative_tokens":1}'

Native NVFP4 kernels require Blackwell (SM100/SM12x). See the SM12x caveat below.

Known gaps and limitations

• Calibration length: limited to 2048 tokens per sample; long-context behavior is unevaluated.
• No multimodal calibration: no image or video data was used. The vision tower is BF16, but expert activations on image tokens were not calibrated. Image/video performance is unevaluated, as in the base.
• English-centric calibration: GLM is bilingual (zh/en); the calibration set is not.
• Plain round-to-nearest quantization with min-max observers: no GPTQ/AWQ-style error compensation and no MSE clipping search.
• All-expert calibration: moe_calibrate_all_experts exposes each expert to tokens it would not normally receive, which can make activation global scales conservative.
• SM120/SM121 serving: stock vLLM 0.30 currently fails for all GLM-5.3-Flash checkpoints on SM120/SM121 (RTX PRO 6000, DGX Spark/GB10) with pe_dim must be 64 for fp8_ds_mla (vllm-project/vllm#55773, #53963). Community patches exist (Libertai/vllm-sparse-mla-blackwell, tonyd2wild/DGX-Spark).
• Inherited limitations: all limitations of the abliterated base apply. Refusal behavior is modified broadly, the variant was not separately evaluated, and image/video performance was not evaluated.

Reproducibility

The quantization recipe is in recipe.yaml. The logs quantize.log and verify.log and the calibration statistics calib_stats.json are included in this repository.