Skip to content

fix: make the Gemma3n xla-diagnostics probe pass on CUDA - #2810

Merged
inureyes merged 1 commit into
mainfrom
fix/issue-2808-gemma3n-probe-cuda
Nov 1, 2026
Merged

inureyes merged 1 commit into
mainfrom
fix/issue-2808-gemma3n-probe-cuda

Conversation

@inureyes

@inureyes inureyes commented Nov 1, 2026

Copy link
Copy Markdown
Member

Summary

  • pinned_e2b_real_projection_qmv_probe aborted on CUDA before any sparse stage could run on a real checkpoint (issue fix: make the Gemma3n xla-diagnostics probe pass on CUDA past the weight-carrier and AltUp stages #2808, both failures recorded in the PR fix: align Gemma3n gelu_topk bf16 rounding across eager, XLA and mlx-lm #2801 body). The weight-carrier check pinned MLX CPU dequant semantics while the probe executes MLX on GPU: every 4-bit weight failed with about 51% 1-ulp mismatches, and the probe never reached the sparse stages. The AltUp chain, once reachable, drifted by f32 reduction noise that a strict bit comparison cannot tell from a formula error.
  • The weight carrier is now built with the rounding the default device applies: dequantize_affine_bf16_single_round matches MLX's GPU mx.dequantize (q * scale + bias in f32, one round to BF16; verified bit-exact for the pinned E2B checkpoint, 4194304/4194304, and synthetic wide-magnitude weights), while the CPU path keeps dequantize_affine_bf16_sequential. The strict comparison itself is unchanged and still rejects any wrong scale, bias, group layout, or nibble order, which move carriers far beyond one ulp.
  • The AltUp coefficient chain now compares under the feat(xla): define operator-level MLX/IREE numeric contracts #932 per-op rule with the measured noise recorded: coefficients at max_abs 1e-4 (measured 1.514e-5 prediction / 5.603e-6 correction), and the bf16 stages that consume them at max_abs 3.2e-3 plus two bf16 ulps at the operand's binade, the measured shapes of the compiled TF32 reduction noise (absolute 1.5e-4 on near-zero values, three ulps mid-binade, one ulp at [2, 8)).

Test plan

  • pinned_e2b_real_projection_qmv_probe (--features xla-diagnostics, IREE CUDA tree ~/.cache/mlxcel/iree-cuda-3.12.0rc20260721, GEMMA3N_MODEL_DIR=models/gemma3n-e2b-4bit, GB10 under gpu-lock, --test-threads=1): runs to completion, test result: ok. 1 passed in 22.45s
  • Every layer0_sparse_* stage and layer0_sparse_activation_manual_vs_compiled still reports bit_mismatches=0 (the fix: align Gemma3n gelu_topk bf16 rounding across eager, XLA and mlx-lm #2801 result is not weakened)
  • The AltUp stages pass under the feat(xla): define operator-level MLX/IREE numeric contracts #932-based threshold with the measured f32 noise recorded in the code comments and printed by the probe
  • Negative control for a deliberately wrong dequant: not added; this PR states the bounds instead. The strict comparison is unchanged and still rejects any wrong scale, bias, group layout, or nibble order, which move carriers far beyond one ulp; the device-consistent carrier only changes which MLX dequant rounding the reference matches.

Closes #2808

@inureyes inureyes added the status:review Under review label Nov 1, 2026
@inureyes
inureyes marked this pull request as ready for review November 1, 2026 05:51
…e stage could run on a real checkpoint (issue #2808, both failures recorded in the PR #2801 body). The weight-carrier check pinned MLX CPU dequant semantics while the probe executes MLX on GPU, so every 4-bit weight failed with about 51% 1-ulp mismatches and the probe never reached the stages; the AltUp chain, once reachable, drifted by f32 reduction noise that a strict bit comparison cannot tell from a formula error.

The weight carrier is now built with the rounding the default device applies: dequantize_affine_bf16_single_round matches MLX's GPU mx.dequantize (q * scale + bias in f32, one round to BF16; verified bit-exact for the pinned E2B checkpoint, 4194304/4194304, and synthetic wide-magnitude weights), while the CPU path keeps dequantize_affine_bf16_sequential. The strict comparison itself is unchanged and still rejects any wrong scale, bias, group layout, or nibble order, which move carriers far beyond one ulp.

The AltUp coefficient chain now compares under the #932 per-op rule with the measured noise recorded: coefficients at max_abs 1e-4 (measured 1.514e-5 prediction / 5.603e-6 correction), and the bf16 stages that consume them at max_abs 3.2e-3 plus two bf16 ulps at the operand's binade, the measured shapes of the compiled TF32 reduction noise (absolute 1.5e-4 on near-zero values, three ulps mid-binade, one ulp at [2, 8)). The layer0_sparse_* stages and manual_vs_compiled keep their bit-exact comparison and reported 0 mismatches in the final probe run.

Closes #2808
@inureyes
inureyes force-pushed the fix/issue-2808-gemma3n-probe-cuda branch from 5bb90d5 to 5849bcf Compare November 1, 2026 06:30
@inureyes
inureyes merged commit 99e3baf into main Nov 1, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status:review Under review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: make the Gemma3n xla-diagnostics probe pass on CUDA past the weight-carrier and AltUp stages

1 participant