[bong-water-water-bong]

Complements #101 (the non-FA path fix). This PR makes HRX0 claim flash attention for the two MoE shapes that previously declined it, so the FA path is used, and fixes a batched MoE-routing bug that made GLM's FA output garbled.

Problem

resolve_fused_ops logged "Flash Attention not supported, set to disabled" for two MoE shapes:

Model	GGUF arch	Why FA was declined	Before	After
GLM-4.7-Flash Q4_K_M	deepseek2/MLA	key_length 576 > 512 head-size cap	FA off	FA on
Qwen3-Coder-30B-A3B Q4_K_M	qwen3moe	262144 ctx > 32768 KV cap	FA off	FA on
Qwen3.6-35B-A3B Q8_0	qwen35moe	(already claimed)	pass	pass

Fix (in the llama.cpp submodule, 1bit-MONSTER/llama.cpp@ff07f1c71)

1. MLA head size — raise kPrefillMaxHeadSize 512 → 576 (GLM/DeepSeek key_length), and widen the FA prefill/decode kernels' qk_head_size config range to 576.
2. MLA value stride — the MLA KV cache pads the value row to key_length, so the value's row stride is 576, not value_head_size (512). Match the value against the QK head-size stride and pass the value stride to the kernel (ggml.flash_attention.value_stride).
3. Full-context KV — raise the prefill/decode KV caps 32768/2048 → 262144 (and the kernels' KV loop bounds) so Qwen3-Coder's 262144 context is claimed at load.
4. Strided MoE route ids — the route ids are a [n_expert_used, n_tokens] view of the [n_expert, n_tokens] argsort, so the stride between a token's route ids is n_expert, not n_expert_used. The mul_mat_id routing bundle assumed route_stride == route_count and read the wrong experts for batched prompts (GLM answered garbled at the default ubatch). Read route_stride from route_ids->nb[1] and size the span as (token_count - 1) * route_stride + route_count.

Verification

• tests/serve_e2e.sh build/1bit <model> hrx answers "Paris" (and streams) on all three: Qwen3-Coder-30B-A3B-Instruct Q4_K_M, GLM-4.7-Flash Q4_K_M, and Qwen3.6-35B-A3B Q8_0 (no regression).
• A ~2048-token prompt on Qwen3-Coder at full 262144 context returns HTTP 200 via FA in 1.4 s.
• Both GLM-4.7-Flash and Qwen3-Coder-30B-A3B load with no "Flash Attention not supported" (FA claimed).
• Retained artifacts: ~/evidence/hr-95-fa/*.log.

[context7]

Docs7 for 1bit-monster/engine

Result	Status	Action
Deployment	➖ Not used	—
Content review	➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan.	—

Commit f55b923

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit f55b923)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 101 - Partially compliant Compliant requirements: Fix the 500 "Compute error" for MoE models on HRX0 when context exceeds flash-attention KV cap Resolve the three bugs in the HRX backend's non-FA path Ensure tests/serve_e2e.sh build/1bit <model> hrx returns HTTP 200 for affected models Verify that a ~2048-token prompt returns HTTP 200 for Qwen3-Coder-30B-A3B and GLM-4.7-Flash Confirm no regression on Qwen3.6-35B-A3B Non-compliant requirements: The GLM-4.7-Flash still fails the "answers Paris" check due to a separate correctness bug in fused batched dispatch (not part of this PR) Requires further human verification: The GLM-4.7-Flash "answers Paris" check

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Subproject commit update The PR updates the llama.cpp submodule commit, which may include changes that affect the HRX backend's flash attention behavior. It's important to ensure that the updated commit aligns with the intended fixes and doesn't introduce regressions or unexpected behavior in the flash attention path for MoE models. Subproject commit e44cab1209c371e90b1e4364dc2dae0045c09135

[github-actions]

Persistent review updated to latest commit cf8511c

[bong-water-water-bong]

Regression found while verifying #95 on top of this branch (fork 19096bf3, merged with d7ff3afb, llama-server with --llama-server, Strix Halo): Qwen3-0.6B-Q4_K_M on hrx now fails with 500 "Compute error.". It passes at c075cc1.

E compile_kernel: ... ggml_flash_attention_f32_f16_wmma ... query_token_count=22 key_value_token_count=256
  qk_head_size=128 value_head_size=128 key_value_head_count=8 ggml.flash_attention.value_stride=1024: Loom root linking failed
  diagnostic[0] CONFIG/INVALID: symbol '@ggml.flash_attention.value_stride' value 1024 violates constraint 'range'

The new value_stride is the whole V row (8 KV heads x 128 = 1024), but the kernel config's range stops at 576. Any GQA model with n_kv_head x head_dim > 576 hits this, and serve_e2e on the MoE models alone does not catch it.

The GLM batched wrong answer mentioned in the Notes is a separate bug: MUL_MAT_ID ignored the route-id stride. The fix is on fork branch fix/hr-95-mmid-route-stride; see #95.

[github-actions]

Persistent review updated to latest commit f55b923

[bong-water-water-bong]

Review from the commit poll: the new pin, e44cab1, is not on any branch of 1bit-MONSTER/llama.cpp.

• Fork PR Server: 1bit-server speaks Lemonade's backend protocol #4 (1bit/hrx-95) was merged into 1bit/hrx-vulkan-patched as 96f6b89. That merge carries d7ff3af, 07a2ee4 and ff07f1c.
• Instead of e44cab1, the merge carries the value-stride fix as d925875 plus 19465d2, the dispatch debug switches.
• The GitHub compare API reports e44cab1 → 96f6b89 as diverged, ahead 3, behind 1.

The submodule is declared branch = 1bit/hrx-vulkan-patched, shallow = true. A branch-shallow clone won't contain e44cab1, and once unreferenced GitHub may garbage-collect it. Pinning 96f6b89 would carry the same #95 fixes on the tracked branch.

Heads-up: 1bit/zaya (fork PR #2, ZAYA1) will be rebased onto 96f6b89. The engine's ZAYA PR will then bump the pin to that rebased head, which includes #95.

[bong-water-water-bong]

Superseded by #104: this PR's llama.cpp commit is in the pin 96f6b89 (1bit-MONSTER/llama.cpp#4), with the MUL_MAT_ID route-stride fix and the FA value_stride range fix. All 7 HRX models pass serve_e2e.