Skip to content

Bump gpu-kernels: abaa6bc484d5 (batched MoE decode on our kernels) - #521

Merged
bong-water-water-bong merged 1 commit into
mainfrom
bump-gpu-kernels/abaa6bc484d5
Oct 23, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
bump-gpu-kernels/abaa6bc484d5

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Moves config/gpu-kernels.config from 4f586caca414 to abaa6bc484d5.

gpu-kernels #25: llama-server --parallel N batches N sequences into one decode step; the 2..8-token MoE expert steps now run on our HIP MoE decode kernels instead of prompt-shaped WMMA tiles (single-token decode and prompts untouched; GGML_HRX_DISABLE_DISPATCH=hip.moe_tokens turns the pair off).

Evidence (strixhalo, 90 W cap, PCM pad): Qwen3-Coder-30B-A3B aggregate decode N=2 76.5 -> 95.0-95.6 tok/s (+24%), N=4 100.2 -> 119.6-121.7 (+20%), N=8 flat; dense models unchanged; N=1 bit-identical (KLD RMS 0.001%, identical greedy text on 3 prompts); test-backend-ops MUL_MAT_ID 132/132 vs CPU; hw-gate ops 695/695, kat 9/9, placement all on HRX.

Merge by hand after hw_gate passes (auto-merge off).

🤖 Generated with Claude Code

…ur kernels

gpu-kernels #25 (pi bw220): llama-server --parallel N batches N sequences into one decode step; the MoE expert steps with 2..8 tokens now dispatch our HIP MoE decode kernels instead of the prompt-shaped WMMA tiles. Qwen3-Coder-30B-A3B aggregate decode (90 W cap, PCM pad): N=2 76.5 -> 95.0-95.6 (+24%), N=4 100.2 -> 119.6-121.7 (+20%), N=8 flat, N=1 untouched (bit-identical single-stream: KLD RMS 0.001%, identical greedy text, test-backend-ops MUL_MAT_ID 132/132, hw-gate ops/kat/placement PASS). Dense models unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@bong-water-water-bong bong-water-water-bong added the hw-gate Run the HRX hardware gate on Strix Halo (docs/hw-gate.md) label Oct 23, 2026
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

25 - Partially compliant

Compliant requirements:

  • Merge upstream lemonade-sdk/lemonade main into 1bit-MONSTER/lemonade fork
  • Disable all workflows in the fork to prevent upstream workflows from running

Non-compliant requirements:

  • Clean merge: if 'onebit' is still registered, push to fork's main
  • Conflicts: create sync/ branch and PR in fork listing conflicting files

Requires further human verification:

  • Verification that 'onebit' is still registered after the merge
  • Confirmation that the sync branch and PR were created in case of conflicts
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Kernel SHA Update

The PR updates the SHA for the gpu-kernels dependency from 4f586caca414 to abaa6bc484d5. While this is a straightforward version bump, it's important to verify that this new commit is compatible with the current build and runtime environment, especially given the hardware-specific nature of the kernels (e.g., HRX, AMD Strix Halo). The PR description mentions performance improvements, but without concrete measurements or logs, it's hard to confirm that the update doesn't introduce regressions or compatibility issues.

sha=abaa6bc484d5884a769226ca74790800f6999aea

@bong-water-water-bong
bong-water-water-bong merged commit d7820ab into main Oct 23, 2026
9 of 10 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the bump-gpu-kernels/abaa6bc484d5 branch October 23, 2026 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hw-gate Run the HRX hardware gate on Strix Halo (docs/hw-gate.md) Review effort 2/5

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant