Skip to content

kernels: process all groups in fused GU and LDS down launches - #1909

Closed
W1nge wants to merge 1 commit into
Niko1221:mainfrom
W1nge:codex/grouped-stride-fix
Closed

W1nge wants to merge 1 commit into
Niko1221:mainfrom
W1nge:codex/grouped-stride-fix

Conversation

@W1nge

@W1nge W1nge commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

Closes #1908.

Summary

When expert groups outnumber launch block rows, mode 8's fused gate/up and modes 7/8's LDS down kernels skip later groups. Stride the group index by gridDim.y in both kernels.

What changed

Add a model-free regression to native_grouped_parity: 11 groups, varied entry counts, scattered outputs, nonzero entry offsets, and block-row counts 1/3/8. Check mode 7's down pass and mode 8's complete pass across five gate/up formats and two down formats. No launch defaults or arithmetic changes.

Validation

MSVC/CUDA 12.4 build for sm60;sm75. On RTX 2080 Ti and P100, all 120 new comparisons per GPU fail before the fix and pass afterward; existing default-path checks remain bitwise equal. Targeted CTest and full CUDA integration build pass.

Draft pending the affected gfx906 HIP build; no AMD runtime result claimed.

Maddoxkkm added a commit to Maddoxkkm/Strata that referenced this pull request Oct 11, 2026
What the release brought for this box (Niko1221#1709 Windows HIP VRAM floor, the HIP stream-capture fix, the
Linux-only PCIe telemetry), the merge table with what was kept and why Niko1221#1439/Niko1221#1841/Niko1221#1656 went, the
measurements (0.1.42+local +5.6% at 16K; the four new PRs +4.4% on top), and the ranked shortlist for
the next cycle (Niko1221#1297 bisect, Niko1221#1909, Niko1221#1324, Niko1221#1167, Niko1221#1298/Niko1221#1151) with the gfx1100 reading list.
Maddoxkkm added a commit to Maddoxkkm/Strata that referenced this pull request Oct 11, 2026
Maddoxkkm added a commit to Maddoxkkm/Strata that referenced this pull request Oct 11, 2026
…10.9%

Six PRs merged individually (Niko1221#1297 Niko1221#1909 Niko1221#1324 Niko1221#1167 Niko1221#1298 Niko1221#1151), none of which changes the default path
(the arms are opt-in and Niko1221#1297 needs a tuning table). Reading the issue tracker produced the real win:
issue Niko1221#1549 documents ring_default_slots() sizing the staging ring at 384 (1024 for a fused ring) when
g_pinned_share >= 0.9, and our STRATA_PREFILL_RING=96 override - left from tuning against a smaller budget
- was costing 16.5% at 4K and 10.9% at 16K. The override is now gone, so the engine's own rule applies.
Measured, with the LRU left off (it buys prefill but eats the resident budget: 11.9 -> 9.1 GiB).
@Niko1221

Copy link
Copy Markdown
Owner

Thanks. This is a correctness fix for the AMD layouts rather than a speed change; our CUDA launches size the grid for every group so they were not affected. We will look at it with the gfx906 changes and keep your authorship.

@Niko1221

Copy link
Copy Markdown
Owner

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

  1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
  2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
  3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
  4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.

@Niko1221 Niko1221 closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: LDS grouped expert kernels skip groups beyond the launch grid

2 participants