Skip to content

sycl/A770: commit all GDN layers' conv history + state replay in two launches (STRATA_GDN_COMMIT_BATCH; bitwise) - #2755

Closed
aslater3 wants to merge 2 commits into
Niko1221:mainfrom
aslater3:a770/up-gdn-commit-batch
Closed

aslater3 wants to merge 2 commits into
Niko1221:mainfrom
aslater3:a770/up-gdn-commit-batch

Conversation

@aslater3

Copy link
Copy Markdown
Contributor

sycl/A770: commit all GDN layers' conv history + state replay in two launches (STRATA_GDN_COMMIT_BATCH; bitwise)

capture_commit replayed the accepted tokens into each GDN layer's state with two launches a layer (gdn_conv_commit and
the split state replay): 36 layers -> 72 launches at the end of every verify window, on the critical path into the
next round's drafting. gdn_commit_replay_layers runs the same two kernels over all layers at once (the layer is the
outermost work-group index, selecting each layer's state, history and inputs), so every thread computes what it did.
It declines (the per-layer loop runs) where the split replay is not the path. Default on on Alchemist, where it is
measured; STRATA_GDN_COMMIT_BATCH=1 forces it on, =0 off. Needs the split replay (#2586).

gdn_bench GDN_CBATCH=1 (new): 36 layers, n_keep 1..6, every float of every layer's state and history compared per
layer vs batched: 0 differ; 1.96 -> 0.83 ms. Engine, A770, Coder IQ1_M, 8K code prompt, 3 pairs: MTP drafting
7.51 / 7.31 / 7.44 -> 6.58 / 6.44 / 6.50 ms a round (-0.91), decode 35.68 / 39.41 / 39.58 -> 35.58 / 40.04 / 41.63 tok/s,
reference tokens (one known low-hit variant).

Stacked on #2586 (the split GDN replay, which this batches); the diff to review is the top commit.

🤖 Generated with Claude Code

aslater3 and others added 2 commits October 15, 2026 06:40
…ce by default on Alchemist (bitwise)

At 128 registers the step kernel spilled ~7.6 KB per SIMD32 thread on the A770 (unitrace). STRATA_GDN_LARGE_GRF (Alchemist
default) launches the recurrence kernels in large-GRF mode; STRATA_GDN_SPLIT (S25/S26, bitwise) becomes the Alchemist
default. gdn_bench (new: 48 layers' states from VRAM), 4 / 6 tokens: 103 / 114 -> 39.5 / 44.9 us; gdn_parity 0
failures; engine 8K GPU wait 39.5 / 38.8 -> 37.6 / 37.9 (split) and 35.9-36.3 -> 33.8-34.0 ms/round (large GRF).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…launches (STRATA_GDN_COMMIT_BATCH; bitwise)

capture_commit replayed the accepted tokens into each GDN layer's state with two launches a layer (gdn_conv_commit and
the split state replay): 36 layers -> 72 launches at the end of every verify window, on the critical path into the
next round's drafting. gdn_commit_replay_layers runs the same two kernels over all layers at once (the layer is the
outermost work-group index, selecting each layer's state, history and inputs), so every thread computes what it did.
It declines (the per-layer loop runs) where the split replay is not the path. Default on on Alchemist, where it is
measured; STRATA_GDN_COMMIT_BATCH=1 forces it on, =0 off. Needs the split replay (Niko1221#2586).

gdn_bench GDN_CBATCH=1 (new): 36 layers, n_keep 1..6, every float of every layer's state and history compared per
layer vs batched: 0 differ; 1.96 -> 0.83 ms. Engine, A770, Coder IQ1_M, 8K code prompt, 3 pairs: MTP drafting
7.51 / 7.31 / 7.44 -> 6.58 / 6.44 / 6.50 ms a round (-0.91), decode 35.68 / 39.41 / 39.58 -> 35.58 / 40.04 / 41.63 tok/s,
reference tokens (one known low-hit variant).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Niko1221

Copy link
Copy Markdown
Owner

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

  1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
  2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
  3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
  4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants