Skip to content

qsa: the default block-scores kernel without #2234's per-query keys (a gfx1151 verify window stalled on v0.1.45); the batch rows get their own kernel - #3151

Closed
gt50 wants to merge 1 commit into
Niko1221:mainfrom
gt50:fix-qsa-scores-default
Closed

gt50 wants to merge 1 commit into
Niko1221:mainfrom
gt50:fix-qsa-scores-default

Conversation

@gt50

@gt50 gt50 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

On v0.1.45 a Radeon 8060S (gfx1151, Windows 11, ROCm 10.2.0a20260930) stalls in its first verify window when gfx1151's default switches are on. The engine prints verify: an earlier window never finished on the GPU (#267) and "the GPU rang 0", the watchdog restarts it after 60 s, and no token comes out. It happens with a stock config and with mine. The installed 0.1.40.2 engine, with the same switches and config, does not stall.

Cause: f629fb0 (#2234), its qsa_select.cu change. block_scores_multi_kernel is the kernel every decode window's QSA block scores run on. #2234 gave it a by-value QsaScoreRows argument (16 pointers and a count) and a per-query branch for the opt-in STRATA_BATCH_ATTN_ROWS path. Both apply with that switch off.

Fix: block_scores_multi_body is a template on OWN.

How it was found

All on that Radeon 8060S, Qwen3.8-Flash-Next UD-IQ4_XS, --spec 4 --mtp, --kv int8. Each trial was a fresh start, then a 14-token request with max_tokens 8. No answer in 30 s counted as a stall; a normal answer takes 0.4-0.7 s.

  • The switches: 26 trials with STRATA_GFX1151_DEFAULTS=0 and subsets turned on. STRATA_PLE_BATCH is on in every set that stalled. It needs STRATA_LFUSE and STRATA_TSUM or STRATA_GDN_SPLIT beside it, and other switches change which set is enough. With the other 18 defaults on, STRATA_PLE_BATCH=0 stops the stall.
  • Not every start stalls: one build stalled in 6 of 7 starts (c8265fc, the commit just before v0.1.45 on main). So each step below used 3 starts.
  • Commits: a bisect from v0.1.40.2 (good) to v0.1.45 found the first bad main-line merge, e8b1edb "perf-PR group batch". Its parent 5a6f4ed answered 3 of 3. Inside the merge, the commits from f629fb0 up to f755a06 do not build on HIP (qsa_decode_attn.cu; f755a06 fixes that). So each part was applied alone to the merge base 296b59d:
On 296b59d Starts
a2de744 (PleTable lock) 3 of 3 answer
f629fb0 + 739bc94 + f755a06 stall
only the qsa_decode_attn files (f755a06's refactor) 3 of 3 answer
only f629fb0's GDN change (verify_kernels) 3 of 3 answer
only f629fb0's qsa_select.cu / .hpp change stall

The f-145-fix3 merge with #2622 answered 3 of 3, so the PLE gather change is not the cause. Several 0.1.45 merges do not build on Windows HIP without e6ad6c8's 4-line hip_compat fix, which these tests applied.

Checked with this change

On v0.1.45 (477ac13) plus this commit, on the same machine:

  • The stall trial: 5 of 5 starts answer with all 19 gfx1151 defaults on.
  • The parity tests: batch_rows_parity passes (it checks qsa_block_scores_rows at 1, 2, 4 and 8 rows against one call per row, bitwise). So do qsa_parity, qsa_topk_parity, hip_native_qsa_score and verify_batch_parity. qsa_scores_gfx1100_parity builds and skips itself, since it is gfx1100 only.
  • Answers: three greedy answers (284-344 tokens) are byte-identical to v0.1.45 with STRATA_PLE_BATCH=0.
  • Speed: decode 34.9 / 48.9 / 40.3 tok/s, against 34.9 / 48.5 / 40.8 with the workaround.

Not covered

  • Other backends: not built for CUDA or SYCL. The code uses if constexpr, as other .cu files here already do.
  • The mechanism: why the argument and branch hang the window only beside those switches is not explained here. The tests show this change caused it, and that taking it out of the default kernel stops it.
  • Multi-row speed: the per-row path's speed with --batch and STRATA_BATCH_ATTN_ROWS=1 was not measured. Only its parity test ran.
  • Unrelated test build: a full tools\hip\build_windows.bat tests stops at hc_persist_parity.cpp. It uses POSIX setenv/unsetenv, which this change does not touch.

Found while testing #1916 on v0.1.45. cc @yi-sudo (#2234)

🤖 Generated with Claude Code

…y keys; the batch rows get their own kernel

f629fb0 (Niko1221#2234) gave block_scores_multi_kernel, the kernel every decode window's
QSA block scores run on, a by-value QsaScoreRows argument (16 pointers and a
count) and a per-query branch for the opt-in STRATA_BATCH_ATTN_ROWS path. With
them, a Radeon 8060S (gfx1151, Windows, ROCm 10.2.0a20260930) stalls in its first
verify window ("an earlier window never finished on the GPU (Niko1221#267)", the GPU rang
0) when gfx1151's default switches are on: it needs STRATA_PLE_BATCH, STRATA_LFUSE
and STRATA_TSUM or STRATA_GDN_SPLIT beside it. It does not stall every time: 6 of
7 starts of one build stalled (c8265fc, the commit just before v0.1.45 on
main), and every start of v0.1.45 with all the defaults on.

block_scores_multi_body is a template on OWN: the default kernel (OWN false) is
again the pre-Niko1221#2234 kernel, with no QsaScoreRows argument and no branch, and
qsa_block_scores_rows launches block_scores_multi_rows_kernel (OWN true).

Checked on that Radeon 8060S, UD-IQ4_XS, the 19 gfx1151 defaults on:
- bisected to f629fb0's qsa_select.cu change alone (3 starts each step);
- with this change: 5 of 5 starts answer;
- batch_rows_parity (qsa_block_scores_rows at 1/2/4/8 rows against one call per
  row, bitwise), qsa_parity, qsa_topk_parity, hip_native_qsa_score and
  verify_batch_parity pass;
- three greedy answers (284-344 tokens) byte-identical to v0.1.45 with
  STRATA_PLE_BATCH=0, decode 34.9 / 48.9 / 40.3 tok/s against 34.9 / 48.5 / 40.8.
Not built for CUDA or SYCL here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Niko1221

Copy link
Copy Markdown
Owner

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

  1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
  2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
  3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
  4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.

@Niko1221 Niko1221 closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants