You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Feature]: SYCL: read prompts faster on Battlemage by attending over the union of neighbouring positions' selections #2807
Read long prompts faster on the SYCL port on an Arc Pro B70 (Battlemage) with INT8 KV.
Today the QSA prompt attention on SYCL runs the FP32 decode kernel (qsa_decode_attn_batch), 32 prompt positions per
launch, and every position gathers its own ~2,051 selected K/V cells. The XMX port of the tensor-core kernel
(STRATA_PROMPT_ATTN_XMX=1) is correct but slower than that fallback: it is gather-bound and only 12 of its 16 matrix
rows are real heads (docs/INTEL.md, "XMX prompt attention v2"). docs/INTEL.md also measured that 8 consecutive
positions of an 80K prompt select 3.3x one position's cells between them, and left a kernel over that union "not
built" because it multiplies the arithmetic.
What I would like: an opt-in kernel that takes 8 neighbouring positions together, gathers the union of their selected
cells once, runs the products for all 8 x 12 heads on XMX (every matrix row a real head, the extra arithmetic on the
matrix units), and masks per position so each one still attends to exactly its own cells. The default path stays as
it is.
Workaround today: none; the default FP32 attention is the fastest prompt attention on this card.
I have a working version. On upstream main with 2x Arc Pro B70 (layer split, Qwen3.8-Flash-Next ISTA GSQ-RCO IQ3_XXS,
INT8 KV) it reads 4K and 32K prompts 15-16% faster (1,189 -> 1,382 and 1,956 -> 2,246 tok/s) and cuts the first token
of a turn with 40,796 tokens of context from 23.28 to 20.39 s; decode is unchanged. The output is not bitwise (FP16
inputs to the products, about 1e-3 of the output scale against FP64). Details and the parity test in the pull request.
Prepared with AI assistance (Claude Code): the code, the measurements and this text were produced by Claude on the submitter's two Arc Pro B70s, at the submitter's request.
We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to read properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.
To reopen it takes about two minutes:
Re-test on 0.2.0-beta. It may already be fixed. Write down what you see.
Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
Fill in the parts of the issue form: what happened, the Strata version, your graphics card, memory and system.
Comment /reopen. A check runs and either reopens the issue or lists what is still missing.
Activity
Niko1221 commentedon Oct 19, 2026
Thanks for taking the time to report this.
We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to read properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.
To reopen it takes about two minutes:
/reopen. A check runs and either reopens the issue or lists what is still missing.The rules are in CONTRIBUTING.md.