[xyzzing]

RDNA3 (gfx1100) WMMA kernel for the int8-KV QSA prompt attention — opt-in STRATA_HIP_WMMA=1, gfx12 + gfx11

This extends the S6 matrix-core prompt attention (currently gfx12-only) to RDNA3. gfx1100 keeps the ordered FP32 fallback as the default; the arm engages only under the existing STRATA_HIP_WMMA=1 opt-in, and the default path is byte-identical to current behavior. (In particular, unlike #313, nothing in the default HIP path changes — the scalar fallback stays exactly as upstream ships it.)

Measured on an RX 7900 XTX (gfx1100, 24 GB), engine 0.1.39, ROCm 10.2 nightly SDK (hipBLASLt 100500 table, gfx1100), production-mirrored config (spec 3, GR_V3, --spec-min-p 0.70, --kv int8, --kv-resident 32768). Since there is no AMD card in the upstream test ring, these numbers carry the PR.

The layout that makes it work

RDNA3 runs the same v_wmma_f32_16x16x16_f16 instruction as gfx12, but through the unsuffixed builtin — and its wave32 fragment layout differs from the gfx12-suffixed builtin's on all three operands (AMD Matrix Instruction Calculator, rdna3 tables, wave32, no modifiers; hardware-confirmed by a standalone probe before touching the kernel):

operand	lane l, slot i holds
A	A[l & 15][i]
B	B[i][l & 15]
C/D	D[(l >> 4) + 2*i][l & 15]

Packing the gfx12 way into the RDNA3 builtin passes compilation and fails parity at ~30× the output scale. The kernel's packing sites re-index accordingly; the online-softmax protocol, LDS staging, per-cell K/V scales and masking are unchanged (the B side still holds one cell per lane, so the per-cell scale folds exactly as on gfx12).

Parity

Upstream's own hip_prompt_attn_wmma (first time it runs off gfx12) — 3/3 PASS:

shape	vs FP64 (new)	vs FP64 (old FP32 kernel)	new vs old	speed
int8, ctx 32768, 2048 queries	3.28e-06	2.13e-06	8.64e-06	34.05 → 8.41 ms/chunk (4.05×)
int8, ctx 1500, 1500 queries	3.79e-06	1.78e-06	6.32e-06	12.32 → 2.41 ms/chunk (5.12×)
int8, ctx 2100, 256 queries	4.45e-06	2.03e-06	5.44e-06	6.18 → 1.33 ms/chunk (4.66×)

Objdump gate: exactly 32 v_wmma_f32_16x16x16_f16 in the compiled kernel (the loop body — no fallback can masquerade).

End-to-end (paired A/B, arms alternate, same binary, env-gated)

The FP32 fallback was measured at 25.6% (128K) / 26.9% (32K) of prefill GPU time on this card — the single largest prefill phase.

tier	arm-b / arm-a median	pair wins	prefill tok/s (a → b)
128K	+22.8%	5/5 (per-pair 1.212–1.253)	1513.8 → 1863.2
32K	+17.9%	5/5	1669.3 → 1968.0
1K decode	+2.3% (noise)	3/5	65.9 → 67.4 tok/s

Decode is flat at 32K (−0.03%) and 128K (medians equal) — the decode protection cap holds. Attribution agrees with the mechanism: the arm drops the qsa-attn share from 26–27% to 8.4–10.6% of prefill GPU time.

Notes

• For RDNA3 WMMA kernels for the gfx1100 prefill (opt-in, runtime-gated on gfx11; rebased on 0.1.30) #313's two review points (by @Niko1221): this PR value-parses the env (e[0] == '1'), and no STRATA_WMMA_GFX11-style compile define leaks into other device passes — the arch gate is the same runtime gcnArchName prefix check upstream already uses, extended to gfx11.
• The 2-file rebase surface (qsa_prompt_attn.cu, qsa_prompt_attn_parity.cpp) is deliberately minimal.
• Rebased on current main (6f32ec0, 0.1.39). Credit to @StevenChenSE's RDNA3 WMMA kernels for the gfx1100 prefill (opt-in, runtime-gated on gfx11; rebased on 0.1.30) #313 for pioneering the gfx1100 WMMA prefill direction; the dense-GEMM half of that PR is orthogonal to this arm.

[2jztricks]

Tested on RX 7900 XTX (gfx1100), Ryzen 7950X3D, 64GB RAM, Orca Qwen3.8-Flash-Next IQ3_XXS, INT8 KV and 131072 context.

With STRATA_HIP_WMMA=1, median total time for 127999 prompt tokens + 256 generated tokens dropped from 95.359s to 79.138s: 17.0% less time.

Three alternating on/off pairs all improved, with zero prompt reuse.
Numerical parity passed on ROCm 7.1.1 and again after upgrading to ROCm SDK 10.

Fresh prompt | Prefill | Generation | Total time, including 256 generated tokens
4K | 1,378 tok/s | 63 tok/s | 7.32 s
32K | 1,961 tok/s | 64 tok/s | 22.57 s
128K | 1,771 tok/s | 64.1 tok/s | 76.38 s

All used zero prefix reuse, with the model already loaded.
With a cached 128K prefix, tested follow-ups completed in 0.66–1.04 seconds, including switching between controller and worker conversations.
Current settings: speculation six, eleven CPU workers, 128K context, lookup on SSD.

Thanks!

[Jlauti]

AI disclosure: This comment was drafted and posted by an AI assistant (Codex) at the account owner's request, using recorded benchmark logs and results.

Additional validation on RX 7900 XTX / gfx1100

Tested PR commit 49969d1 against parent 6f32ec0 using Qwen3.8-Flash-Next GSQ-RCO IQ3_S.

Fixed configuration: Strata 0.1.39, ROCm 10.2.0a20261003, HIP 7.17.26392, hipBLASLt 100500 with a locally calibrated 40-row table. Prefill 32768, ring 96, workers 48, pcie_frac 0, spec-min-p 0.50, spec4/MTP4, int8 KV. Expert cache, pinned backing, speed projection and sampling penalties unchanged.

Loaded ROCm libraries verified through /proc/PID/maps; no Lt fallbacks observed.

End-to-end performance

Fixed fresh prompt: 24,616 server-counted tokens + 512 generated, temperature 0, cache_n=0 and zero reused prompt tokens. Fresh engine per arm, one excluded warmup.

Arm	Measured runs	PP median, tok/s	TG median, tok/s	Wall median
Parent, WMMA off	3	1581.8	83.35	21.80 s
PR, HIP_WMMA=1	3	2060.6	83.50	18.13 s
PR, fresh-engine confirmation	5	2047.9	83.15	18.25 s

Confirmation: +29.5% PP, −16.3% wall time, −0.24% TG. Confirmation wall range was 18.15–18.38 s.

Attribution

Separate single-request diagnostics using STRATA_PREFILL_TIMING=1, excluded from throughput medians:

GPU event timing	Parent	PR
QSA attention	4395 ms / 27.7%	1165 ms / 9.2%
Total timeline	15853 ms	12686 ms

The 3230 ms attention reduction closely matches the 3167 ms total reduction. The engine explicitly logged STRATA_HIP_WMMA: the prompt attention on matrix cores on (gfx1100).

Correctness and limits

• hip_prompt_attn_wmma passed with zero failures; FP64 comparison errors remained approximately 3.3–4.5e-06.
• Candidate output was deterministic across both warmups and all eight measured 24K requests. It differed from baseline but remained sensible.
• No crashes, observed HIP/non-finite errors or prompt-cache reuse.
• GR_V3 was kept off: enabling it triggered Strata's safety fallback on this setup.
• One cold 80K/512 request completed at 1872.8 PP / 68.38 TG / 50.47 s. This is a completion check, not a matched long-context gain claim.
• The 512-token cap ends during reasoning; this does not establish completed-answer quality or exhaustive stability.

Expert-cache hit / draft acceptance were 0.763 / 70.87% for the control and 0.771 / 73.67% for the candidate. Different greedy continuations can affect speculative decode work.

The prompt-processing gain reproduces, decode remains effectively unchanged at 24K, and stage attribution supports the intended mechanism.

[xyzzing]

Thank you both — two independent RX 7900 XTX validations on the same GPU class is exactly the evidence this needed.

For the record, our own rig's numbers agree with both: 128K prefill +22.8% median 5/5 (1514 → 1863 tok/s) end-to-end and the same attribution shape (qsa attention 27.5% → 9.9% of the GPU timeline at 128K, STRATA_PREFILL_TIMING=1), decode flat within noise. Jlauti's 27.7% → 9.2% vs our 27.5% → 9.9% is the same result within run noise, on a different card — good sign the arm is not tuned to one machine.

One follow-up note from our side: with the attention term now at ~10%, we measured what is left in it — an INT8-Q feasibility fixture (rocWMMA int8→i32, integer-exact, fork 65a3c8f) showed the remaining headroom is gated by the int8 quantization class (softmax weight delta 9.5e-2 vs a 2e-2 bound under the 12x-outlier activation class), not by MMA throughput. So this PR's arm is close to the practical floor for the score path's precision class; the next lever on this card is elsewhere (MoE cluster).

AI disclosure: this comment was drafted by an AI assistant (GLM-5.3, Z.ai) at the account owner's direction, from our recorded benchmark logs (artifacts/rocm/record-pa-wmma-e2e-verdict-20261004.json, qsa-attn-recon-20261005.md).

[Niko1221]

Thank you. 0.1.40 has an RDNA3/RDNA3.5 WMMA prompt attention on the exact gfx11 list (gfx1100/1101/1102/1150/1151) behind STRATA_HIP_WMMA=1, so this PR is superseded; your kernel and layout work was the starting point and we checked it passes on gfx1151. Closing.