From e326847f73927a49dc5ab00bcc3b7a1ef25a69d2 Mon Sep 17 00:00:00 2001 From: Vop3p <69450118+Vop3p@users.noreply.github.com> Date: Fri, 23 Oct 2026 20:42:31 -0400 Subject: [PATCH] docs(AMD_HIP): why STRATA_SH_STREAM goes both ways, and leave it off on Windows The fork only pays when it has a wait to hide: with CPU experts in the window it forks after the doorbell (STRATA_SH_FORK_LATE) and fills the GPU's wait for the CPU pool (#1272); with every expert resident it is two cross-stream event dependencies per layer (7fd67dd, R9700). On Windows 11 + RX 6900 XT (0.2.3 rdna2 package) the fork together with --kv-resident faulted the first request; bisected to STRATA_SH_STREAM. Co-Authored-By: Claude Opus 5.5 --- docs/AMD_HIP.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/docs/AMD_HIP.md b/docs/AMD_HIP.md index 9dd8aaaf14a..a02c5e72e4b 100644 --- a/docs/AMD_HIP.md +++ b/docs/AMD_HIP.md @@ -354,6 +354,25 @@ both ways on different PCs, `./setup.sh --calibrate` measures it on yours (off a `STRATA_SH_STREAM=1` into the config's `"env"` only when it wins by more than 3%; a later calibration that finds it no longer faster removes it again. +Why the same card goes both ways: the fork has to hide something. With experts on the CPU in the window, the shared +expert forks off only after the doorbell is published (`STRATA_SH_FORK_LATE`, `src/core/verify.cpp`), so it runs while +the GPU would otherwise wait for the CPU pool; on a 16 GB card whose cache holds only part of the experts that wait is +about half of a verify window (2026-10-23, one RX 6900 XT under Linux, 4,859 slots, `STRATA_DECODE_TIMING=1`: GPU-reach wait 20.7 of 39.1 ms per window), +which is where #1272's gain comes from. When every routed expert is on the GPU there is no wait to fill and the fork is +two cross-stream event dependencies per layer and nothing else: 7fd67dd measured it on an R9700 with +`--resident-experts` and the KV stream path running (48,067-token prompt), 43.4 tok/s with the fork against 62.9 +without; a 256-token prompt barely moved (65.4 / 62.7). Windows adds its own cost to every cross-stream dependency +(#816). + +**Windows: leave it off.** On one RX 6900 XT under Windows 11 (the 0.2.3 `rdna2` package, engine 0.1.48) the fork +together with KV streaming (`--kv-resident 32768` under a 64K context) stopped the engine on its first request +("verify: unspecified launch failure", reported first by the MTP draft step that synchronises next; exit code +0xC0000409); with the fork off, or with KV streaming off, the same configuration answered normally. Bisected switch by +switch with `STRATA_HIP_PROMPT_F16`, `STRATA_HC_PERSIST`, `STRATA_HIP_ADAPT_KERNEL_COPY`, `STRATA_PREFILL_CPU_SHARE`, +`STRATA_PIPED_CONCURRENT`, `STRATA_PREFILL_STREAM_MIN` and `STRATA_IO_THREADS` set beside it: only removing +`STRATA_SH_STREAM` cleared it. Two of the same card under Linux (layer split, the same model, `STRATA_SH_STREAM=1`, +`--kv-resident 32768`) have served without this fault since 0.1.40. The faulting kernel was not isolated. + **The draft's side stream (`STRATA_MTP_SIDE_STREAM`):** HIP does not use the MTP drafter's branch stream, and since 0.1.40.2 it does not create it (#1240: an R9700 lost prompt speed with it). gfx103x is the exception and creates it: on an RX 6800 under Windows (`STRATA_HIP_PROMPT_F16=1`, IQ3_XXS, two runs each) a 7.9K-token prompt reads at 798-808 tok/s