Repository navigation
pcie: the measured PCIe share default (PcieAuto) on gfx1100 too — gate-measured on a native pack - #2224
pcie: the measured PCIe share default (PcieAuto) on gfx1100 too — gate-measured on a native pack#2224xyzzing wants to merge 1 commit into
Conversation
…port of the XeStrata stabilization cluster's finding Upstream aa7b0cb gates PcieAuto to CUDA single-GPU (#if !defined(STRATA_USE_HIP)); the machinery itself is engine-generic (pool ms per missed expert + miss counts vs the probed link rate), and this build's start-up H2D probe is already best-of-4 with a +-0.7% spread, so the HIP exclusion was unmeasured-only, the same class as the ROUTE_TAIL_SKIP gate. Same opt-outs as CUDA: layer splits excluded, STRATA_PCIE_FRAC_DEFAULT=old restores the fixed start-up rule. Provenance: MistVVK/XeStrata commits 0530909/055cf9b0/c71cac34/ 5952e8e (their measurement-stabilization cluster) demonstrated the failure mode and the fix class; their exact subsystem is not portable (it is their own engine code). (cherry picked from commit acb3deb0522a8fc351a4047d2bd11b8ab7a73dca)
|
Gate run on 2x RX 7900 GRE (gfx1100, ROCm 10.2, NixOS) — the arm is neutral here, and the in-run refinement never fires with this configuration. Build. The v0.1.43 line ( Model and config. Swift 1.5 IQ3_XXS (native pack), 131072 context, Method.
The spread inside one arm (74.7-91.8 on the fresh pairs) is wider than the difference between arms, so by the PR's gate rule (no regression) the arm stays on this card too. The refinement does not engage here. Every session printed only the start-up probe: No With 1-4 % misses on a 28 GB/s link there is nothing for the pool measurement to re-share, so on this machine the PR changes the default without changing anything measurable — in the direction the PR claims (no regression), on the arch it claims. Not tested: no correctness/KL check in these arms (the PR's KL evidence is on the XTX); no canonical pack; no Windows; no CUDA; no single-GPU; no 262K context; arms are 8 requests each plus the daily-driver session quoted above. Full report with per-trial numbers and the log excerpt: |
…TA_PCIE_FRAC_DEFAULT=auto), not a gfx1100 default: the author's gate was no-regression only and a second gfx1100 report saw it never engage
|
Thanks. We ported it with your authorship but as an opt-in: STRATA_PCIE_FRAC_DEFAULT=auto turns the refinement on for a HIP card outside gfx12, the default is unchanged. Your gate was a no-regression one and the GRE report shows it does nothing when 95% of the experts are cached, with small negatives on follow-up prefill, so we did not want to change a default on a card we cannot measure. With a gain measured on a card with real PCIe traffic (16 GB-eq, lots of misses) we would revisit. |
|
The 7900 XTX fork keeps the measured default-on form: under serve the refinement moves pcie_frac 0.55 -> 0.25 and the PCIe share of routed reads from 8-13% to 3.9%, self-limiting at 3 updates; 0.1.45 composes it with your opt-in |
0.1.43 made PcieAuto the default under the gfx12 decode defaults on HIP (
decode_defaults_arch_ok→ gfx1200/gfx1201, measured on 2×R9700) and for the canonical pack on CUDA (ab97f17). Every other AMD part keeps its earlier behaviour.This extends the default to gfx1100 (RX 7900 XT/XTX/GRE) — the one other arch where the in-run refinement has been gate-measured, on a native (IQ) pack:
No-regression gate (4 alternating engine starts @64k, same args/env, paired medians): prefill 2189 (base) vs 2180 (with the arm), decode 65.4 vs 65.4 — inside noise. The gate's rule was no-regression, so the arm stays.
Engagement (under
--serve, real ~95K-token request): the in-run refinement fires and printspool 42-45 us/missed expert (45-48 GB/s of expert bytes), the link 28.4-28.5 GB/s -> pcie_frac 0.25 (was 0.55), self-limited to 3 updates — the engine's ownf = L/(L + 1.85 r). It has run as our daily-driver production config since 2026-10-12 and was re-verified after the 0.1.43.1 refresh.Scope, deliberately narrow:
pcie_auto_fork_arch_ok()pins the arch string exactly. gfx1200/1201 keep 0.1.43's default; every other AMD part keeps the gate's earlier behaviour. Nothing is claimed for unmeasured archs.STRATA_PCIE_FRAC_DEFAULT=old,--pcie-frac N, the layer-split rules (incl. theSTRATA_PCIE_AUTO_SPLITopt-in).Build: HIP gfx1100, 373/373 targets on the nightly ROCm 10.2 toolchain. The arm is host-side policy (no kernel change); the CUDA path is untouched —
pcie_auto_fork_arch_ok()returns false there anddecode_defaults_arch_ok()is always true.