HIP: time the PCIe probe on the host on Windows by BlueKingMuch · Pull Request #377 · Niko1221/Strata

BlueKingMuch · 2026-10-01T14:26:26Z

The PCIe probe sets the share of the missed experts the GPU reads over the link (--pcie-frac): the native default 0.55 from 20 GB/s up, less on a slower link (#44). On Windows HIP it timed its copies with events that do not bracket them there, so it read impossible speeds and every link kept the x16 share.

What changes

On Windows HIP builds, probe_pcie_h2d_gbps times the same 4 x 256 MiB copies on the host clock, between two cudaDeviceSynchronize calls; the warm-up copy is synchronized before the clock starts. The 1 GiB takes about 40 ms on this card, so the synchronize around it hardly matters.
CUDA and Linux HIP builds keep the events (#if defined(STRATA_USE_HIP) && defined(_WIN32)).
The layer split's per-GPU probe calls the same function, so it gets the same timing.

Measured

RX 6800 on PCIe 4.0 x16, Windows 11, HIP SDK 7.2:

	events (before)	host clock (this PR)
probe reading	3,348 to 25,999 GB/s over 24 engine starts (median about 19,000)	27.2 GB/s in all 4 starts
resulting `pcie_frac`	0.55	0.55

A standalone copy of the probe on that card shows why: its events measure 0.13 ms for the 1 GiB, the host clock 43 ms (25.1 GB/s). The same test reads 27.2 GB/s for a copy from registered memory and 27.0 GB/s for a kernel reading mapped pinned memory, so 27.2 GB/s is what this link carries. In one engine start with GPU_FLUSH_ON_EXECUTION=1 (the runtime submits every command right away) the events read 26.9 GB/s.

On this card the share is 0.55 either way, so nothing changes in speed here. On a slower link the share now shrinks as #44 intended, e.g. to 0.28 at 13 GB/s (about what PCIe 3.0 x16 carries); not tested on such a link.

🤖 Generated with Claude Code

https://claude.ai/code/session_015Ld5YDzuKW1hgMVwwgRZDP

The PCIe probe sets the share of the missed experts the GPU reads over the link (--pcie-frac): the native default 0.55 from 20 GB/s up, less on a slower link (Niko1221#44). On Windows HIP it timed its copies with events that do not bracket them there: an RX 6800 read between 3,348 and 25,999 GB/s in 24 engine starts, so every link kept the x16 share. A standalone copy of the probe on that card: its events measure 0.13 ms for the 1 GiB, the host clock 43 ms (25.1 GB/s). On Windows HIP the probe now times the same copies on the host, between two synchronizes (1 GiB, tens of milliseconds): the same RX 6800 on PCIe 4.0 x16 reads 27.2 GB/s in every run, which keeps 0.55 there. On a PCIe 4.0 x8 or 3.0 x16 link the share now shrinks as Niko1221#44 intended. CUDA and Linux keep the events. Claude-Session: https://claude.ai/code/session_015Ld5YDzuKW1hgMVwwgRZDP Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… samples logged The probe timed one 1 GiB burst; an x16 PCIe 4 laptop (RTX A3000) read 18.5, 6.9 and 5.8 GB/s in three starts, and the low reading set pcie_frac (0.12-0.15 where calibrate.py found 0.35). The same 1 GiB is now four bursts of 256 MiB, each timed on its own (events between them on the stream; on Windows HIP the host clock, #377), and the fastest is the link's figure: a disturbance only ever slows a copy, while the median would still follow one lasting through half the bursts. The probe copies no more than before. The share was base above 20 GB/s and base * bw / 26 below (19.9 GB/s -> 0.42, 20.0 -> 0.55; 4.0 -> 0.08, 3.9 -> 0). It is now base * min(1, bw / 20): continuous, unchanged from 20 GB/s up (an x16 PCIe 4/5 link reads ~26-28 GB/s, so the default stays 0.55 there), in proportion to the link below, so the link's time on its share stays about what the x16 share costs. The rule is one function for CUDA0 and the layer split's stages. The log names every burst: "PCIe probe: 28.9 GB/s host->device (best of 28.9 28.9 28.9 28.9)" (the RTX 5070 here; pcie_frac 0.55 as before). Noted, not changed: a given --pcie-frac is every stage's share and skips the stages' probes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN

…d the host-timed PCIe probe (#377) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN

…s the desktop's VRAM (#380 #377 #497), a steadier PCIe probe (#485), fixes #496 #495 #498 #505 #493; setup requires it Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN

Niko1221 · 2026-10-02T19:51:30Z

Released in 0.1.37: the host-timed probe on Windows HIP, now per burst together with #485's best-of-four probe. Merged under your name (GitHub doesn't show it as merged because it went in by hand). Thank you!

… samples logged The probe timed one 1 GiB burst; an x16 PCIe 4 laptop (RTX A3000) read 18.5, 6.9 and 5.8 GB/s in three starts, and the low reading set pcie_frac (0.12-0.15 where calibrate.py found 0.35). The same 1 GiB is now four bursts of 256 MiB, each timed on its own (events between them on the stream; on Windows HIP the host clock, #377), and the fastest is the link's figure: a disturbance only ever slows a copy, while the median would still follow one lasting through half the bursts. The probe copies no more than before. The share was base above 20 GB/s and base * bw / 26 below (19.9 GB/s -> 0.42, 20.0 -> 0.55; 4.0 -> 0.08, 3.9 -> 0). It is now base * min(1, bw / 20): continuous, unchanged from 20 GB/s up (an x16 PCIe 4/5 link reads ~26-28 GB/s, so the default stays 0.55 there), in proportion to the link below, so the link's time on its share stays about what the x16 share costs. The rule is one function for CUDA0 and the layer split's stages. The log names every burst: "PCIe probe: 28.9 GB/s host->device (best of 28.9 28.9 28.9 28.9)" (the RTX 5070 here; pcie_frac 0.55 as before). Noted, not changed: a given --pcie-frac is every stage's share and skips the stages' probes.

…d the host-timed PCIe probe (#377)

…s the desktop's VRAM (#380 #377 #497), a steadier PCIe probe (#485), fixes #496 #495 #498 #505 #493; setup requires it

…are, the samples logged The probe timed one 1 GiB burst; an x16 PCIe 4 laptop (RTX A3000) read 18.5, 6.9 and 5.8 GB/s in three starts, and the low reading set pcie_frac (0.12-0.15 where calibrate.py found 0.35). The same 1 GiB is now four bursts of 256 MiB, each timed on its own (events between them on the stream; on Windows HIP the host clock, Niko1221#377), and the fastest is the link's figure: a disturbance only ever slows a copy, while the median would still follow one lasting through half the bursts. The probe copies no more than before. The share was base above 20 GB/s and base * bw / 26 below (19.9 GB/s -> 0.42, 20.0 -> 0.55; 4.0 -> 0.08, 3.9 -> 0). It is now base * min(1, bw / 20): continuous, unchanged from 20 GB/s up (an x16 PCIe 4/5 link reads ~26-28 GB/s, so the default stays 0.55 there), in proportion to the link below, so the link's time on its share stays about what the x16 share costs. The rule is one function for CUDA0 and the layer split's stages. The log names every burst: "PCIe probe: 28.9 GB/s host->device (best of 28.9 28.9 28.9 28.9)" (the RTX 5070 here; pcie_frac 0.55 as before). Noted, not changed: a given --pcie-frac is every stage's share and skips the stages' probes.

…#380) and the host-timed PCIe probe (Niko1221#377)

…MD counts the desktop's VRAM (Niko1221#380 Niko1221#377 Niko1221#497), a steadier PCIe probe (Niko1221#485), fixes Niko1221#496 Niko1221#495 Niko1221#498 Niko1221#505 Niko1221#493; setup requires it

…are, the samples logged The probe timed one 1 GiB burst; an x16 PCIe 4 laptop (RTX A3000) read 18.5, 6.9 and 5.8 GB/s in three starts, and the low reading set pcie_frac (0.12-0.15 where calibrate.py found 0.35). The same 1 GiB is now four bursts of 256 MiB, each timed on its own (events between them on the stream; on Windows HIP the host clock, Niko1221#377), and the fastest is the link's figure: a disturbance only ever slows a copy, while the median would still follow one lasting through half the bursts. The probe copies no more than before. The share was base above 20 GB/s and base * bw / 26 below (19.9 GB/s -> 0.42, 20.0 -> 0.55; 4.0 -> 0.08, 3.9 -> 0). It is now base * min(1, bw / 20): continuous, unchanged from 20 GB/s up (an x16 PCIe 4/5 link reads ~26-28 GB/s, so the default stays 0.55 there), in proportion to the link below, so the link's time on its share stays about what the x16 share costs. The rule is one function for CUDA0 and the layer split's stages. The log names every burst: "PCIe probe: 28.9 GB/s host->device (best of 28.9 28.9 28.9 28.9)" (the RTX 5070 here; pcie_frac 0.55 as before). Noted, not changed: a given --pcie-frac is every stage's share and skips the stages' probes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN

…#380) and the host-timed PCIe probe (Niko1221#377) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN

…MD counts the desktop's VRAM (Niko1221#380 Niko1221#377 Niko1221#497), a steadier PCIe probe (Niko1221#485), fixes Niko1221#496 Niko1221#495 Niko1221#498 Niko1221#505 Niko1221#493; setup requires it Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN

Niko1221 closed this Oct 2, 2026

Niko1221 added a commit that referenced this pull request Oct 6, 2026

docs: Windows AMD's free-VRAM figure within the WDDM budget (#380) an…

b038240

…d the host-timed PCIe probe (#377)

Niko1221 added a commit that referenced this pull request Oct 6, 2026

Engine 0.1.37: a silent engine is restarted (#481), Windows AMD count…

fc4edd2

…s the desktop's VRAM (#380 #377 #497), a steadier PCIe probe (#485), fixes #496 #495 #498 #505 #493; setup requires it

KongChengZhi mentioned this pull request Oct 6, 2026

Windows AMD: prebuilt HIP zip running a model on a discrete card (Radeon AI PRO R9700, gfx1201) #1141

Closed

anon761 pushed a commit to anon761/Strata-upstream that referenced this pull request Oct 6, 2026

docs: Windows AMD's free-VRAM figure within the WDDM budget (Niko1221…

a5d8632

…#380) and the host-timed PCIe probe (Niko1221#377)

Navigation Menu

Repository navigation

HIP: time the PCIe probe on the host on Windows - #377

HIP: time the PCIe probe on the host on Windows#377
BlueKingMuch wants to merge 1 commit into
Niko1221:mainfrom
BlueKingMuch:win-probe

BlueKingMuch commented Oct 1, 2026

Uh oh!

Niko1221 commented Oct 2, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

BlueKingMuch commented Oct 1, 2026

What changes

Measured

Uh oh!

Niko1221 commented Oct 2, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants