[Vop3p]

Branch: Vop3p/Strata:prefill-timer-coalesce (one commit on main 7dd5cf1, +5 lines in PfTimer::mark). Not run on NVIDIA, SYCL or Metal hardware.

Problem and result

STRATA_PREFILL_TIMING=1 records a timing event at every PfTimer::mark, and compute() marks dequant once per expert. Under WDDM each cudaEventRecord is a submission of its own (~80 µs on an RX 6900 XT under Windows 11, ~3 µs on Linux), so the timer put ~16 records per MMQ group on the compute stream. The launch gaps it made were charged to dequant, so the breakdown pointed at the wrong stage, and it slowed the run it measured.

Two marks in a row in the same phase add nothing to the totals: the gap between them is charged to that phase either way. This change skips such a mark. Totals are the same, with a fraction of the records.

The same 32K prompt, the same build otherwise (main 7dd5cf1 vs this branch), STRATA_PREFILL_TIMING=1, ms of GPU timeline:

stage	main	this branch
GPU timeline	39,163	38,786
wait copy	5,179	7,784
dequant	6,163	2,864
every other stage	within ±0.4 s

About 3.3 s that main reports as dequant is the compute stream waiting for expert copies; with this change it shows as wait copy, which is where the time goes (the copy-event PR, #4005, then removes most of it).

Speed with the timer on, 5 interleaved cold-start pairs, prompt tok/s (median): 4K 587.4 (main) vs 589.1 (this branch, faster in 4/5); 32K 823.2 vs 830.0 (5/5). The timer off, same config: 589.2 / 828.7. So on main alone the timer costs 0.2-0.7%. It costs more once the copy waits are gone: with the copy-event switches on, the timer cost 8% of a 32K read before this change and 2.7% after (measured on a v0.2.3-based build of both patches).

Related discussion

None. Found while measuring the prompt path on Windows (the copy-event PR, #4005).

Approach

if (used > 0 && ph[used - 1] == phase) return; at the top of mark(), after the on check. fold() keeps the last mark and its phase, so a same-phase mark after a fold is skipped too, and the interval simply continues. Only builds run with STRATA_PREFILL_TIMING set are affected.

Validation

Passed:

• Windows 11, RX 6900 XT (gfx1030), driver 32.0.21045.16010, built with tools\hip\build_windows.bat (ROCm 10.2.0a20260930): the pairs and the breakdown above.
• HIP Linux build with the setup.py options, gfx1030; CUDA 13.2.86 strata_prefill compiles.

Not run:

• Not executed on NVIDIA hardware (CUDA compile only); no SYCL or Metal toolchain here.

Limits

• On main alone the speed effect is 0.2-0.7%: the change is about a correct breakdown, and about the timer's own cost once the copies are fast.

• The peer timer (PeTimer) has the same pattern on a second GPU; not changed here (no multi-GPU Windows box to measure it).

Developed with an AI coding assistant; every number above was measured on the RX 6900 XT machines named in it (Windows 11 / i9-10900K and Linux / Ryzen 5 5600X).

🤖 Generated with Claude Code

[Vop3p]

The metal-link failure is not from this change. The job stops in metal/src/mtl_rt.mm, which this PR does not touch:

mtl_rt.mm:69:70: error: use of undeclared identifier 'MTLGPUFamilyMetal4'; did you mean 'MTLGPUFamilyMetal3'?
mtl_rt.mm:92:11: error: property 'mathMode' not found on object of type 'MTLCompileOptions *'
mtl_rt.mm:92:34: error: use of undeclared identifier 'MTLMathModeFast'

Those are macOS 26 SDK symbols, and the runner builds with Xcode_15.4.app (its MacOSX.sdk has MTLGPUFamilyMetal3 as the newest family). Other open PRs fail the same step the same way (#4010, #4039, #4040). main does not run this workflow, so it does not show there.

Marking it ready for review. It has still not run on NVIDIA, SYCL or Metal hardware (CUDA compile only; see "Not run" in the description), and the switches it adds are off by default.