Repository navigation
Description
Is your feature request related to a problem? Please describe.
After #810 split the Windows build into two parallel jobs, the Windows lane is no longer what sets the length of a CI run. Over the 20 successful push runs on main since that merge, the Windows lane finished at a median of 8.3 minutes after the run started, while the whole run took a median of 13.3 minutes. The remaining critical path is changes -> clippy -> build-and-test -> E2E tests -> E2E consolidated report. In those runs build-and-test started at about 2.9 minutes and ended at about 6.5, so E2E tests only started at about 6.9 minutes.
E2E tests uses build-and-test only as a gate (it checks the job's result and does not consume any of its output), but build-and-test also runs several steps that nothing downstream waits for. Median step durations over the same 20 runs: Acceptance install lifecycle 106s (range 76 to 161s), GPU reclaim self-test 31s and Pinned key consistency check 10s, out of a median job time of 218s. Together that is about 147s of a job that otherwise takes about 70s (checkout, native deps, toolchain, cargo build --workspace at 26s on a warm sccache, and the local smoke test).
Describe the solution you'd like
Move the steps that E2E tests does not depend on out of build-and-test into a sibling job on the same trigger conditions: Acceptance install lifecycle, GPU reclaim self-test, Pinned key consistency check, and the three near-instant self-test steps (Acceptance harness self-tests, Portable WSL build deps self-test, Release readiness self-test). build-and-test keeps the build and the local no-fallback smoke test, so E2E tests can start about 2.4 minutes earlier. This is the same shape as the windows-debug / windows-release split. The expected saving is an estimate from step medians, not something I have run on a branch.
Open points that need an answer before or while implementing this:
- Required checks. If
build-and-testis a required status check, the new job has to become one too, otherwise a failing lifecycle run would stop blocking merges. That needs someone with admin rights on the repository. Keepingbuild-and-testas an aggregator of the two jobs, aswindows-build-and-testdoes, does not help here, becauseE2E testswould then wait for the slow job again. - E2E gate semantics.
E2E testswould no longer be skipped when the lifecycle steps fail. That is fine as long as the new job is itself required. - Build dependence.
cargo xtask e2ebuilds its own release binaries (cargo build --release -p rocm -p rocmd), so the lifecycle step does not reuse the debug build frombuild-and-test. What the sibling job gives up is the warm dev-profile dependencies for the e2e harness, the same trade-off documented forwindows-release. Cold-cache duration (fork pull requests, first run after the split) has not been measured. - Cache pressure. The rust-cache key includes the job id, so a new job starts cold and orphans the old entry, and the workflow comments note the Actions cache is already at its size cap.
- Contract tests and docs.
xtask/src/workflow_contract.rspins the Windows lane in detail; a new Linux gate with a required result would need equivalent pins, and the comments and docs that describe whatbuild-and-testruns would need updating.
Describe alternatives you've considered
- Do nothing beyond the companion change that stops
build-and-testfrom waiting forclippy. That change alone is estimated to bring the median run from about 13.3 to about 11 minutes and has none of the open points above. - Change the
needsofE2E testsso it runs alongsidebuild-and-test. This is the largest saving on paper, but it changes the E2E gate and the gain is small once the Windows lane (8.3 minutes) becomes the floor.
Additional context
Suggested sequencing: land the clippy wait change first, re-measure over at least 20 push runs on main, and only then decide whether this split is still worth its cost. After both changes the Windows lane and Test (affected crates) would be close to the critical path, so a further saving would need work on those as well. Run timings above come from the CI workflow's job and step durations via the GitHub API for push runs on main.
Activity