Repository navigation
ci: fetch only the host's crates in the queue lane - #1057
Conversation
`cargo fetch --locked` without `--target` downloads crates for every platform. The lane builds, pre-warms and runs the suite on the host and sets no build target, so fetch the host triple's crates only, reading the triple from `cargo -vV`. Offline `cargo metadata` after a host-only fetch into an empty CARGO_HOME resolves the full workspace (386 MB downloaded, against 876 MB for the unrestricted fetch). Also note that the first `cargo --version` relies on rustup's proxy installing the toolchain named by rust-toolchain.toml, and is a no-op where cargo is not rustup-managed. Follow-up to review comments on #1009. Signed-off-by: Tomas Saaristola <tsaarist@amd.com>
juhovainio
left a comment
There was a problem hiding this comment.
Looks good. The host-only cargo fetch --target is a sensible scoping, it fails closed when cargo -vV reports no host triple (with a regression test), and the fake-cargo fixtures only answer the exact host-scoped fetch, so a revert to plain fetch --locked would fail the tests.
Non-blocking notes:
- The "host-only is enough" claim rests on the offline
cargo metadatacheck; no real offline build was run. If a later lane step ever builds for another target, a cold lane would hit the network without a retry (see inline). - Cosmetic: one over-long line left by the doc reflow (see inline).
- The change is CI-internal, so no Gherkin scenario is needed; worth one line in the PR text saying so.
CI at this head is green for all completed checks; the pending GPU/E2E lanes are the known runner-pool outage.
Security scan found no critical or high findings on lines changed by this pull requestView the security scan run for complete reports. “New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta. |
nowycondro
left a comment
There was a problem hiding this comment.
Review — no blocking findings
Full review of the whole change (3 files: scripts/run-e2e-lane.sh, xtask/src/workflow_contract.rs, docs/nightly-on-the-queue.md).
Follow-up to the review comments on #1009: that PR's two open threads (restrict the fetch to the host triple; document that the toolchain download relies on rustup's proxy) are both addressed here — confirmed against #1009's own review thread, where the author committed to this as a follow-up and later marked both threads "addressed in #1057."
Blocking
None
Non-blocking
- A doc paragraph was rewrapped unevenly, leaving one line much longer than its neighbours —
docs/nightly-on-the-queue.md:122
The edited paragraph's lines are 73/77/107/79 characters; every other prose line nearby is 70–92 chars. This is the same class of nit a past review on this exact file/script pair caught and the author fixed by rewrapping evenly (see the review history on #999, where a reviewer wrote "the rewrap leaves line 112 very long; worth rewrapping evenly" and the paragraph was reflowed in response). Nothing lints this automatically.
Confidence 80 · mechanical · Fix: reflow the paragraph evenly at roughly the same width as the surrounding lines. - The new host-triple plumbing is only ever exercised with one hardcoded triple string —
xtask/src/workflow_contract.rs:9814(stubuname),:9832-9855(stubcargo's-vV/fetch --targetarms)
Every test that exercises the newcargo -vV→--target <host>path (the default case and the new "no host line" test) usesLaneCase::default()'smachine: "x86_64", and the stubcargo's-vVoutput and thefetch --locked --target …match arm are both hardcoded to the literalx86_64-unknown-linux-gnu. The one existing test that variesmachine(the_rustup_pin_follows_the_machine, pre-existing, for the rustup installer's triple selection) doesn't touch this path. So the harness proves the plumbing works for that one literal string, not that an arbitrary host string flows through correctly end to end. Low real-world risk, since the script's handling is just text capture-and-interpolate rather than architecture-specific logic, but it's a gap in what the new tests actually prove.
Confidence 72 · mechanical · Fix (optional): parametrize the stub's-vVhost line and the fetch match arm oncase.machine/an explicit triple, and add or extend a test with a non-x86_64triple.
Decisions for the author
None
Positive signals
- Using a herestring (
<<<"${cargo_info}") rather than piping into thewhile readloop is the correct, deliberate choice — it avoids the classic bash bug where a pipedwhile readruns in a subshell and loses the variable it's trying to set. - The missing-host-triple case fails fast with a clear message and a dedicated test (
a_lane_stops_when_cargo_reports_no_host_triple) that asserts no fetch, no build, and no output directory — rather than silently passing an empty--target. - Leaving
cargo -vVoutside theretrywrapper is well-reasoned and matchesretry's stated scope (network-reaching commands only):-vVis a local introspection call, and by the time it runs the toolchain is already installed via the preceding, already-retriedcargo --version.
Deployment notes
As of this review, two of the self-hosted E2E checks — E2E tests (MI300X, current) and E2E tests (W7900, current) — were still queued and had not completed, even though every other check (including the Strix Halo self-hosted lanes, clippy, the xtask test job, and CodeQL) was green. These are the actual hardware lanes this script change touches, so it's worth confirming they finish green before merging rather than relying only on the checks that had completed at review time.
What this covered
Diff between origin/main's merge-base 61fdb25f and head 96f6275e. scripts/run-e2e-lane.sh and docs/nightly-on-the-queue.md read in full (both versions). In xtask/src/workflow_contract.rs (10,367 lines, almost entirely a #[cfg(test)] mod tests block): the full LaneCase struct and Default impl, the complete fake-executable/fixture generation in run_lane, and every test function exercising LaneCase/run_lane were read in full; the remaining unrelated workflow-YAML contract tests earlier in the file were not read line by line (confirmed by grep that no other location references the touched fields or command strings). Checked the repo's root AGENTS.md in full: the docs/architecture.md-entry and module-size rules don't apply (no file added/renamed/removed; the file is exempt as a #[cfg(test)]-gated module), and the Gherkin-scenario requirement doesn't apply (CI-only plumbing, no user-observable CLI behavior changes). Checked git history and past reviews on PRs #950, #999, and #1009 touching these same files for still-applicable concerns. Confirmed the workspace genuinely has per-platform (cfg(windows)/cfg(target_os = "macos")) dependency sections in several crates, and no .cargo/config.toml build-target override, so restricting the fetch to the host triple cannot starve the later untargeted build. CI checked via the live status rollup (see Deployment notes for the one caveat). Did not run cargo test/clippy/shellcheck directly in this environment (no Rust toolchain available here); relied on reading the code directly plus CI's own green checks and the author's stated verification.
Not reconciled against this PR's own review discussion — only commit-metadata was checked beforehand, to confirm no duplicate review already existed at this head.
The lane fetches only the host's crates, so a CARGO_BUILD_TARGET set from outside would need crates it skipped and meet the network without a retry. Refuse it up front, and exercise the host-scoped fetch with a non-x86_64 triple. Rewrap the paragraph the earlier reflow left uneven. Signed-off-by: Tomas Saaristola <tsaarist@amd.com>
Clippy's format_in_format_args refused the nested format! introduced with the host-triple stub. Signed-off-by: Tomas Saaristola <tsaarist@amd.com>
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · head 2838b5a6
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — needs work
Full review of the whole change (61fdb25f…2838b5a6, 3 files: scripts/run-e2e-lane.sh, docs/nightly-on-the-queue.md, xtask/src/workflow_contract.rs).
Follow-up to the review comments on PR #1009; no tracker ticket is named by this PR.
Blocking
- The PR description doesn't mention the new hard refusal on a
CARGO_BUILD_TARGETenvironment variable —scripts/run-e2e-lane.sh:33-38,docs/nightly-on-the-queue.md:125, PR description
The diff adds a real new failure mode: the lane now exits 1 immediately, before anything else runs, ifCARGO_BUILD_TARGETis already set in its environment ("CARGO_BUILD_TARGET is set; unset it: the lane fetches only the host's crates"), and a dedicated test (a_lane_refuses_a_build_target_and_scopes_the_fetch_to_the_reported_host) exercises it. The PR's own description only says the lane "sets no build target, so the other platforms' crates were downloaded for nothing" and that "a future step that sets a build target would need its own fetch" — it never states that the lane will now refuse to start at all if that variable happens to already be set for an unrelated reason (e.g. something upstream in the container image). A reader going by the description alone would not learn this guard exists.
Confidence 78 · mechanical · Fix: add one sentence to the PR description noting the lane now refuses to start whenCARGO_BUILD_TARGETis set, matching what the code, the doc, and the second commit's own message already say — no code change needed.
Non-blocking
None
Decisions for the author
None
Positive signals
- Using a herestring (
<<<"${cargo_info}") instead of piping into thewhile readloop is correct and deliberate — confirmed by direct testing that bash'sset -edoes abort a barevar="$(cmd)"assignment on a failing command substitution, with the failing command's own exit status, which is exactly what the adjacent comment claims about an unreachable/brokencargo -vV. - The restriction to
cargo fetch --locked --target <host>is consistent with the rest of the lane:cargo buildis never given an explicit--target, so it already builds for the host implicitly, and nothing downstream (cargo xtask e2e-prewarm/e2e) needs a different target. - Test coverage for the new host-triple and build-target logic held up under hand-tracing of several plausible single-line mutations of the new bash conditionals (inverted test, dropped
exit, wrong status) — the existing assertions already catch all of them. - The doc paragraph this change edited is evenly wrapped (72–78 characters per line, matching its neighbours), and the host-triple stub is exercised with a non-
x86_64triple (aarch64-unknown-linux-gnu) as well as the default — both already tightened within this PR's own commit history before this head, so there's nothing further to ask for on either front.
Deployment notes
- One CI job,
Security scan / Post PR findings, is red on this head, but its own log shows the cause is a transientGitHub API GET .../pulls/1057 returned HTTP 503— an infrastructure flake unrelated to this diff, not a scanner finding (the actual scanners — CodeQL, Trivy, Bandit, Zizmor, Gitleaks — all passed on this head). Worth a rerun before merging rather than treating it as a real finding. - This PR's own CI does not exercise the real
scripts/run-e2e-lane.shend-to-end on hardware: the queue lane only runs via a manualqueue-run.ymldispatch or the nightly schedule, not per-PR. The host-scoped fetch's real-world effect is validated here only by thextaskcontract tests and the author's described manual local verification; worth confirming the first real dispatch after this merges completes its fetch and build cleanly.
What this covered
Diff between base 61fdb25f7a494f566d1d64de891315673f831595 and head 2838b5a625b212bb4611444c3e80b73dde85cfe3, fetched read-only by commit SHA into a scratch repository (the PR branch was never checked out). All three changed files read in full at the head commit: scripts/run-e2e-lane.sh, docs/nightly-on-the-queue.md, and the complete LaneCase/run_lane/lane-test block of xtask/src/workflow_contract.rs (the remaining ~10,000 lines of that file are unrelated workflow-YAML contract tests, confirmed by grep that none reference the touched fields or command strings). Checked the repository's root AGENTS.md in full; none of its guardrails (module-size, architecture-doc entries, Gherkin-scenario requirement, subprocess/FFI checklist) apply to a CI-internal shell/test-only change with no new source file and no user-observable CLI behavior. Walked the file history for all three files from the head commit back to the base: this PR is exactly three commits on top of 61fdb25f (#1009); the two earlier commits in this PR already fixed an unevenly-wrapped doc paragraph and broadened the host-triple test to a second, non-x86_64 machine value, both confirmed directly by reading the current file content rather than taken on anyone's claim. Confirmed no .cargo/config.toml or other in-repo mechanism sets a build target outside the CARGO_BUILD_TARGET environment variable the new check covers. Ran two independent fan-out passes as fresh workers (one for the script+doc, one for the Rust test module), each given only the file contents and no PR description or discussion; one candidate finding from the second pass (a minor test-assertion asymmetry between two sibling tests) was checked against the source and dropped after tracing that it does not survive any realistic single-line mutation of the guarded code — the existing assertions already catch those. Checked the live CI status rollup for the head commit (39 check runs) and read the one failing job's own log directly (see Deployment notes); did not re-derive what any passing check covers from its workflow definition. Did not run cargo test/clippy/shellcheck directly (no Rust toolchain available in this environment); relied on reading the code, hand-tracing the bash control flow (including one live local bash -c experiment confirming set -e's behavior on a failing command substitution), and the passing CI jobs.
Disclosure: while performing the mandatory pre-publish check for an existing review from this same account at this head, the API response also surfaced the full body of a pre-existing review already on this PR, left at an earlier commit in this same chain. That could not be avoided given the tool's response shape. Findings above were independently re-derived and verified against the current source rather than taken from that text, and both fan-out passes were run by workers who never saw it — but the consolidation and Step-9 judgement were made by a context that had incidentally seen it, which should be weighed accordingly. Not otherwise reconciled against this PR's own review discussion.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
|
Thanks for the review of
|
juhovainio
left a comment
There was a problem hiding this comment.
Looks good. I read the full diff and found nothing blocking. The host triple comes from cargo -vV rather than being hard-coded, the no-triple case stops before any fetch, and the tests cover a non-x86_64 host. The docs are updated alongside the script.
Two small non-blocking notes, one inline and one here:
- The refusal on
CARGO_BUILD_TARGETis described as happening before anything is installed, but the test doesn't assert that the apt/curl/preflight stubs never ran. Worth adding so the claim is held by a test. - Only the env var is guarded. A
build.targetin a cargo config would still need crates the host-only fetch skips. The script comment and PR body say so, butdocs/nightly-on-the-queue.mddoesn't; a short "(environment variable only)" note would close that.
One thing I couldn't confirm from the diff: whether anything in the lane runs an unfiltered cargo metadata offline, which could now miss other-platform manifests. The first real queue-lane dispatch will show it. This is a diff-only read; no lane run was exercised.
| channels="${E2E_CHANNELS:-release nightly}" | ||
|
|
||
| # The lane fetches only the host's crates, so a build target set from outside | ||
| # would need crates it skips and meet the network without a retry. Refused |
There was a problem hiding this comment.
The comment and PR body say the refusal happens before anything is installed, but the new test doesn't assert that no apt/curl/preflight stub ran. Consider asserting it so the claim is held by a test.
| retry 4 30 cargo --version | ||
| retry 4 30 cargo fetch --locked | ||
| # Captured first, so a cargo that fails here aborts the lane with its own error. | ||
| cargo_info="$(cargo -vV)" |
There was a problem hiding this comment.
Nit: cargo -vV isn't retried and aborts the lane on failure. That's fine since cargo --version just succeeded, and it matches the comment above.
Follow-up to the review comments on #1009.
cargo fetch --lockednow passes--target <host>, with the triple read fromcargo -vV. The lane builds, pre-warms and runs the suite on the host and sets no build target, so the other platforms' crates were downloaded for nothing. A lane whose cargo reports no triple stops before the fetch.CARGO_BUILD_TARGETis set in its environment, since a build target set from outside would need crates the host-only fetch skips. Anything that exports it into the lane's container would now fail the lane; the repo sets it nowhere. A target set through a cargo config is not covered.cargo --version: it relies on rustup's proxy installing the toolchain named byrust-toolchain.tomlon first call, and is a no-op where cargo is not rustup-managed.Verified: into an empty
CARGO_HOME, a host-only fetch downloads 386 MB, andcargo metadata --offline --locked --filter-platform <host>then resolves the whole workspace; the unrestricted fetch downloads 876 MB.cargo test -p xtask workflow_contract::tests::a_lanepasses, including a new test for the no-triple case, andcargo fmt --checkis clean. Not run: a real build offline, and shellcheck (not installed here). This is CI-internal, so no Gherkin scenario applies. The real lane runs only on a manualqueue-run.ymldispatch or the nightly schedule, so the first dispatch after this merges is the real-world check.