Skip to content

docs(ci): document the three-stage validation ladder (ROCMAI-431) - #465

Merged
juhovainio merged 8 commits into
mainfrom
rocmai-431-ci-doc-ladder
Oct 15, 2026
Merged

juhovainio merged 8 commits into
mainfrom
rocmai-431-ci-doc-ladder

Conversation

@juhovainio

@juhovainio juhovainio commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

In plain English

Documentation only, no code changes. Explains the three levels of automated testing we now have (a quick check on every pull request, a deeper check every night, and a full regression check before a release), so it is clear what each one covers.

Summary

Adds the missing framing to docs/ci-hardware-testing.md:

  • A three-stage validation ladder section naming and separating the per-PR smoke gate (e2e-selfhosted.yml), the nightly coverage gate (nightly.yml, now covering every self-hosted platform plus R9700/rad3 and MI350P, ROCMAI-125/ROCMAI-429), and the release-candidate regression gate (e2e-selfhosted.yml on release/**, ROCMAI-120/EAI-8761). States plainly that cargo xtask e2e-prewarm has no --version/--build-date pin today, and that wiring a version pin into the release-branch trigger is pending work (ROCMAI-430) — this is stated once, in this section.
  • A short note on workflow_contract.rs's self_hosted_workflow_owns_the_gpu_lanes pinning the three per-PR job names by name, and its sibling ci_yml_schedules_no_self_hosted_job keeping self-hosted runs-on labels out of ci.yml (EAI-7548 rationale).
  • A one-sentence note on the consolidated report's (platform_slug, channel) column keying (ROCMAI-429).
  • An engine is not an independently selectable axis section, quoting effective_serve_engine() from capability.rs, explaining why every Strix lane is lemonade-only and e2e-gpu (MI300X) is the only per-PR vLLM lane, with an accurate description of select_serve_engine's (serve_cmd.rs) real precedence: explicit --engine → configured default → GPU-family preference (only if the recipe supports that engine) → the recipe's own preferred engine → the platform default. Also corrects "Any Windows host" to "native Windows host", since WSL reports os_family "linux" and stays vLLM-eligible.
  • A short addendum to "The shared pre-warmed runtime" cross-referencing the ladder section for the pending SDK-version-pin work, instead of re-explaining it.

No lane, artifact, or job-order changes in this PR, so every contract-tested table/list/sentence in the doc (hardware_testing_docs_cover_all_self_hosted_platforms) is left byte-identical — verified by running the full xtask suite before and after.

Rebased onto current main now that #415, #454 (adding the W7900 lane), and #463 have merged. Every factual claim above was re-verified directly against the current workflow YAML and Rust source after rebasing, including the select_serve_engine precedence and the e2e-prewarm CLI surface (xtask/src/main.rs), and all three newly-added source citations now use relative markdown links per AGENTS.md §5.

Verification

  • cargo fmt --all -- --check clean.
  • cargo test -p xtask --all-targets: 247/0/0/0, including hardware_testing_docs_cover_all_self_hosted_platforms.
  • grep -rn 'e2e-gpu|strix|mi350p|rad3' docs/ README.md outside this file: no hits (no duplicated lane claims elsewhere).

No Gherkin scenario: this PR only edits documentation prose, no product or test behavior changes.

@juhovainio
juhovainio marked this pull request as ready for review September 30, 2026 11:59
@juhovainio
juhovainio requested a review from a team as a code owner September 30, 2026 11:59
@juhovainio
juhovainio added this pull request to stack #468 September 30, 2026 12:03
@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from 46f87e6 to 930f816 Compare September 30, 2026 12:55

@tomastola tomastola left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What the change does. Docs-only: adds a "three-stage validation ladder" section to docs/ci-hardware-testing.md (per-PR, nightly, release-candidate gates), a section on why the serve engine is not a matrix axis, and notes on the (platform, channel) report columns and the e2e-prewarm pin flags. I checked the new claims against the code at this head. The workflow triggers, lane names, nightly cron and channel matrices, effective_serve_engine behaviour and report column keying are accurate. Two claims are not (below).

Coverage. Single full read of the PR's own diff (one file), checked against the workflows, xtask and capability.rs at head 930f816. This head is a rebase of the previously reviewed 46f87e6; the PR's own diff is byte-identical, and the parent commits that moved touch none of the code this doc describes. I did not read the earlier discussion before forming the findings and did not run any tests.

Needs fixing before merge

  • The e2e-prewarm paragraph (inline) describes --version/--build-date in the present tense, but that code is not in this tree, and the ladder section of the same doc says it ships in #464. Someone following the prewarm section would pass a flag that does not exist.

Non-blocking

  • The per-PR row says it covers the 4 platform jobs, but e2e-gpu-strix-ubuntu already runs a [release, nightly] channel matrix per PR (inline).
  • "cannot be silently undone" overstates what self_hosted_workflow_owns_the_gpu_lanes checks (inline).

Open decisions for the author (not defects)

  • The effective_serve_engine source is pasted verbatim into the doc. It reads well, but it drops the source's WSL comment and will drift silently when capability.rs changes. Referencing the function by name, as the rest of the doc does, would age better. Deliberate?
  • The paragraph hard-codes #415 and #464 and "as of this writing". It goes stale once either merges. A ticket reference, or a marker whoever merges them will find, is easier to keep true.

The merge decision is yours; this review is posted as a comment and files no approval or change request.

Comment thread docs/ci-hardware-testing.md Outdated
- prunes with `rocm storage remove-old-installs` after any install, update, or
repair, so the multi-version cache stays bounded.

`e2e-prewarm` also accepts mutually exclusive `--version`/`--build-date`

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flags described here do not exist yet. This paragraph says e2e-prewarm "also accepts mutually exclusive --version/--build-date flags" and that "today no caller in this repo passes either flag yet". There is no --version/--build-date handling under xtask/src at this head, and the ladder section of this same doc (lines 43-49) says they ship in #464, which is open and stacked on the also-open #415. The code and the ladder paragraph are the accurate side; this paragraph is not. Reword it as future/conditional ("#464 adds…"), drop it in favour of the single mention in the ladder section, or land this PR after #464.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 6534052 — reworded to future/conditional tense and cross-referenced to #464 (stacked on #415), matching the ladder section. No longer reads as already landed.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved at 201409b: the pre-warm paragraph now says the flags do not exist yet and points at ROCMAI-430.

Comment thread docs/ci-hardware-testing.md Outdated

| Rung | Workflow | Trigger | Lanes | Blocks a merge/tag? |
|---|---|---|---|---|
| Per-PR smoke gate | `e2e-selfhosted.yml` | `push`/`pull_request`/`merge_group` (see "Triggers") | The 4 jobs in the Platforms table below | No — absent from the required-status-check list |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per-PR row is incomplete (non-blocking). "The 4 jobs in the Platforms table" leaves out that e2e-gpu-strix-ubuntu in e2e-selfhosted.yml already has strategy.matrix.channel: [release, nightly] and uploads e2e-gpu-strix-ubuntu-${{ matrix.channel }}-report (from the ROCMAI-125 parent commit). So the per-PR gate is really 5 job runs and is not release-channel-only; the channel matrix and the two-column report (line 177) are not nightly-only. Worth stating in the row.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 6534052 — the row now states e2e-gpu-strix-ubuntu runs both its [release, nightly] channel legs per PR (ROCMAI-125).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved at 201409b: the per-PR row now states the e2e-gpu-strix-ubuntu [release, nightly] legs. (The count in that row is a new issue; see the review.)

Comment thread docs/ci-hardware-testing.md Outdated
thus their own concurrency group — means an offline runner can only ever stall
that workflow's own supersession, never `ci.yml`'s required checks. See
`EAI-7548`.
`EAI-7548`. `xtask/src/workflow_contract.rs`'s

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"cannot be silently undone" overstates the test (non-blocking). self_hosted_workflow_owns_the_gpu_lanes (xtask/src/workflow_contract.rs) only asserts sh.contains(job) for the three job keys, a substring check that any mention in a comment satisfies. It also does not cover e2e-wsl, which a separate test handles. Suggest "requires the three job keys to appear in the file".

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 6534052 — replaced with your suggested wording: the test requires the three job keys to appear in e2e-selfhosted.yml, so moving one back to ci.yml fails CI rather than merging unnoticed.

@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from 930f816 to a01b7b8 Compare September 30, 2026 16:32
@juhovainio

Copy link
Copy Markdown
Collaborator Author

Addressed the review in a01b7b8d (also rebased onto #463's latest, which landed its own review fixes):

  • e2e-prewarm tense: the --version/--build-date paragraph claimed present-tense availability, contradicting the ladder section two paragraphs up which correctly says that flag pair ships in feat(e2e-prewarm): pin the SDK version instead of always tracking latest (ROCMAI-430) #464. Reworded to future tense ("will also accept"), consistent throughout.
  • Per-PR row: now notes e2e-gpu-strix-ubuntu runs both its [release, nightly] channel legs per PR, not just "4 flat jobs."
  • "cannot be silently undone": reworded to match what self_hosted_workflow_owns_the_gpu_lanes actually checks — the three job keys appear as text in e2e-selfhosted.yml, not a full structural pin.

Left the two open-decisions-for-the-author items (the pasted effective_serve_engine source, the hardcoded #415/#464 references) untouched — non-defects per your review, and I don't have a strong opinion either way.

cargo fmt --all -- --check and cargo test -p xtask --all-targets (155 passed, including hardware_testing_docs_cover_all_self_hosted_platforms) both clean.

@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from a01b7b8 to 272a672 Compare September 30, 2026 18:10
@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from 272a672 to 6534052 Compare October 1, 2026 10:05
@juhovainio
juhovainio requested a review from tomastola October 2, 2026 08:21
@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from 6534052 to 3b59f96 Compare October 2, 2026 12:14
@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from 3b59f96 to 77d03ef Compare October 5, 2026 09:01
Base automatically changed from rocmai-125-lane-trim to main October 5, 2026 14:43
@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from 77d03ef to 6557bc8 Compare October 5, 2026 14:43

@jussielo-amd jussielo-amd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on this one — the ladder table and the effective_serve_engine writeup are exactly the kind of grounding this doc needed, and the level of cross-checking against the actual YAML/Rust (down to verifying the "suspected stale" artifact-naming paragraph was actually still correct) is the right way to write CI docs. I verified the quoted effective_serve_engine/family_prefers_vllm logic, the nightly 6-platform/channel-matrix claims, the self_hosted_workflow_owns_the_gpu_lanes test description, and the no-duplicate-lane-claims grep — all check out against current main, and cargo fmt/cargo test -p xtask --all-targets are clean.

One thing needs fixing before merge though: PR #415 (the release/** trigger) already merged on 2026-10-01, and e2e-selfhosted.yml on main already has push: branches: [main, "release/**"]. The doc currently says (twice — in the ladder section and in the pre-warm section) that "neither [#415 nor #464] is merged... e2e-selfhosted.yml on main today still triggers on push: branches: [main] only." That was true when you wrote it (Sep 30) but is now stale relative to main — the branch just needs a rebase and those two passages reworded to reflect that #415 merged and only #464 (the SDK pin) is still pending.

Everything else here looks solid.

@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from 6557bc8 to f679385 Compare October 6, 2026 11:32
@juhovainio

Copy link
Copy Markdown
Collaborator Author

Addressed the review in f6793857: reworded the two merge-status passages (ladder section + pre-warm section) now that #415 is merged — only #464 is still pending. Also rebased onto main with git rebase --onto since the upstream #463 had squash-merged (two already-upstream commits auto-dropped cleanly).

@jussielo-amd jussielo-amd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independently re-verified every factual claim in this doc against source at head (f679385): workflow_contract.rs's job-key test, e2e-selfhosted.yml's push branches, nightly.yml's cron/channel matrix across all 6 platforms, capability.rs's effective_serve_engine logic, and therock.rs's pub(crate) visibility. All check out, and #415/#464 merge-status wording is now accurate (confirmed #415 merged, #464 still open).

The staleness issue I flagged in my prior CHANGES_REQUESTED review was fixed in f679385. No remaining blocking issues — approving.

@siloteemu siloteemu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Automated review · pr-review-watcher · f679385

This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.

Review — needs work

Full review of the whole change (docs-only: docs/ci-hardware-testing.md).
The description says "documentation only" and lists five additions, and all five are present. The title cites a ticket ID, which was not retrieved.

All three findings below are new this round (first review of this PR).

Blocking

  • The engine section says the wrong code makes rocm serve pick its engine — docs/ci-hardware-testing.md:211-214
    The doc says rocm serve "picks it via effective_serve_engine() in tests/e2e-cucumber/src/capability.rs". The product selects the engine in select_serve_engine (apps/rocm/src/main.rs:5930), through preferred_serve_engine_for_host_gpu_summary. effective_serve_engine is the harness's own re-implementation. capability.rs:13-15 says it is "RE-IMPLEMENTED here … duplicating the product's select_serve_engine". The "(mirroring …)" aside reverses which side is the copy. The product governs because the harness's own header calls itself the duplicate.
    Confidence 90 · mechanical · Fix: say the harness's effective_serve_engine() predicts what rocm serve picks with no --engine.

  • "The engine is not selectable independent of hardware and OS" overstates the product — docs/ci-hardware-testing.md:210-211, :238-239
    select_serve_engine (main.rs:5936-5948) takes an explicit --engine first and a configured default_engine second. The GPU preference only applies when neither is set, and then only if the model's recipe supports that engine (main.rs:5950-5956). capability.rs:37 itself says the helper models "what rocm serve <model> would pick with no --engine". The conclusion for the lanes holds: no workflow passes an engine, and the harness skips a vLLM @requires-engine pin on hosts that cannot run it. The stated reason is wrong, and "Adding a vLLM lane means adding hardware" is only true of the harness's default path. The code governs.
    Confidence 85 · logic · Fix: narrow the claim to the E2E harness's default engine, and mention the explicit/configured override and the @requires-engine pins.

  • Doc states the open status of PR #464 and will go false once it merges — docs/ci-hardware-testing.md:44-56, :349-354
    "still pending as of this writing", "will also accept", "what every lane in this tree runs today" and "not part of either PR" are all true only while #464 is open. At head, xtask/src/main.rs:219-231 has no --version/--build-date. The PR's own third commit had to reword the #415 status for the same reason. The same status is also written twice (lines 46-56 and 349-354), so one merge makes two passages stale. No test constrains this prose. The other time-bound statement in this doc is dated ("confirmed 2026-09-11"), and AGENTS.md §4 says docs should not carry scratch or planning state.
    Confidence 80 · logic · Fix: describe only what the tree supports today ("e2e-prewarm has no version pin"), or date the statement and drop one of the two copies. Keep the PR dependency in the PR body.

Non-blocking

  • The effective_serve_engine code block is presented as a quote but is not exact — docs/ci-hardware-testing.md:216-227. It drops the two comment lines at capability.rs:47-48. The block is also an unguarded copy: no xtask test reads code blocks in this doc. capability.rs:44-45 already plans to replace this function, so the copy is likely to go stale. The prose at lines 229-231 already carries the rule. Confidence 75 · mechanical · Fix: drop the block, or mark it as abbreviated.
  • "The 4 jobs in the Platforms table" is ambiguous — :36, :38. That table has 5 rows including the mock e2e job from ci.yml. The four are the e2e-selfhosted.yml jobs. Confidence 75 · mechanical · Fix: "the 4 e2e-selfhosted.yml jobs".
  • self_hosted_workflow_owns_the_gpu_lanes is described more strongly than it checks — :22-26. xtask/src/workflow_contract.rs:755-773 checks that the strings e2e-gpu: etc. appear in e2e-selfhosted.yml, not that they are job keys. A lane moved back into ci.yml would still fail it. Confidence 70 · mechanical · Fix: "must appear in e2e-selfhosted.yml".
  • New text repeats existing text in the same doc — :188-191 (existing :197-203 already says release and nightly runs get separate columns), :36 (repeats :74-79), :40-42 (repeats :184-185). Each is a maintenance cost, not a defect. Confidence 70 · mechanical · Fix: keep one copy of each.
  • Stray line wrap — :51-53 ("not part of either\nPR: nothing in this\nrepo maps") looks like an editing leftover. Confidence 70 · mechanical · Fix: reflow.

Decisions for the author

  • A ladder table with job names, cron time and lane counts that no test guards — tradeoff
    The facts check out against the workflows at head, including the 06:00 UTC cron, the e2e-gpu-nightly-rad3/-mi350p names, the channel matrices, the release/** push trigger and the (platform_slug, channel) column keying. The doc-contract test parses only the | Job | Workflow | Platform | Runner labels | table, so none of this will fail CI if it drifts. One side: this is the framing the doc was missing. The other: it adds drift-prone counts. Confirm the maintenance cost is acceptable.
  • "None are required-status checks" — tradeoff
    Branch protection is not visible from the repo, so this could not be verified here. It is consistent with the header of e2e-selfhosted.yml and the existing "Blocking vs. non-blocking" section. Confirm it is still true.

Positive signals

  • The doc states plainly that no sdk_version: [current, n-1, n-2] matrix exists and why. That avoids implying a mechanism that is not there.
  • Every ladder-table claim about triggers, lanes and channels matched the workflows at head.

Deployment notes

None

What this covered

Every added line of docs/ci-hardware-testing.md and the doc in full, against prw-base...f6793857 (3 commits, one file). Claims were checked against .github/workflows/{e2e-selfhosted,nightly,release}.yml, xtask/src/{workflow_contract,main,e2e_prewarm}.rs, tests/e2e-cucumber/src/capability.rs, apps/rocm/src/main.rs, crates/rocm-core/src/lib.rs, apps/rocm/src/therock.rs and crates/e2e-report/src/consolidated.rs. Three read-only workers covered the code-claims, engine/prewarm claims and the instruction/history/comment passes. The engine-selection and capability.rs findings were re-checked against source. Nothing was built or run. Impact analysis was a docs-only grep of other docs and the xtask doc-contract tests.

  • Checks at this head: all reported checks succeeded. These did not report (skipped, so they gave no signal): E2E tests (MI300X), E2E tests (Strix Halo, Ubuntu), E2E tests (Strix Halo, Windows), E2E tests (Strix Halo, WSL2), E2E consolidated report, E2E consolidated report (self-hosted), Skill checks (skillscope), Sphinx docs build (-W), Build Linux PR binaries and remote control channel (containerised). The Sphinx -W docs build did not run, so docs build warnings from this file are unchecked.
  • Did not run: the prior-changes pass (earlier reviews are not read, by rule). Ticket references and the branch-protection claim could not be verified from the repo.
  • Not reconciled against any discussion here.

@juhovainio

Copy link
Copy Markdown
Collaborator Author

Addressed the review in 04234c79: the engine section had it backwards (effective_serve_engine() is the harness's re-implementation that predicts what select_serve_engine picks, not the other way round) and overstated the lack of an override (explicit --engine/configured default_engine win before the GPU-family preference, and @requires-engine pins around it). Reworded the #464 merge-status prose in both spots to state what the tree supports today instead of PR status, so it won't go stale when #464 merges.

@siloteemu siloteemu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Automated review · pr-review-watcher · 04234c7

This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.

Review — needs work

Full review of the whole change, 4 commits, one file: docs/ci-hardware-testing.md. The doc was also checked against the current base tip (prw-base), which is many commits ahead of the merge-base, because it lands on top of that.
The PR description says it is stacked on #463 ("Depends on #463"). It also cites ROCMAI-431, ROCMAI-125, ROCMAI-429, ROCMAI-430, ROCMAI-120 and EAI-8761. None of those tickets could be retrieved from here.

Round 2. Each finding is marked new (first raised this round) or standing (raised in the round at f679385 and still holding). Of the three blockers filed at f679385, two no longer hold and are dropped: the engine-selector direction and the "not selectable independent of hardware" overstatement. Neither was re-found by this round's blind review, and each was then put to the reviewer, which found the reworded engine section resolves it. The third (status prose) still holds and is carried below as standing.

Blocking

  • [new] The PR description no longer matches the doc — PR body vs docs/ci-hardware-testing.md:44-56, :355-359, :219-230

    • The body says the ladder section "States plainly that the release-candidate rung's trigger ships in PR #415 and its SDK-version pin flags in PR #464, both stacked but not yet merged".
    • The doc now says the opposite for #415 ("merged in PR #415, so e2e-selfhosted.yml on main already triggers on…") and does not mention #464 at all.
    • The body says it quotes effective_serve_engine() "verbatim". The doc's block drops the two comment lines inside the function (capability.rs: "The vLLM adapter bails on native Windows (WSL builds as Linux, …)"), so it is not verbatim.
    • The body also says the addendum describes "e2e-prewarm's --version/--build-date pin flags". The addendum (:355-359) says those flags do not exist at HEAD and that another ticket "adds" them.
    • The description claims things that are not in the change. Remedy: rewrite the description to match the doc as it now stands.
    • Confidence 85 · logic · Fix: update the body.
  • [new] The doc contradicts itself about whether --version/--build-date exist — docs/ci-hardware-testing.md:53-56 vs :46-47, :355

    • :53-56 says "the release gate's SDK version is whatever --version/--build-date its caller passes".
    • :46-47 and :355 say e2e-prewarm has no such flags.
    • The tree supports the second reading. git grep -n "build-date\|build_date" prw-base -- xtask returns nothing. The E2ePrewarm clap enum has only channel, keep and prewarm_dir.
    • Every lane calls e2e-prewarm --channel … --prewarm-dir … with no pin. On a release/** push the SDK is therefore whatever the channel index currently serves. No caller exists that could pass a pin.
    • The "whatever its caller passes" clause governs nothing: it describes a future state in the present tense.
    • Confidence 85 · logic · Fix: say the release gate currently tracks the channel index's latest build. Describe the pin as future work, or drop that clause.
  • [new] A new source citation points at the wrong file on the current base, and the new citations break an AGENTS.md rule — docs/ci-hardware-testing.md:213, :22-23, :215

    • Line 213 says select_serve_engine is in apps/rocm/src/main.rs. That is true at the PR head (main.rs:5930).
    • On prw-base it is not: git grep -n "fn select_serve_engine" prw-base -- apps/ gives only apps/rocm/src/serve_cmd.rs:62. main.rs just re-imports it (use crate::serve_cmd::{…, select_serve_engine, …}).
    • prw-base:AGENTS.md:182-199 (§5) says: "cite a specific repo file in markdown as a link relative to the citing file — … not as a bare backtick path: the docs-links CI job and the lychee prek hook fail when a link's target is moved or deleted, but cannot see a bare path go stale." It covers "citations you add or edit". docs/ci-hardware-testing.md is not in its exclusion list.
    • The PR adds bare backtick citations of xtask/src/workflow_contract.rs (:22-23), apps/rocm/src/main.rs (:213) and tests/e2e-cucumber/src/capability.rs (:215). The stale :213 path is the failure the rule exists to prevent.
    • The base's rule governs, because the doc lands on top of it.
    • Confidence 90 · mechanical · Fix: use relative links such as [serve_cmd.rs](../apps/rocm/src/serve_cmd.rs), name the symbol rather than a line, and point at serve_cmd.rs.
  • [standing] The doc states the open status of the SDK-pin work and will go false once it lands — docs/ci-hardware-testing.md:44-56, :355-359

    • This was raised at f679385 against wording that named PR #464. The rewording replaced the PR number with a ticket ID and "at HEAD", but the passages are still status: "ROCMAI-430 adds mutually exclusive flags for that" (:46-48) and "ROCMAI-430 adds …" (:357) describe, in the present tense, code that is not in the tree. "a separate, not-yet-started step" (:51) and "what every lane in this tree runs today" (:356) are also true only until that work merges.
    • The same fact is still written twice (:46-50 and :355-359), each copy pointing at the other, so one merge makes both stale.
    • The "no pin" clause on its own does describe the tree today and is fine.
    • This round's blind review did not re-find it as worded; it was settled by putting the earlier objection to the reviewer after its report was returned, which found that it still holds. The blind review independently filed the sharpest instance (:53-56) above and the status wording as a non-blocking item below.
    • Confidence 80 · logic · Fix: describe only what the tree supports today ("e2e-prewarm has no version pin; release-branch runs track the channel index"), state it once, and keep plans for the pin work in the PR or issue that adds it.

Non-blocking

  • [standing] The workflow_contract paragraph overstates the test — docs/ci-hardware-testing.md:22-26

    • The test at prw-base:xtask/src/workflow_contract.rs:755 requires only that the raw text of e2e-selfhosted.yml contains e2e-gpu:, e2e-gpu-strix-ubuntu: and e2e-gpu-strix-windows:. It does not parse job keys.
    • The guard that fails a PR moving a lane back into ci.yml is the separate test at :735, ci_yml_schedules_no_self_hosted_job. It rejects self-hosted labels in ci.yml's runs-on. The doc does not name it.
    • Confidence 70 · logic · Fix: name both tests, or say "contains".
  • [standing] Hard-coded lane counts, and one is ambiguous — :36, :37, :38

    • "The 4 jobs in the Platforms table" is wrong against that table, which has 5 rows (e2e is the GitHub-hosted mock job). Four is right only for the self-hosted jobs.
    • Nothing guards these counts, unlike the lane lists in hardware_testing_docs_cover_all_self_hosted_platforms.
    • Confidence 70 · mechanical · Fix: say "the four self-hosted jobs" or name them.
  • [new] The engine section understates how rocm serve chooses an engine — :210-217, :232-245

    • The doc says explicit and configured overrides come first, then "falls back to the GPU-family preference". The product (serve_cmd.rs:62-106 on prw-base) also requires the recipe to support the preferred engine, then falls through to the recipe's first preferred engine, then the platform default.
    • "Any Windows host resolves to lemonade" should say native Windows. The source comment the doc dropped says WSL stays vLLM-eligible.
    • The harness's normalize_family is a subset of the product's normalize_therock_family. For example it does not map gfx125* or the generic gfx90* prefix to the -dcgpu families. The advice "add hardware from a vLLM-eligible family" is reliable only for the families the harness maps.
    • Confidence 70 · logic · Fix: soften the "what rocm serve would pick" framing and say native Windows. Link capability.rs instead of restating its rule.
  • [standing] The pasted effective_serve_engine block — :219-230

    • It is copied code that already exists in the repo. The block calls family_prefers_vllm, which is not shown, and nothing keeps the copy in sync. Only link resolution is checked in this repo, not snippet content.
    • Confidence 80 · mechanical · Fix: link the symbol and keep the prose rule.
  • [new] Status and time-capsule prose — :44, :47, :51, :355-359

    • "merged in PR #415", "at HEAD", "not-yet-started" and "ROCMAI-430 adds…" are status wording that goes stale as soon as the pin flags land.
    • Confidence 78 · mechanical · Fix: describe the current unpinned behaviour only. Document the flags in the change that adds them.
  • [standing] Repetition — :46-50 vs :355-359

    • The "no pin at HEAD / ROCMAI-430" fact is stated twice, with cross-references between the copies. The R9700/MI350P demotion also appears at :40-42 and again at :184-185.
    • Confidence 65 · mechanical · Fix: state each once and link.
  • [new] Nightly lane names disagree within the doc — :37 vs :184, :193-199

    • The new ladder table uses e2e-gpu-nightly-rad3 and e2e-gpu-nightly-mi350p. These are the job keys in nightly.yml (:490 and :588 at HEAD).
    • Pre-existing wording at :184 says nightly "also runs e2e-gpu-rad3 and e2e-gpu-mi350p". Those are artifact and slug names, not job keys. The PR edits that same paragraph without reconciling the two.
    • The table's names govern, because they match the workflow job keys.
    • Confidence 70 · mechanical · Fix: reword :184. This is a pre-existing wording connected by pattern.
  • [new] Commit structure — 83a1ce1, 498b180, f679385, 04234c7

    • Three follow-up commits re-edit passages from the first, and f679385 reverses 498b180's framing. This is review-fixup noise on a single logical change.
    • 83a1ce1's body says it adds "the validation axes (OS, hardware, channel, SDK-version)", but the doc has no such section.
    • Confidence 70 · mechanical · Fix: squash, or reword the message.

Decisions for the author

  • Copy the rule into the doc, or link it — tradeoff

    • A pasted snippet reads well at a glance. A link cannot drift silently and matches the base's citation rule. Moving to a link is a call for the author to make.
  • Stacked on #463 — tradeoff

    • The description says this needs #463 merged first so the doc matches that PR's lane changes. The PR is not a draft. I did not check whether #463 has landed. Confirm the dependency is satisfied or mark the PR draft.

Positive signals

  • Almost every checkable claim in the added text is accurate:
    • the report's (platform_slug, channel) column keying (crates/e2e-report/src/consolidated.rs:527-541);
    • the nightly cron 0 6 * * *;
    • the [release, nightly] matrix on the nightly lanes and on e2e-gpu-strix-ubuntu;
    • the release/** push trigger;
    • the absence of any n-1/n-2 mapping;
    • the private therock.rs index parsers;
    • the @requires-engine:<vllm|lemonade> spelling;
    • the vLLM-preferred family list.
  • No lane, artifact or job-order text that hardware_testing_docs_cover_all_self_hosted_platforms keys on was touched.
  • Docs-only scope is stated in the PR body, so no Gherkin scenario is owed.

Deployment notes

None

What this covered

  • Read: the whole of docs/ci-hardware-testing.md at the PR head, and the full diff prw-base…HEAD (4ee61616…04234c79). Cited claims were checked against code at both the PR head and prw-base (34fbdfe). A three-way merge of this file against prw-base was reported conflict-free by a worker.
  • Fan-out: three read-only workers: ladder and pre-warm claims; engine section and the harness against the product; and the source-reading passes (AGENTS.md adherence, history, comments, doc quality, commit messages). Each ran to completion. Their results were then re-checked directly, namely:
    • the serve_cmd.rs:62 location;
    • AGENTS.md §5 at prw-base;
    • the two workflow_contract.rs tests;
    • the dropped comment lines in capability.rs;
    • the absence of any build-date flag in xtask;
    • the PR description.
  • Not run: history and prior-changes passes ran only against file history. No prior reviews were read during the review, by design. No cargo build or tests were run (docs only, so the cargo gates are not exercised by this change).
  • Standing objections: settled after the blind review was returned, by putting each one to the reviewer with the author's reply. Those verdicts are anchored evidence (the reviewer was told what to look for), and they did not add, drop or change any finding above.
  • Checks that could not be made:
    • The cited ticket IDs were not retrieved.
    • The branch-protection required-check list lives in GitHub settings. The "none are required" claim rests only on the dated comment at e2e-selfhosted.yml:11-17.
  • CI: all reported checks passed on this head. These had not reported: E2E Strix Halo Ubuntu, E2E Strix Halo Windows, E2E Strix Halo WSL2, E2E MI300X, E2E consolidated reports, Sphinx docs build, Skill checks, remote control channel and Build Linux PR binaries. For a docs-only change the Sphinx docs build is the one whose absence matters. That class is not evidenced here.
  • Not reconciled against the review discussion beyond the standing objections above.

@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from 04234c7 to a7f1121 Compare October 13, 2026 11:14
@juhovainio

Copy link
Copy Markdown
Collaborator Author

Addressed the review (all three blocking findings, across b4ec502e, 8c428ca1, 1873322e, and the final rebase a7f11217):

  1. Wrong code attribution for engine selection: the doc now cites select_serve_engine in serve_cmd.rs as the product original, with effective_serve_engine() described as the harness's own re-implementation predicting what rocm serve picks with no --engine.
  2. Overstated "not selectable independent of hardware/OS" claim: reworded to describe select_serve_engine's full precedence (explicit --engine, configured default_engine, recipe-support gate, then GPU-family preference), and the @requires-engine override.
  3. Stale PR ci(e2e): auto-trigger self-hosted E2E matrix on release-branch push (EAI-8761) #415/feat(e2e-prewarm): pin the SDK version instead of always tracking latest (ROCMAI-430) #464 merge-status prose (two copies, would go false on merge): ci(e2e): auto-trigger self-hosted E2E matrix on release-branch push (EAI-8761) #415 merged 2026-10-01 and is now rebased onto main (which already has ci(e2e): auto-trigger self-hosted E2E matrix on release-branch push (EAI-8761) #415/ci(e2e): channel-aware report columns and nightly matrix axis (ROCMAI-429) #460/ci(e2e): trim per-PR GPU lanes and add nightly-channel leg to strix-ubuntu (ROCMAI-125) #463). Reworded to state what the tree supports today, with the --version/--build-date pin noted once in the ladder section as pending (ROCMAI-430) and cross-referenced rather than repeated.

Also fixed in the final pass: the bare-backtick-path citations are now relative markdown links per AGENTS.md §5, and the "native Windows" vs. WSL lemonade-only engine case is corrected (WSL reports os_family linux and stays vLLM-eligible).

Non-blocking items (ambiguous "4 jobs" phrasing, repeated text, stray line wrap, abbreviated code-block quote) were also cleaned up in the same passes.

cargo test -p xtask workflow_contract clean (56/56), including hardware_testing_docs_cover_all_self_hosted_platforms and self_hosted_workflow_owns_the_gpu_lanes.

@juhovainio
juhovainio dismissed stale reviews from siloteemu and siloteemu October 13, 2026 13:52

Superseded by round-2 review and the subsequent fix commits; round-1 blockers (engine-selector direction, overstated hardware/OS claim) confirmed resolved in round 2.

@juhovainio

Copy link
Copy Markdown
Collaborator Author

Round 2 (review 5488244550) addressed in a7f11217:

Also rebased onto current main now that #415/#460/#463 have merged. cargo test -p xtask workflow_contract and cargo fmt --all -- --check both clean.

Round 1 (5429469170) and round 2 (5488244550) reviews dismissed as resolved; re-requesting review.

@juhovainio
juhovainio requested a review from siloteemu October 13, 2026 13:54

@tomastola tomastola left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Automated review · pr-review-watcher · a7f1121

This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.

Follow-up review — one blocking item, one-line fix

Re-review of the new commits since my 2026-09-30 review (full review of the whole change, one docs file). Filed as a comment rather than a change request: the item below is a link-style fix with no effect on what the doc says.

Earlier findings (my 2026-09-30 review)

  • Resolved: the e2e-prewarm paragraph described --version/--build-date as existing; it now says they do not exist yet and points at ROCMAI-430.
  • Resolved: the per-PR row now includes the [release, nightly] channel legs of e2e-gpu-strix-ubuntu.
  • Still stands (non-blocking, reworded): "cannot be silently undone" became "requires the job keys, so moving a lane back into ci.yml fails CI". See the first non-blocking item below.
  • Resolved in spirit: the pasted effective_serve_engine source was flagged as a decision; it is still pasted, see the second non-blocking item.

Blocking

  • (new) A newly added citation of a tracked source file is a bare backtick, against the AGENTS.md citation rule — docs/ci-hardware-testing.md:52 (`therock.rs`'s index-version parsers are private)
    AGENTS.md §5 says to cite a specific repo file "as a link relative to the citing file … not as a bare backtick path", because docs-links cannot see a bare path go stale. The change itself applies the rule to workflow_contract.rs, serve_cmd.rs and capability.rs. The therock.rs mention is a tracked file and is not in any of the rule's exemptions. apps/rocm/src/therock.rs holds the private parse_simple_index_versions; tests/e2e-cucumber/tests/e2e/therock_steps.rs is a different file, so the bare name is also ambiguous.
    Confidence 85 · mechanical · Fix: link it as [therock.rs](../apps/rocm/src/therock.rs), and name parse_simple_index_versions.

Non-blocking

  • (standing) The guard test is described more strongly than it behaves — docs/ci-hardware-testing.md:22-26
    The text says the test requires the lanes "to appear as job keys … so a PR that moves one of these lanes back into ci.yml fails CI". self_hosted_workflow_owns_the_gpu_lanes (xtask/src/workflow_contract.rs:755-773) only does sh.contains("e2e-gpu:") on the raw file text. A comment or needs: mention satisfies it, and a duplicate lane left in ci.yml is not checked. Deleting the job from e2e-selfhosted.yml does fail. It also does not cover e2e-wsl.
    Confidence 70 · mechanical · Fix: say "must contain the job names as text", or leave the claim out.
  • (standing) The effective_serve_engine function body is copied into the doc — docs/ci-hardware-testing.md:228-238
    It matches the source today (tests/e2e-cucumber/src/capability.rs:46-57), but a verbatim copy goes stale silently. The prose already describes the rule and the function is linked.
    Confidence 70 · mechanical · Fix: drop the block and keep the prose and the link.

Decisions for the author

  • The pending-work note (ROCMAI-430, the pin and the sdk_version matrix) appears twice — tradeoff
    It is stated in full in the ladder section and again, shortened, in the pre-warm section. Repetition is a drift risk, but the pre-warm reader benefits from seeing it where the unpinned command lives. Confirm it is deliberate.
  • The doc records PR numbers and merge state ("merged in PR #415") — tradeoff
    Accurate now, and the wording avoids "pending PR" claims, but it will age. Naming the trigger by what the workflow contains would be more durable.

Positive signals

  • The claim list was checked against the tree and held up: the nightly schedule, the release/** push trigger, the (slug, channel) report column identity (crates/e2e-report/src/consolidated.rs:530-541), the Strix/WSL gfx1151 lemonade-only behaviour, and the select_serve_engine precedence including the recipe-support gate.
  • The select_serve_engine description is now accurate and no longer collapses the explicit override and configured default.

Deployment notes

None

What this covered

Every changed line of docs/ci-hardware-testing.md, read against prw-base...a7f1121775aa7f5b5e162b65f66f0f2e3e1e10a3, with each factual claim cross-checked against the workflows, xtask, e2e-report, serve_cmd.rs, therock.rs, capability.rs, host_gpu.rs and AGENTS.md §5. I re-confirmed the blocking item and the first non-blocking item in the checkout before posting. CI on this head is green apart from the self-hosted lanes, which were skipped on this PR (statusCheckRollup).

Did not run: tests and the link check (docs-only); the PR description and ROCMAI-431 were not retrievable, so claims-versus-change was checked against commit messages only. Not reconciled against any discussion here, and I did not read the author's replies when forming the findings.

@juhovainio
juhovainio requested a review from tomastola October 13, 2026 18:31
@juhovainio

Copy link
Copy Markdown
Collaborator Author

Fixed in 1de88fe: linked the therock.rs citation at docs/ci-hardware-testing.md:52 as [therock.rs](../apps/rocm/src/therock.rs) and named parse_simple_index_versions, per AGENTS.md §5. (Review 5507143696 was filed as COMMENTED, so it's not dismissable via the GitHub API — re-requested review from @tomastola instead.)

@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from 1de88fe to 201409b Compare October 14, 2026 11:06

@tomastola tomastola left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 201409b (last review a7f1121). One docs file. Read in full against the workflows, xtask, e2e-report, serve_cmd.rs, capability.rs and AGENTS.md. Not reconciled against any discussion.

Earlier points

  • Resolved: the e2e-prewarm tense; the per-PR row now mentions the Strix [release, nightly] legs; the bare therock.rs citation is now a link (201409b).
  • Still open, now blocking: the guard-test claim (inline at :26).
  • Still open, non-blocking: the pasted effective_serve_engine body (:228-239).

Blocking

  • Lane counts will be wrong once this lands — docs/ci-hardware-testing.md:36-38 (inline at :36). Merge-base is 2844d79; origin/main already has e2e-gpu-w7900 and e2e-gpu-nightly-w7900 (#454) and its copy of this doc has W7900 rows. The merge is textually clean, so the ladder would say "4" / "the same 4 platforms" against a Platforms table with more. The workflows on main govern. Fix: rebase, and drop the counts ("the self-hosted jobs in the Platforms table"). Even at this head, "4 jobs in the Platforms table" is off, since the table has 5 rows including the ci.yml mock lane. hardware_testing_docs_cover_all_self_hosted_platforms pins that table, so keep it green when rebasing.
  • The description misstates the change. It says the no-pin fact is "stated once, in this section" and that the pre-warm addendum cross-references rather than re-explains; the doc states the no-pin fact and ROCMAI-430 at :46-52 and again at :365-368. It also says "rebased onto current main" (it is not) and "6 platforms" (7 on main). Either rebase and trim the duplicate, or narrow the description.

Non-blocking

  • The artifact-name paragraph (:199-212) says only nightly.yml lanes append the channel, and :186-189 says two columns per platform only for nightly. Per-PR e2e-gpu-strix-ubuntu also uploads e2e-gpu-strix-ubuntu-${{ matrix.channel }}-report (e2e-selfhosted.yml:410-411, :644), which contradicts the new ladder text at :36. "Silently overwriting" is also loose: before ROCMAI-429 the runs merged into one column (consolidated.rs:531-534).
  • @requires-engine skips only when the scenario is also @requires-gpu (expectation.rs:602); :253-255 omits that condition.
  • The doc presents the harness's engine prediction as the product's rule. The product maps gfx125* and other gfx90* targets to -dcgpu (vLLM); the harness normalize_family does not (host_gpu.rs:107-146, capability.rs:75-97). No current lane is affected.
  • :182 says e2e-gpu-rad3 / e2e-gpu-mi350p; the job keys are e2e-gpu-nightly-rad3 / e2e-gpu-nightly-mi350p. :333-337 "every other lane is release-only" is wrong for nightly.yml, which passes --channel ${{ matrix.channel }} on all six lanes.
  • ROCMAI-431 asks for the @requires-engine:vllm "silently vanish" trap to be explicit, and for each axis (OS, HW, channel) to be mapped to its stage; both are only implied.
  • Time-relative prose ("merged in PR #415", "today", "at HEAD"), the 06:00 UTC cron and the pasted snippet will go stale; state the invariant once and link.

Positive signals

  • Every added relative link resolves and follows AGENTS.md §5.
  • The select_serve_engine precedence and the Strix / MI300X conclusions match the code.

CI at this head: everything that ran is green; the self-hosted GPU lanes, Sphinx and skillscope were skipped.

Comment thread docs/ci-hardware-testing.md Outdated
`self_hosted_workflow_owns_the_gpu_lanes` test requires `e2e-gpu`,
`e2e-gpu-strix-ubuntu`, and `e2e-gpu-strix-windows` to appear as job keys in
`e2e-selfhosted.yml`, so a PR that moves one of these lanes back into
`ci.yml` fails CI rather than merging unnoticed.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: this credits the wrong test. self_hosted_workflow_owns_the_gpu_lanes does sh.contains("e2e-gpu:") and the two sibling strings on the raw text of e2e-selfhosted.yml (workflow_contract.rs:755-774). It never reads ci.yml, so a lane copied into ci.yml with the original left in place still passes, and a comment mentioning the name satisfies it. The code governs. Say what it checks (the three job names appear in e2e-selfhosted.yml), and credit ci_yml_schedules_no_self_hosted_job for keeping self-hosted runs-on labels out of ci.yml.

Comment thread docs/ci-hardware-testing.md Outdated

| Rung | Workflow | Trigger | Lanes | Blocks a merge/tag? |
|---|---|---|---|---|
| Per-PR smoke gate | `e2e-selfhosted.yml` | `push`/`pull_request`/`merge_group` (see "Triggers") | The 4 jobs in the Platforms table below (`e2e-gpu-strix-ubuntu` runs both its `[release, nightly]` channel legs per PR, ROCMAI-125) | No — absent from the required-status-check list |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: "4" will be wrong after rebase. origin/main already has e2e-gpu-w7900 (#454) and its nightly counterpart; this branch's merge-base predates them, so the merge is clean but the counts here and in the next two rows ("the same 4 platforms", "the same 4 per-PR lanes") would miscount. Rebase and drop the numbers. At this head the Platforms table already has 5 rows, so "4 jobs in the Platforms table" is only right for the e2e-selfhosted.yml rows.

Add the missing framing to docs/ci-hardware-testing.md: the three
self-hosted GPU gates (per-PR smoke, nightly coverage, release-candidate
regression) and how they differ; the validation axes (OS, hardware,
channel, SDK-version); why the serve engine isn't an independently
selectable matrix axis (quotes effective_serve_engine()); and a short
addendum on e2e-prewarm's --version/--build-date pin flags. No lane,
artifact, or job-order changes, so every contract-tested table/list/
sentence in the doc is left byte-identical.

ROCMAI-431

Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Review found the --version/--build-date paragraph claiming present-tense
availability for a flag pair that isn't in this tree yet (ships in #464,
stacked on #415, neither merged) — contradicted the ladder section's own
"ships in #464" framing two paragraphs up.

- e2e-prewarm --version/--build-date: reworded to future tense, matching
  the ladder section.
- Per-PR row: note e2e-gpu-strix-ubuntu already runs both channel legs
  per PR, not just 4 flat jobs.
- "cannot be silently undone": reworded to match what
  self_hosted_workflow_owns_the_gpu_lanes actually checks (job keys
  present as text, not a structural pin).

Left the two open-decisions-for-the-author items untouched per review
(non-defects).

Signed-off-by: Juho Vainio <juho.vainio@amd.com>
PR #415's release/** trigger merged 2026-10-01; e2e-selfhosted.yml on
main already triggers on push: branches: [main, "release/**"]. Only
#464 (the SDK pin) is still pending. Fixes the two passages in the
ladder section and the pre-warm section that still said neither PR
was merged.

Signed-off-by: Juho Vainio <juho.vainio@amd.com>
…rose

pr-review-watcher flagged 3 blocking issues at f679385:
- the engine section said rocm serve picks its engine via the harness's
  effective_serve_engine(), when capability.rs's own header says that
  function is the re-implementation and select_serve_engine (main.rs)
  is the product original.
- "the engine is not selectable independent of hardware and OS"
  overstated it: select_serve_engine checks an explicit --engine and a
  configured default_engine before falling back to the GPU-family
  preference, and a scenario can pin either side with
  @requires-engine:<vllm|lemonade>.
- the #464 merge-status prose (two copies) asserted "still pending as
  of this writing", which goes false the moment #464 merges. Reworded
  both to state what the tree supports today instead of PR status.

Signed-off-by: Juho Vainio <juho.vainio@amd.com>
- rebase onto main (PR #415/#460/#463 already merged there)
- cite select_serve_engine's real location (serve_cmd.rs, not main.rs) and
  turn the three new bare-backtick file citations into relative markdown
  links per AGENTS.md's citation rule
- drop the sentence implying e2e-prewarm already has --version/--build-date
  flags; state once (in the ladder section) that the pin is pending work
  (ROCMAI-430) and cross-reference it from the pre-warm addendum instead of
  repeating it
- describe select_serve_engine's full precedence (recipe-support gate,
  recipe's own preferred engine, platform default), not just the two-step
  simplification
- say 'native Windows' for the lemonade-only engine case; WSL reports
  os_family linux and stays vLLM-eligible

Signed-off-by: Juho Vainio <juho.vainio@amd.com>
A bare backtick path doesn't get checked by docs-links, so it can go
stale silently. Link it relative to this file, and name the actual
private function instead of just the file.

Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Rebase picks up #454's W7900 lane (PR #415/#454/#463 all now merged).
Drop the hardcoded '4 jobs'/'4 platforms' counts from the validation
ladder table so they can't drift again as lanes are added, and credit
ci_yml_schedules_no_self_hosted_job alongside
self_hosted_workflow_owns_the_gpu_lanes for what each test actually
checks.

Signed-off-by: Juho Vainio <juho.vainio@amd.com>
@juhovainio
juhovainio force-pushed the rocmai-431-ci-doc-ladder branch from 201409b to a167000 Compare October 15, 2026 10:28
@juhovainio
juhovainio dismissed tomastola’s stale review October 15, 2026 10:29

Rebased onto main (picks up #454's W7900 lane). Fixed: lines 22-26 now credit both ci_yml_schedules_no_self_hosted_job (keeps self-hosted runs-on out of ci.yml) and self_hosted_workflow_owns_the_gpu_lanes (checks the three job names appear in e2e-selfhosted.yml). Dropped the hardcoded '4' counts from the ladder table (now 'the self-hosted jobs in the Platforms table below' etc). PR description's nightly-platform count and rebase note updated to match. cargo fmt/clippy/test all clean.

The pre-warm section restated the no-pin/ROCMAI-430 fact that the
validation-ladder section already explains, contradicting the PR's own
claim that it's stated once. Cross-reference instead of repeating it.

Signed-off-by: Juho Vainio <juho.vainio@amd.com>
@juhovainio
juhovainio requested a review from tomastola October 15, 2026 10:34
@juhovainio

Copy link
Copy Markdown
Collaborator Author

Addressed both blocking findings from the latest review (201409b), across a167000a and 2ff02f4d:

  1. Wrong test credited for keeping self-hosted lanes out of ci.yml: the doc now says what self_hosted_workflow_owns_the_gpu_lanes actually checks (the three per-PR job names appear in e2e-selfhosted.yml), and credits ci_yml_schedules_no_self_hosted_job for the separate guarantee that self-hosted runs-on labels stay out of ci.yml.
  2. Stale lane/platform counts after rebase: rebased to pick up ci(e2e): add a W7900 self-hosted E2E lane (EAI-8763) #454's W7900 lane (now on main via ci(e2e): auto-trigger self-hosted E2E matrix on release-branch push (EAI-8761) #415/ci(e2e): add a W7900 self-hosted E2E lane (EAI-8763) #454/ci(e2e): trim per-PR GPU lanes and add nightly-channel leg to strix-ubuntu (ROCMAI-125) #463). Dropped the hardcoded '4 jobs'/'4 platforms'/'4 per-PR lanes' counts from the validation-ladder table entirely so they can't drift again as lanes are added, instead of re-counting to a number that's already wrong at this head (5 rows in the Platforms table).

Also trimmed the duplicated no-pin/ROCMAI-430 explanation (stated once in the ladder section, the pre-warm addendum now just cross-references it instead of re-stating it) and brought the PR description back in sync with the actual rebase state.

cargo fmt --all -- --check clean, cargo test -p xtask --all-targets: 261/0/0/0 including hardware_testing_docs_cover_all_self_hosted_platforms.

@juhovainio
juhovainio added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit ea147d0 Oct 15, 2026
31 checks passed
@juhovainio
juhovainio deleted the rocmai-431-ci-doc-ladder branch October 15, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants