Repository navigation
dedup: share HTML page shell and platform sort key in e2e-report - #585
Conversation
rominf
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 13cdfd7
This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.
Review — needs work
Full review of the whole change (one commit, three files in crates/e2e-report).
The description puts the change in the ROCMAI-200 quality-findings backlog, which came out of the ROCMAI-27 review. Neither ticket could be retrieved from here because no tracker tool was available, so this review does not judge the change against either ticket.
Blocking
-
[new] Required E2E lanes are not green at the reviewed head — PR #585 checks at
13cdfd79
gh pr view 585 --json statusCheckRollupreportsE2E tests (Strix Halo, WSL2): FAILURE,E2E tests (Strix Halo, Windows): CANCELLED andE2E tests (MI300X): QUEUED. Every other check passed. The diff only changes howe2e-reportlays out its HTML and sorts platforms, so it probably did not cause these failures. Logs were not read, so that is not established.
Confidence 100 (red status read directly) · architectural · Fix: check which runner the WSL2 failure ran on and which attempt it was, then rerun the cancelled and failed lanes. Do not merge until the required lanes pass on this head. -
[new] The test plan claims coverage that does not exist — PR description, Test plan item 4. Relevant code:
crates/e2e-report/src/components.rs:19-33,crates/e2e-report/src/consolidated.rs:847-854,:1882-1926,:2158,:2198
The plan says "existing tests already cover output equivalence for both the HTML shell and the sort order". It citesthe_last_naming_artifact_sets_the_sort_positionandtwo_channels_same_platform_yield_two_columnsas "the platform-ordering tests". Neither test callsplatform_sort_key: they exercisegrid_orderandGrid::build, which sort separately (consolidated.rs:620).
What the code actually tests:- HTML shell: the only assertion that touches it is
html.contains("Consolidated E2E Report")ingenerate_consolidated_writes_html. That string also appears in the body<h1>(:889). Dropping the<style>,<title>,DOCTYPE,langor charsetmetafrompage_shellwould leave every test passing. single_report::generate: no unit test at all. Its only caller (tests/e2e-cucumber/tests/e2e.rs:1358) just.expect()s theio::Result.- Sort key: only the channel element is pinned, by
:1917-1925for HTML and:2834-2848for markdown. Dropping or reversingplatform,osorknown_bugsbreaks no test.
Pure refactors are allowed to ship without new tests, but this description tells later readers that equivalence is covered when it is not.
Confidence 85 (traced each mutation against the tests by reading; nothing was run) · logic · Fix: narrow the claim to what is actually pinned, or add the tests described under Non-blocking so the claim becomes true. - HTML shell: the only assertion that touches it is
Non-blocking
-
[new] The new shared helpers have no direct test —
crates/e2e-report/src/components.rs:19-33,crates/e2e-report/src/consolidated.rs:847-854
The gap existed before this change. Now that both generators share one function, a single test covers both:- Shell: assert
<title>{title}</title>and the presence of the<style>body inpage_shelloutput. - Sort key: one consolidated test with two platforms and a known-bugs row, asserting row order. That would pin the three tuple elements nothing checks today.
Confidence 75 · logic.
- Shell: assert
-
[new]
STYLEis wider than it needs to be —crates/e2e-report/src/components.rs:164
After this change the only user ofpub(crate) const STYLEispage_shellin the same file; grep finds no other reference in the repo. Making it private makespage_shellthe only way to get the stylesheet, so no future generator can skip the shell. PR #433, which split this crate, needed a follow-up commit to narrow visibility for the same reason. Nothing will flag this automatically, becauseclippy::redundant_pub_crateis allowed workspace-wide (Cargo.toml:108).
Confidence 70 · mechanical · Fix:const STYLE. -
[new] A twin of the platform label, with a record of drifting, was left duplicated (pre-existing, related by pattern) —
crates/e2e-report/src/consolidated.rs:967(markdown platform cell) and:1615(HTMLmatrix_tablecell). The column keywith_channel_suffix(&format!("{} {}", platform, os), effective_channel())is rebuilt at:1477and:1489, but it already exists asPlatformReport.label(:749-750).
Commit 7585358 (#460) records that "the markdown and HTML consolidated reports had drifted again" on exactly this cell. The newplatform_sort_keydoc comment gives the same rationale ("so a future ordering tweak can't land in only one of them"), and that rationale applies here too.
Confidence 70 · architectural · Fix (here or in a follow-up): user.labelincommand_coverage_markdown, and add aplatform_cell()helper shared by the two tables.
Decisions for the author
None
Positive signals
- Keeping
sort_bywith a shared key function instead of switching tosort_by_keywas right, because the key borrows from the element. Sharing the key also means the HTML channel-order test (:1917) now covers the markdown path as well.
Deployment notes
None
What this covered
I read every file in the change completely, including the consolidated.rs tests, plus lib.rs, Cargo.toml, the crate's callers in xtask/src/e2e_report.rs and tests/e2e-cucumber/tests/e2e.rs, and the module description in docs/architecture.md. Diff range: 2fdff46c…13cdfd79.
Checks that ran:
- Byte-identical output: a spliced
Markuprenders raw in maud, and neither title contains characters that need escaping. - Identical ordering:
Stringandstrcompare the same way, and the tuple order is unchanged. - Commit hygiene: the commit has a DCO sign-off and passed the signature gate, contains no internal hostnames or URLs, and needs no Gherkin scenario because the change is internal and the PR text says so.
Every check was static, because the shared disk is short on space: nothing was compiled, tested or mutated by running. The coverage findings come from tracing each mutation against the test assertions by reading.
The code-pass and history/prior-changes/code-comments passes ran as two independent read-only workers. The prior-changes pass covered the reviews of #460 and #433. The design review (Step 7) was done in the driver context without separate workers, because the change is small; it found nothing beyond the twin noted above. The PR description reached the driver context together with the CI-status query, before the findings were settled, rather than after them. CI was read by check name and conclusion only.
I did not reconcile this against any PR discussion here.
Publisher's note, not part of the reviewer's findings: the reviewer did not read CI logs; the publisher did. The WSL2 lane failed on one unexpected regression, comfyui-source-download-shows-live-progress (comfyui-04). It timed out after 30s while the CLI was still on "Preparing Python 3.12... Downloading uv" from github.com, which happens before the step under test runs. All 3 other failures on the lane were expected xfails. The Windows lane was cancelled and MI300X is still queued. This change touches only crates/e2e-report, which is not on that code path, so these look like lane or network problems rather than a regression. A rerun should clear them. This round is published as a comment, not a change request: the required checks already block the merge until they pass, and the test-plan overclaim lives only in the PR text. The commit message makes no coverage claim, and its "behavior-preserving" statement held up under static review.
|
Thanks for the review — this reached me as an automated review round under your account rather than a human comment, so replying here since a review body carries no inline thread to answer in. Pushed Required E2E lanes not green (blocking) — confirmed, settled without a code change. Both failures are pre-existing, already-tracked, unrelated flakes:
Neither touches Test plan claims coverage that does not exist (blocking) — confirmed and fixed. Added New shared helpers have no direct test (non-blocking) — fixed by the same two tests above.
Platform-label twin, pattern from #460 (non-blocking) — fixed rather than deferred: extracted All of |
rominf
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 15f1aec
This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.
Review — needs work
Full review of the whole change.
This addresses some of the dedup findings logged in ROCMAI-200 (an inbox ticket for ROCMAI-27 cleanup findings). It does not claim to close that ticket. ROCMAI-27 is cited only as where the findings came from.
Blocking
-
[new] The new sort-key test says it pins the
platformelement, and nothing pins it —crates/e2e-report/src/consolidated.rs:2241-2250, PR description (Test plan, second "Added" item)- The test is named
platform_sort_key_orders_by_platform_then_os_then_known_bugs. - Its comment says the fixture's rows "distinguish the other three" tuple elements. The description says it pins "the
platform/os/known_bugstuple elements". - All three fixture rows parse to
Strix Halo, so theplatformelement is never compared. - Run against the full
e2e-reportsuite (56 tests), all three of these mutations stayed green:- replacing
&r.desc.platformwith""inplatform_sort_key - swapping the platform and os elements
- moving channel to the front
- replacing
- No other test in the crate pins platform ordering. So the test's name, its comment and the PR description all claim coverage of a branch it cannot fail on.
- The same comment also says channel is pinned "elsewhere (generate_consolidated_writes_html)". Dropping channel fails both that test and
command_coverage_distinguishes_channels_on_same_platform.
Confidence 100 · logic · Fix: make the claim true rather than narrowing it. Add a row on a second platform, e.g. a MI300X/Linux artifact, whose OS sorts ahead of Ubuntu, so that dropping
platformfrom the key reorders it. Then correct the "pinned elsewhere" sentence to name both tests. - The test is named
Non-blocking
-
[new] The
labelfield doc is now stale, and the field has become a key —crates/e2e-report/src/consolidated.rs:166,:1486-1494- The doc still says "Human label kept for the per-platform detail sections".
- After this change,
labelis also the identity used to de-duplicate command-coverage columns across tiers. Before this change it was already used in the XPASS/Regression notes at:1035/:1042. - A future display-only tweak to
label, such as a known-bugs marker, would silently split coverage columns per tier. No test pins the cross-tier merge.
Confidence 70 · mechanical · Fix: document every consumer, including that the coverage table uses
labelas a column key. Optionally add a test that two tiers of one platform share one coverage column. -
[new] The
platform_celldoc comment cites a drift that never happened —crates/e2e-report/src/consolidated.rs:856-860- It says the markdown summary and
matrix_tablecould drift "the way they already have once (see the command-coverage columns below, and the markdown/HTML drift fixed by #460)". - The #460 half is accurate: commit 7585358 fixed the platform cell in
matrix_table. - The command-coverage columns were never a
platform_celldrift. They render platform plus OS by design, history shows they always matchedlabel, and after this change they user.label, notplatform_cell. "See below" now points at code unrelated to this helper.
Confidence 70 · mechanical · Fix: keep the #460 reference and drop the command-coverage clause.
- It says the markdown summary and
Decisions for the author
platform_sort_key/platform_cellare free functions next toeffective_channel(a method) andlabel(a field) — non-blocking-improvement- That gives three different shapes for facts about the same
PlatformReport. - As methods (
r.sort_key(),r.platform_cell()) they would sit with their siblings. Nothing about the borrowed tuple forces free functions.
- That gives three different shapes for facts about the same
- Expectation-grid columns follow a third ordering — non-blocking-improvement
- Grid columns follow input (artifact-name) order, not
platform_sort_keyorder. - This predates the change, and the grid columns are a different type with a different identity, so the key cannot apply directly.
- Worth a follow-up only if the grid and the matrix should read in the same order.
- Grid columns follow input (artifact-name) order, not
Positive signals
- The output is unchanged. A dump-and-diff of consolidated HTML, consolidated markdown and single-report HTML, on a fixture with channels, a known-bugs artifact,
commands.jsonlandplatform.json, was byte-identical toprw-baseonce timestamps were stripped. - Sharing
platform_sort_keybetween the HTML and markdown paths closes the paired-copy pattern that review on #460 had flagged repeatedly. A key mutation is now caught through either path. - Narrowing
STYLEto private oncepage_shellbecame its only user keeps the shell the single route to the stylesheet.
Deployment notes
None
What this covered
- Read: all three changed files completely, plus
xtask/src/e2e_report.rsand the other consumers of the crate. Reviewed againstprw-baseb863f77…15f1aec9 (merge base; head 15f1aec). - Passes: coupled code pass, agent-instruction adherence, history, prior-change reviews of #460/#433/#354, code comments, and design questions, each run as an independent worker.
- Tests run:
cargo test -p e2e-reportpassed 56/56.cargo clippy -p e2e-report --all-targets -D warningswas clean. - Mutation testing: 26 single-site mutations.
- Verdicts on the added tests:
page_shell_carries_title_and_styledoes not compile against the pre-change code, becausepage_shellis new. It pins DOCTYPE, title and stylesheet. It stays green if the charset meta,langor the<body>wrapper is removed.platform_sort_key_orders_by_platform_then_os_then_known_bugspasses against the pre-change code. That is expected for a refactor: it pins os and known_bugs ordering. It stays green if the platform element is dropped or swapped (see the blocking finding).
- CI: all reported checks on 15f1aec succeeded, except Skill checks and the Sphinx build, which were skipped.
- Did not run: none of the passes was skipped.
- Low-confidence items dropped:
docs/architecture.md:54's description ofcomponents.rsis mildly stale.- The
style: cargo fmtfixup commit; the project squash-merges.
- Prompt injection: none found.
- Discussion: not reconciled against any PR discussion here.
Publisher's note, not part of the reviewer's findings: this round reviewed the whole change again, not just the commits since 13cdfd7. The two objections the 13cdfd7 round raised were then each put to the reviewer after its findings were final. Those answers are anchored evidence, not blind findings.
- Test plan overclaims coverage (standing): partly resolved.
- The HTML-shell part is resolved: dropping the DOCTYPE, the style or the title now fails
page_shell_carries_title_and_style. - The
platformpart still holds and is the blocking finding above. It was re-found blind in its new form: a committed test name and comment that make the claim, not only the PR text. - Observation for the next round: the last Test plan item says the existing tests check output "byte-for-byte". Those tests match substrings, not exact output, though mutating either call site does make them fail.
- The HTML-shell part is resolved: dropping the DOCTYPE, the style or the title now fails
- Required E2E lanes not green (standing): no longer applies as worded.
- The reviewer could not tell either way.
- The publisher queried the live branch-protection required set. It lists 17 contexts and none of them is a GPU E2E lane. All 17 have reported success on 15f1aec, apart from the Sphinx build, which was skipped.
mergeStateStatusis BLOCKED only because a review is required.- The self-hosted GPU E2E workflow for 15f1aec is still
pendingand has created no check runs. It is not required, so it is mentioned only so nobody reads its absence as a pass.
Why this round files a change request when the previous one did not: the overclaim no longer lives only in the PR text. It is now in a committed test name and comment, so if this merges, a future change that drops platform from the key passes CI while the test's name says that ordering is guarded. The fix is one more fixture row.
|
Thanks — same situation as the first round: this reached me as an automated review round under your account, so replying here since a review body carries no inline thread ( Blocking — platform_sort_key test didn't pin
Also corrected the "pinned elsewhere" sentence in the test's comment to name both tests, not just one. Non-blocking — Non-blocking — Decisions for the author:
All pushed in |
Feedback addressed. Ready for approval.
rominf
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · a0ad40f
This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.
Review — needs work
Review of changes since the review at 15f1aec (one commit, a0ad40f), plus the file it touches.
Carries out part of the ROCMAI-200 quality-findings backlog (the e2e-report dedup items) and does not claim to close it.
Blocking
- [new] The claim that the sort key's channel element is covered elsewhere is false: channel's position in the key is not pinned by any test. —
crates/e2e-report/src/consolidated.rs:851-858(platform_sort_key),crates/e2e-report/src/consolidated.rs:2246-2248(test comment), commit message of a0ad40f, PR description test plan ("previously onlychannelwas pinned")- What the change claims. The commit message says the "moving channel to front" mutation stays green "as expected, since that's covered by generate_consolidated_writes_html and command_coverage_distinguishes_channels_on_same_platform". The test comment now cites the same two tests as pinning channel.
- Why that doesn't hold. Both tests use one platform with two channels. That proves channel is a tie-breaker, but not where it sits in the key. Every row in the new sort-key test has no channel (
None). - What I ran. I changed the key to
(effective_channel(), platform, os, known_bugs)and rancargo test -p e2e-report --lib: 56 passed, 0 failed, rc=0. The three named tests were in the output. - Second mutation (worker run). Swapping only
known_bugsandchannelalso leaves the suite green. - Which side governs. The function doc at :848 ("…then known-bugs flag, then channel") and the code govern. They are the stated ordering, so the coverage claims are the wrong side.
- Confidence 100 · logic · Fix: add a fixture where the channel order and the platform order disagree, for example two platforms with opposite
nightly/releasechannels, and assert that platform order wins. Or correct the comment, the commit message and the PR description to say channel position is unpinned.
Non-blocking
- [new] The new
labeldoc lists two of its three consumers. —crates/e2e-report/src/consolidated.rs:166-170- The doc names two consumers and tells readers to "keep that consumer in mind alongside the detail sections". Those two are the detail sections (:1687) and the command-coverage column key (:1489, :1497).
r.labelis also printed in the markdown XPASS and Regression notes (:1038, :1045), and the doc leaves that out.- The code's actual uses govern.
- Confidence 70 · mechanical · Fix: name the notes as a consumer too, or drop the attempt to list consumers.
- [new] The commit message gets the sort direction backwards. — commit message of a0ad40f
- It says the new row's platform "sorts before Strix Halo's". "Unknown" sorts after "Strix Halo". The test's own comment (:2252-2253) and its assertion
windows_pos < unknown_pos, which passes, govern. - Confidence 85 · mechanical · Fix: reword before squash, if the message is carried into the squash body.
- It says the new row's platform "sorts before Strix Halo's". "Unknown" sorts after "Strix Halo". The test's own comment (:2252-2253) and its assertion
Decisions for the author
None
Positive signals
- The added "Unknown"/"Unknown" row is well chosen. Its OS ("Unknown") sorts ahead of "Windows" while its platform sorts after "Strix Halo", so dropping the platform element or swapping it with os now fails. I checked both mutations: the old test stayed green on each, the new test fails on each.
- Dropping the "see the command-coverage columns" reference from
platform_cell's doc is correct. Those columns key onlabel, notplatform_cell, and the #460 history confirms it.
Deployment notes
None
What this covered
- Read and run:
- The diff 15f1aec..a0ad40f, which touches only
crates/e2e-report/src/consolidated.rs. A worker read all 3042 lines. - Every caller and callee of the touched symbols, all of which are in that file.
git log/git blameonprw-basefor the edited regions, and the #460 history.- The bodies of earlier PRs #460 and #433 that touched these regions.
- The diff 15f1aec..a0ad40f, which touches only
- Verdicts for the changed test (
platform_sort_key_orders_by_platform_then_os_then_known_bugs):- It fails with the platform element dropped or swapped, so it is not a revert-survivor.
- Single mutations that stay green: channel moved to the front, and known_bugs swapped with channel. This is the blocking finding.
- Tests run: targeted e2e-report tests and the full
--libsuite (56). Clippy, fmt and the workspace suite were not run locally. - CI: CI on a0ad40f is all green. Skillscope and Sphinx were skipped as path-gated.
mergeable_stateisblocked. - Leak scan: clean apart from the DCO sign-off email. Sign-off is present and the commit is signed.
- Fan-out: two independent workers, one for the code pass and one for the source-reading passes. The Step 7 design review ran in the driver, not as independent workers. Given a comment-and-test-only delta, it produced no design questions worth reporting.
- Intent: PR description read, and ROCMAI-200 retrieved by its ID. ROCMAI-27 was not retrieved.
- Not reconciled against any PR discussion.
Publisher's note, not part of the reviewer's findings:
- The standing objection from 15f1aec (the test does not pin
platform) is settled: it no longer holds. This round's reviewer did not re-find it. Its scope included the file. After its findings were final, the objection and your reply were put to it, and it confirmed that droppingplatform, or swapping it withos, now fails the newwindows_pos < unknown_posassertion, while the old test stayed green under both. That confirmation is anchored evidence: the reviewer was told what to check. The change request it backed was dismissed on 2026-10-09 with "Feedback addressed". The publisher agrees it is resolved and is not re-filing it. - This round is a comment, not a change request, even though it lists a blocking finding. That is a judgement about the merge gate, not about the finding.
- What stays wrong after merge is narrow. A test comment, a commit message and one description line say channel is "pinned elsewhere", but only its role as a tie-break is pinned, not its position. If the key were reordered, rows in the generated report would display in a different order, and nothing in CI or the CLI depends on that order.
- The 15f1aec round also listed channel-to-front as a mutation that stayed green, and asked only that the "pinned elsewhere" sentence name both tests, which you did. Blocking now on a mutation that round had already seen would move the goalposts.
- Correcting the three sentences, or adding the channel-vs-platform fixture, is still worth doing before merge.
|
Thanks — same situation as the prior rounds: this reached me as an automated review round under your account, so replying here since a review body carries no inline thread ( Blocking — channel's position in the sort-key tuple wasn't actually pinned: fixed. Non-blocking — Non-blocking — commit message gets the sort direction backwards: left as-is. The repo squash-merges, so the message will be rewritten into the squash commit body at merge time rather than carried into permanent history as-is; not worth a force-push to fix in place. All pushed in |
rominf
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 6fdda20
This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.
Review — needs work
Full review of the whole change.
Takes three duplication items from the ROCMAI-200 backlog (an open running list for ROCMAI-27) and does not claim to close either ticket. The description says the output is unchanged, and the code matches that; the only overclaim is about test coverage.
Blocking
-
[standing, narrowed] The sort-key test does not pin all four tuple elements, though its comment, the PR description and commit 6fdda20 all say it does —
crates/e2e-report/src/consolidated.rs:2245-2269(test comment),:852-859(platform_sort_key), PR description test-plan bullet 5, subject of 6fdda20
The test comment says it "Pins all four elements of the sort-key tuple, in precedence order: platform, os, known_bugs, channel". The PR description repeats this ("pins all four tuple elements").- No two fixture rows differ in
known_bugsand channel at once. The only rows with a channel (MI300X/release, R9700/nightly) are on different platforms. - So channel's position is pinned only against
platform, not againstosorknown_bugs. - I ran both reorderings below with
cargo test -p e2e-report --lib. Each time 56 passed, 0 failed, andplatform_sort_key_orders_by_platform_then_os_then_known_bugs ... okwas in the run:(platform, os, channel, known_bugs)(platform, channel, os, known_bugs)
- Commit 6fdda20's own body lists "swapping it with known_bugs" as one of the mutations that left the suite green. It then reports only the move-to-front mutation as fixed. The swap still passes.
- The two orders can really disagree:
parse_descriptoraccepts names likee2e-gpu-known-bugs-nightly-report, so known-bugs and channel can both vary on the same platform/OS. - The tuple is the side that governs, because it is what the code sorts by. The claims are what is wrong.
Confidence 100 · logic · Fix: either add a same-platform/OS pair where
known_bugsand channel order disagree (e.g.-known-bugs-nightlyvs-release), plus one where OS and channel disagree, or narrow the test comment, the PR description and the squash message to what is actually pinned. The first is better. - No two fixture rows differ in
Non-blocking
-
[new] The
page_shelltest leaves out half the shell —crates/e2e-report/src/components.rs:262-274- The test asserts the doctype, the title and
STYLE, but not the body. - I removed both
meta charset="utf-8"andlang="en"frompage_shelland reran: 56 passed,page_shell_carries_title_and_style ... ok. - The charset matters because the report emits UTF-8 glyphs (✅ ❌
⚠️ ).single_report::generatehas no test at all. - Nothing tested these before either. The new test's name claims only title and style, so it does not overclaim.
Confidence 100 · mechanical · Fix: also assert the charset meta, and that the body markup appears inside
<body>. - The test asserts the doctype, the title and
-
[new]
PlatformReport::loadstill has its own copy ofeffective_channel(), and this change puts more output behind it —crates/e2e-report/src/consolidated.rs:752-753vs:773-778- In
load,versions.channel.as_deref().or(desc.channel_from_name)repeats the body ofeffective_channel()inline. - Before this change the coverage columns went through
effective_channel(). Now they uselabel, which is built from the inline copy. - A later change to channel precedence in only one copy would put coverage columns out of step with the matrix and the sort. That is exactly the drift this PR is trying to remove.
- The duplication predates this PR; the PR makes it matter more.
Confidence 70 · logic · Fix: have
loadcompute the label through the same functioneffective_channel()uses. - In
-
[standing, widened] Commit messages contradict each other, and both land in the squashed message — commits a0ad40f, 6fdda20
- a0ad40f says the move-channel-to-front mutation is "covered by generate_consolidated_writes_html and command_coverage_distinguishes_channels_on_same_platform".
- 6fdda20 then says that mutation "left the whole suite green". 6fdda20 is right.
style: cargo fmtis a fixup commit, anddedup:is not a conventional-commit type.
Confidence 70 · mechanical · Fix: edit the squash message so it says only what is true at the head.
Decisions for the author
- Using
PlatformReport.labelas the command-coverage column key — tradeoff- For: one source of truth for the "platform OS (channel)" string. Today's output is the same as before.
- Against: a display string now also decides column identity. A purely cosmetic edit to
label(e.g. a known-bugs marker) would silently split or merge coverage columns. - The new doc comment points this out but does not prevent it. A dedicated
coverage_column(r)function would keep display and identity separate. - The doc comment also leaves out that
labelis the visible column header text too.
Positive signals
platform_cell()makes the matrix and markdown summary share one function. That is a structural fix for the twin-site drift #460 had to fix by hand, instead of another hand fix.- I traced the refactor and it preserves output byte-for-byte: same shell markup, sort tuple, platform-cell expression, and coverage-column string as before.
Deployment notes
None
What this covered
- Read: all three changed files in full (
components.rs,consolidated.rsincluding its whole test module,single_report.rs). Also the callers of changed symbols,xtask/src/e2e_report.rs, rootAGENTS.md, and the history of the touched regions (#460, #433). - Commits: prw-base 844edaf…6fdda206.
- Runs:
cargo test -p e2e-report --libat the head (56 passed), plus three single-mutation runs, all 56 passed. - CI: no failures. MI300X was still queued, and the Strix Halo Windows and WSL2 E2E lanes were still in progress.
- Ticket: ROCMAI-200 was retrieved. ROCMAI-27 was not retrieved separately.
Did not run:
- The two worker passes did not run mutations. Every mutation result above is from my own runs.
- The four source-reading passes ran in one worker, so they are not independent of each other.
- Prior-change review bodies over about 3000 characters were truncated where read.
Not reconciled against any discussion here.
Publisher's note, not part of the reviewer's findings:
- Full review, not a delta. The a0ad40f round left a finding about a commit message, which no single file holds, so this round read the whole change blind rather than just a0ad40f..6fdda20.
- The a0ad40f blocker was re-found blind, but narrower. That round said channel's position in the key was not pinned at all. The MI300X/R9700 rows now pin it against
platform, and this reviewer, without being told about the old finding, found that it is still unpinned againstosandknown_bugs. The a0ad40f round had already recorded the known_bugs/channel swap as a mutation that stays green. What is new this round is the test comment and description bullet that now say "pins all four". - The
labeldoc non-blocking finding from a0ad40f was not re-found. The doc now names the XPASS/Regression notes as well, so it is dropped. The non-blocking finding about the commit message getting the sort direction backwards still stands; the author chose to fix it in the squash message, which is fine. It is folded into the commit-message item above. - This round is again a comment, not a change request, even though it lists a blocking finding. That is a judgement about the merge gate, not about the finding. If this merges, what is left wrong is the same as last round: a test comment and one description line overclaim, and a future swap of
channelwithosorknown_bugswould reorder rows in the generated report with no test catching it. Nothing in CI or the CLI depends on that order. The a0ad40f round judged this class non-gating, and nothing new makes it a defect with more consequence. Narrowing the comment to "pins platform, os, known_bugs, and channel against platform", or adding the two same-platform rows, is still worth doing before merge.
|
review-coverage · 74ec7e1 Full-scope closing review of the complete remediation for review |
|
Follow-up on my own fix above ( Fixed in With this, the comment's "pins all four elements in precedence order" claim is now actually true (previously three of the four precedence pairs were pinned; this closes the last one). |
74ec7e1 to
1c20e7a
Compare
rominf
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 1c20e7a
This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.
Review — needs work
Full review of the whole change.
The description names ROCMAI-27 and ROCMAI-200. ROCMAI-200 is a quality-findings backlog that this change draws from, and the PR does not claim to close it. ROCMAI-27 was not retrieved.
Blocking
- [new] The description claims a
command_coverage_markdownchange that is not in this diff —crates/e2e-report/src/command_coverage.rs:154,:163
The description says: "command_coverage_markdownrebuilt the full platform+OS label twice within the same function … switchedcommand_coverage_markdownto reuselabelfor the full one."git diff prw-base...HEADdoes not touchcommand_coverage.rs.prw-basealready haslet col = r.label.clone();at both sites; that came in with #591 (c1c2aeb). None of the six commits touches the file either. A reader who trusts the description will credit this PR with a dedup it does not contain.
Confidence 85 · mechanical · Fix: narrow the description. Drop thecommand_coverage_markdownclause, or say the reuse already landed on main through #591. The "Test plan" bullet about "platform_cell()/labelreuse" needs the same change. - [new] A required-looking CI lane is red on this head — check
E2E tests (Strix Halo, WSL2)
gh pr view 585 --json statusCheckRollupat1c20e7a2showsE2E tests (Strix Halo, WSL2)COMPLETED/FAILURE.E2E tests (MI300X)is still QUEUED. All other checks pass. This diff only touches report-generator rendering incrates/e2e-report, so the failure may have nothing to do with it. Per instructions, I did not investigate it; attributing it against main happens separately.
Confidence 100 · architectural · Fix: show the WSL2 failure is unrelated (for example, the same failure on main) or fix it, and let MI300X finish before merging.
Attribution, added by the poster: the job ran 171 scenarios; 4 failed, 3 of them expected failures. The one unexpected failure iscomfyui-source-download-shows-live-progress, which timed out after 30s waiting for a download progress frame. That scenario exercises the CLI's ComfyUI download output, and this PR changes none of that code. The same lane passed on main's last six commits, so this is not a failure main shares. It looks like a host or timing flake that this change did not cause. A re-run should clear it.
Non-blocking
-
[new]
platform_sort_keyordering is only pinned on the markdown path; the HTML call site can drift without a test failing —crates/e2e-report/src/consolidated.rs:890
Mutations at the HTML call site ingenerate_consolidated, with the markdown path still onplatform_sort_key:- Sorting without
known_bugs: all 56 tests pass. - Sorting os-before-platform: all 56 tests pass.
- Unsorted, or sorted by platform only: caught, but only by
generate_consolidated_writes_html.
The new test reads only
consolidated_summary_markdownoutput. Sharing the key makes drift unlikely, but nothing enforces that the HTML path uses it. HTML coverage before this PR was the same, so this is not a regression.
Confidence 100 · mechanical · Fix: unit-testplatform_sort_keydirectly onPlatformReport::load(...)-built reports, as the existing tests atconsolidated.rs:1567already do. That covers both paths and removes the row-identity-by-scenario-count trick. - Sorting without
-
[new] The test name understates what it pins, and the test leans on brittle markdown matching —
crates/e2e-report/src/consolidated.rs:2001
platform_sort_key_orders_by_platform_then_os_then_known_bugsalso pins channel's precedence (the MI300X/R9700 rows and the MI300X known-bugs/nightly row). Problems with its structure:- Its comment is about 30 lines and recounts its own mutation history.
- It identifies rows by scenario counts 1–7.
- It locates rows with
md.find("| MI300X (release) | Linux | 5 |"), so any markdown column change breaks it.
Confidence 85 · mechanical · Fix: rename it (e.g.
platform_sort_key_precedence). Moving it to the direct unit test suggested above lets most of the comment go. -
[new] The description's "pins all four tuple elements" holds only for the suite, not for this test —
crates/e2e-report/src/consolidated.rs:852-859
Replacing channel withNoneinplatform_sort_keyleaves the new test green. Onlycommand_coverage_distinguishes_channels_on_same_platformandgenerate_consolidated_writes_htmlfail. The test pins channel's position relative to platform and known_bugs, not its presence.
Confidence 100 · mechanical · Fix: reword the test-plan bullet, or add a same-platform/os/known_bugs pair with different channels to the new test.
Test verdicts:
page_shell_carries_title_and_style- With the production change reverted,
page_shelldoes not exist, so the test exercises the new helper. - Dropping the
styleelement or hard-coding the title fails it. - Dropping
meta charset="utf-8"orlang="en"leaves it, and the whole suite, green. Its comment says it pins "the shared pieces", which overstates that. This is minor.
- With the production change reverted,
platform_sort_key_orders_by_platform_then_os_then_known_bugs- The PR is behavior-preserving, so this test would pass with the extraction reverted (inline tuple). It pins output order, not the refactor, which is expected for a dedup.
- Swapping platform/os, dropping platform, moving channel to the front, and swapping known_bugs/channel each fail it alone.
- Dropping channel does not fail it (see above).
Decisions for the author
platform_cell,PlatformReport::labelandGridColumn::labelremain three label builders overwith_channel_suffix— tradeoff
They take different inputs on purpose: platform only, platform+OS, and slug.load(consolidated.rs:756) also repeats theversions.channel.or(desc.channel_from_name)expression thateffective_channel()(:776) holds. Collapsing them further would mean a free helper for the channel resolution; leaving them keeps each call site obvious. Either is defensible.
Positive signals
- Making
STYLEprivate now thatpage_shellis its only consumer enforces the dedup at the type level. A third generator cannot quietly inline its own shell. - The
labeldoc comment now names all three consumers and warns that a display-only edit would split or merge command-coverage columns. That hazard is real (command_coverage.rs:154,163key on it) and was undocumented.
Deployment notes
None
What this covered
All three changed files read completely, plus command_coverage.rs and the Descriptor/parse_descriptor/with_channel_suffix/effective_channel code they rely on. Range c1c2aeb3…1c20e7a2 (prw-base...HEAD).
- Tests:
cargo test -p e2e-report --libpassed at head with 56 tests, including both new ones. - Mutation runs: 17, one at a time, on the sort key, both call sites,
page_shellandplatform_cell. - Comment claims checked against the code and history: the #460 drift claim (confirmed in
7585358's message), the artifact-name → platform/OS/channel mappings, and the cited test names. - HTML output identity: reasoned from maud semantics, not rendered and diffed.
- Passes run: coupled code pass, agent-instruction adherence, history, prior changes, code comments, and design questions, as independent workers.
- Not run or not checked:
- Coverage floor:
cargo xtask coveragewas not run. - Commit signing:
%G?showsUlocally; the CICommit signatures + sign-offcheck passed. - ROCMAI-27: not retrieved. ROCMAI-200's comments were not read.
- Coverage floor:
Not reconciled against any discussion on the PR.
Round notes (added by the poster)
- The earlier round's blocker on sort-key channel position no longer applies. This round's blind review did not re-find it. The objection was then put to the same reviewer after its report was done. That check was anchored, because the reviewer was given the objection and the author's reply. Both reorderings it named,
(platform, os, channel, known_bugs)and(platform, channel, os, known_bugs), now failplatform_sort_key_orders_by_platform_then_os_then_known_bugs(55 passed, 1 failed each). The seventh fixture row is what catches them. Two parts remain, and both are covered by the non-blocking items above:- os-vs-channel is pinned only indirectly.
- Channel's presence in the key is not pinned by this test.
- Why this is a comment and not a change request.
- The description mismatch has no effect on the code: the
command_coverage_markdownreuse is already on main through #591. Merging leaves only an inaccurate line in the squash message, so fixing the description is enough. - The red WSL2 lane is attributed above to a failure outside this diff.
- Neither one warrants holding a merge gate.
- The description mismatch has no effect on the code: the
|
Thanks for round
Non-blocking suggestions and the label-builder tradeoff noted but left as-is for now. |
siloteemu
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 1c20e7a
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — needs work
Review of the whole change: prw-base...1c20e7a2, three files under crates/e2e-report.
The description frames it as three behaviour-preserving dedups; bare ticket IDs are named only as provenance, and no issue was retrieved.
Blocking
-
[new]
page_shell_carries_title_and_stylestays green when most ofpage_shellis broken, and nothing asserts on the single-report path —crates/e2e-report/src/components.rs:262-274,crates/e2e-report/src/single_report.rs:60-88- The test asserts only the doctype, the
<title>and the stylesheet. Its comment says it pins "that it actually renders the shared pieces". - I ran mutations on
page_shell. Removingmeta charset="utf-8"left all 56 tests green. Removinglang="en"left all 56 green. - Replacing
(body)with nothing is caught only bygenerate_consolidated_writes_html(consolidated generator). single_report::generatehas no test incrates/e2e-report. Apage_shellthat dropped the body would ship a blank single-platform report with every crate test green.- Verdict 1: this is a new test and the refactor is behaviour-preserving, so reverting
page_shellwould fail it only by not compiling. Verdict 2: dropping the charset,lang, or the body wrapper leaves it green. - Confidence 100 for the charset and
langmutations (run); 85 for the body and single-report gap · logic · Fix: assert the body is rendered (for exampleout.contains("<body><p>body</p></body>")) plus<meta charset="utf-8">andlang="en". Or add a smoke test ofsingle_report::generateagainst a smallreport.json.
- The test asserts only the doctype, the
-
[new]
platform_sort_key_orders_by_platform_then_os_then_known_bugsclaims to pin all four key elements, but it does not fail whenchannelis removed from the key —crates/e2e-report/src/consolidated.rs:2001-2031(comment),consolidated.rs:854-861- The comment opens with "Pins all four elements of the sort-key tuple", and the PR text repeats it. No two of the seven rows share platform, OS and known-bugs while differing in channel. The only same-platform-and-OS pair, MI300X release vs MI300X known-bugs nightly, differs in known-bugs.
- I replaced
r.effective_channel()withNoneinplatform_sort_key. The tests that fail arecommand_coverage_distinguishes_channels_on_same_platformandgenerate_consolidated_writes_html. The new test is not among them. - The new test does catch dropping or reordering the other elements, and channel moved ahead of platform or known-bugs (all run). Only channel's presence as a tie-breaker is unpinned.
- A comment claiming coverage the test cannot fail on is worse than no comment, because it tells the next reviewer the gap is closed. The test name also omits channel.
- Confidence 100 · logic · Fix: the code governs, because the tuple includes
effective_channel(). Either add a row that differs from an existing one only in channel (same platform, OS and known-bugs, release vs nightly) and assert their order, or narrow the comment and the PR text to what is pinned. Rename the test to match either way.
Non-blocking
- [new] Commit history carries a churn trail and non-conforming subjects —
git log prw-base..HEAD295a0a62,6829d4e4and1c20e7a2are successive corrections to the same new test, and3f7ffa60is a barestyle: cargo fmt. CONTRIBUTING.md asks for one logical change per commit.6829d4e4is typedfix(...)but only adds a test row and a comment, sotest:fits.407402a1usesdedup:, which is not a Conventional Commits type, sorefactor(e2e-report):fits.- Squashing into the two logical commits (page shell, sort key and platform cell, each with its tests) would fix all of it. Confidence 70 · mechanical.
Decisions for the author
- The sort-key test comment runs about 30 lines of mutation-analysis narrative — non-blocking-improvement
It documents the review history, cross-references two other tests by name, and relies on the 1-7 row-count trick. It will rot if one row changes. Per-row inline comments or a short table would hold up better. Not a defect. - No scenario added — tradeoff
e2e-reportis internal CI report tooling, consumed byxtaskand the test harness (docs/architecture.md), notrocmCLI behaviour. AGENTS.md §3 therefore does not require a Gherkin scenario. The PR text does not say so explicitly, which §3 asks for. One sentence would settle it.
Positive signals
- The
platform_sort_keytest uses artifact names whose derived platform, OS, channel and known-bugs values are correct. It is built so that droppingplatform,osorknown_bugs, or swapping platform with OS, each fail it. I confirmed this by running each mutation. - Narrowing
STYLEto private oncepage_shellwas its only caller is a clean encapsulation step. Nothing else referenced it.
Deployment notes
None. Generated HTML and sort order are unchanged by reasoning over the maud output, and the existing consolidated-report tests pass unchanged.
What this covered
Every changed file was read completely (components.rs, consolidated.rs, single_report.rs), plus command_coverage.rs, parse.rs, and the xtask and e2e-cucumber call sites of the public API. Diff prw-base...1c20e7a2, five commits. Two read-only workers covered the code and tests, and the instruction-adherence, history, prior-changes and comment passes.
Run: cargo test -p e2e-report (56 passed), plus about 15 single-branch mutations in the shared scratch target, each reverted. CI rollup at this head: all checks SUCCESS except "Skill checks (skillscope)" and "Sphinx docs build (-W)", both SKIPPED (not run; nothing in the diff touches skills or docs).
Commit signature verification could not run in this checkout, because no allowed-signers file is configured. The DCO sign-off is present on all five commits, and the "Commit signatures + sign-off" check is green.
Not reconciled against any review discussion here.
|
Thanks for round `1c20e7a2` — same situation as prior rounds: this reached me as an automated review round under your account, so replying here since a review body carries no inline thread. Both blocking items addressed in `2f94e7de`:
`cargo test -p e2e-report --all-targets` (57/57), `cargo clippy -p e2e-report --all-targets -- -D warnings`, and `cargo fmt --check` all clean at `2f94e7de`. |
rominf
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 2f94e7d
This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.
Gate: no change request is filed this round. Both items under Blocking are about the accuracy of comments and the description. Merging them would change no behaviour and remove no coverage. The two false comments overstate how much the new tests matter, which can only make someone keep a test, never delete one. Please still fix them before merge. In fairness to the author, the "every crate test green" premise repeats wording from this automation's 1c20e7a round, and that same report had noted the consolidated test catches a dropped body.
Review — needs work
Full review of the whole change.
Says it was "found during ROCMAI-27 code review and logged in ROCMAI-200". ROCMAI-200 is an open inbox for quality findings, and this PR does not claim to close it. ROCMAI-27 was not retrieved.
Blocking
- [new] Two new test comments are wrong about which tests cover
page_shell's body —crates/e2e-report/src/single_report.rs:106-108,crates/e2e-report/src/components.rs:272-274single_report.rssays apage_shell"that silently dropped itsbodyargument would still leave every other test in the crate green". To check this,(body)was replaced with""inpage_shell. Three tests went red:page_shell_carries_title_and_style,generate_consolidated_writes_htmland the new test.components.rssayssingle_report::generate"has no test of its own and depends on this one for coverage of the body wrapper". The same commit (2f94e7d) addsgenerate_renders_the_body_inside_the_shared_page_shelltosingle_report.rs.- The code governs here, because the mutation run shows the actual coverage. These comments are what the next reviewer reads to decide where coverage is thin, so they will look for a gap that does not exist. Commit 2f94e7d's message repeats the same false premise.
- Confidence 100 · mechanical · Fix: delete the false sentence in
single_report.rs. Incomponents.rs, delete the "has no test of its own" parenthetical, or point it at the new test.
- [new] The PR description no longer matches the diff — PR body, "Test plan"
- It reports "all 56 tests pass". The crate now runs 57, because the
single_report.rstest is new and the description never mentions it. - It also does not mention the rewrite of the
PlatformReport::labeldoc comment atconsolidated.rs:167-175. - It says
page_shell_carries_title_and_styleasserts "DOCTYPE/title/style". It now also assertslang, charset and the body wrapper. - Confidence 85 · mechanical · Fix: update the description to match the diff, not the other way round.
- It reports "all 56 tests pass". The crate now runs 57, because the
Non-blocking
- [new; the test-name point is standing from the 1c20e7a round] Test name and comments in the sort-key test have drifted —
crates/e2e-report/src/consolidated.rs:2001,:2030-2047- The name
platform_sort_key_orders_by_platform_then_os_then_known_bugsleaves out channel, which the test now also pins. - The roughly 50-line comment narrates the fix rounds ("confirmed by running that mutation"), and so does
components.rs:268-271("A prior version of this test asserted only…"). That history belongs in commit messages and will go stale here. - Rows 8 and 9 (Ubuntu release/nightly) duplicate the channel tie-breaker coverage that
generate_consolidated_writes_htmlalready has. Dropping channel from the key also turns that test andcommand_coverage_distinguishes_channels_on_same_platformred. - Confidence 85 · mechanical · Fix: rename the test, cut the comment down to what the rows pin, and consider dropping rows 8 and 9.
- The name
- [new] The new
labeldoc no longer says what the label contains —crates/e2e-report/src/consolidated.rs:167-175- The old doc said "platform, OS and effective channel". The new one lists consumers and leaves out the content.
- Its consumer claims are accurate: the detail sections, the XPASS/Regression notes, and the column de-dup at
command_coverage.rs:154,163. - Confidence 70 · mechanical · Fix: put the one-line content description back alongside the consumer note.
- [standing] Commit messages contradict the diff and each other — commits 330e39a, 295a0a6, 6829d4e
- 330e39a says it reuses
PlatformReport.labelincommand_coverage_markdown. That change is not in this diff; it landed via #591. - 295a0a6 says moving channel to the front of the key "still passes" and is covered elsewhere. 6829d4e then calls that a gap and fixes it.
- There is also a stand-alone
style: cargo fmtcommit. The repo squash-merges, so this only matters if these messages feed the squash message. - Confidence 85 · mechanical · Fix: write the squash message from the final state, not the commit trail.
- 330e39a says it reuses
Decisions for the author
- How the sort-key tuple is pinned — non-blocking-improvement
platform_sort_keyis a private function in the same module, so it could be unit-tested directly.- The test instead renders nine markdown fixtures that differ only in scenario count, and needs a long comment to explain which row pins which tuple element.
- Comparing the keys directly would pin precedence with a fraction of the setup and almost no explanation. The current form does have one advantage: it also proves the markdown call site uses the key.
Positive signals
- Deriving the matrix platform cell from one
platform_cell()removes a duplication that drifted once already (#460, confirmed in the commit history). The helper's doc says why it leaves out OS. page_shell_carries_title_and_stylecatches each of the six shell mutations on its own: dropping DOCTYPE,lang, charset, title, style or body each turns it red.
Deployment notes
None
What this covered
I read all three changed files completely at 2f94e7d (diff prw-base...HEAD, merge base c1c2aeb). I also read the surrounding code: command_coverage.rs (the label consumer), descriptor parsing, effective_channel and with_channel_suffix, plus a repo-wide grep for STYLE and page_shell.
- Behaviour: the refactor is behaviour-preserving. A temporary test showed
page_shelloutput is byte-identical to the old inline markup, and both the sort key and the platform cell build the same values as the code they replaced. - Mutations: 19 single mutations were run against
cargo test -p e2e-report(baseline: 57 passed, including the four new or changed tests).- Every mutation to the key, the shell and the platform cell turned at least one test red.
- One stayed green: sorting the HTML call site by channel alone (instead of through
platform_sort_key). - The new tests are characterization tests: they pass with the production change reverted, which is expected for a no-behaviour-change refactor.
- CI: every check on the head commit was green; skillscope and the Sphinx build were skipped.
Passes run: two independent workers, one code pass over the coupled group and one covering instructions, history, code comments and commit structure. Prior-changes pass: no earlier change reviews were read, by instruction. The design review (Step 7) ran in the driver context, not as independent workers. The Step 9 description check read the PR body and ROCMAI-200 by identifier. I did not look up ROCMAI-27.
The PR discussion was not reconciled here.
|
Thanks for round Both blocking items addressed in
Non-blocking items addressed too:
All CI checks on |
rominf
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · fc84b70
This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.
Gate: this round files a change request for the one blocking item, the commit messages. The repo squashes with COMMIT_MESSAGES and also allows rebase-merge, so every commit body below lands on main as written. That confirms what the 2f94e7d round could only say conditionally. That round asked for the 2f94e7d message to be fixed before merge without gating, and the push did not touch it. Rewording or squashing the branch, or the merger hand-editing the squash body, clears this. The code needs no change.
Review — needs work
Full review of the whole change, including all 8 commit messages.
This is a cleanup item from the ROCMAI-200 quality-findings backlog, which was found during ROCMAI-27 review. The change does not claim to close a ticket. The PR description matches the diff.
Blocking
-
[standing since 1c20e7a/2f94e7de, extended with fc84b70] The commit messages that will land on main state things that are false for this change and contradict each other — commits
330e39a,295a0a6,2f94e7de,fc84b70
The repo's squash setting issquash_merge_commit_message: COMMIT_MESSAGES, and rebase-merge is also enabled. Either way, every commit body below lands on main as written.330e39asays it "reuse[s] PlatformReport.label in command_coverage_markdown instead of rebuilding the identical string twice". That commit touches onlycomponents.rsandconsolidated.rs, andcommand_coverage.rsis not in the final diff. That change came from #591, as the PR description itself says.2f94e7desays "replacing channel with None in the key left the whole suite green". I made that exact mutation at HEAD andgenerate_consolidated_writes_htmlandcommand_coverage_distinguishes_channels_on_same_platformboth fail. Both tests already exist at the base.fc84b70then says the opposite, and both bodies would land.295a0a6says the move-channel-to-front mutation is covered by those same two tests. I ran that mutation at HEAD and both of them pass; only the new sort-key test fails.6829d4esays the opposite.fc84b70and2f94e7decite review-round SHAs (2f94e7de,1c20e7a2) that will not exist after a squash.fc84b70also narrates "a fresh Opus review (fanned out to two workers)", which is agent-tooling text in a public commit body (AGENTS.md §11, "avoid AI-generated boilerplate").3f7ffa6is a bare "style: cargo fmt" commit.
The code is the side that governs here, because these are claims about the code. So the fix goes in the messages, not the code.
Confidence 100 · mechanical · Fix: squash or reword the branch into one or two commits that describe the final change. The PR description is already a good basis. Alternatively, the merger rewrites the squash body by hand, but nothing enforces that.
Non-blocking
-
[standing in part since 2f94e7d] New test comments describe how this PR evolved rather than the code —
crates/e2e-report/src/components.rs:268-272,crates/e2e-report/src/single_report.rs:101-104,crates/e2e-report/src/consolidated.rs:2003-2029- "A prior version of this test asserted only doctype/title/style…" refers to a version that existed only inside this PR (
330e39a). After the merge it refers to something that was never on main. single_report.rssaysgenerate"had no test of its own" from inside the test that is now that test.fc84b70says it dropped "the stale clauses from both comments", but this one is still there.- The roughly 27-line sort-key comment is accurate (I checked each claim), but it is written as a log of rounds ("so a fourth row … is added").
Confidence 75 · mechanical · Fix: say what each test pins, in the present tense, and drop the history.
- "A prior version of this test asserted only doctype/title/style…" refers to a version that existed only inside this PR (
Decisions for the author
None
Positive signals
- The sort-key fixture rows are chosen so the correct key and each wrong key give different orders. I confirmed this by mutation: moving channel first, swapping known_bugs/channel, and swapping os/platform each fail the new test. Dropping channel is caught by the two existing tests.
STYLEwent frompub(crate)to private oncepage_shellbecame its only user, so a third generator can't bypass the shared shell by accident.
Deployment notes
None
Standing objections from the 2f94e7d round
The blind review did not re-find either item below. Each was then put to the reviewer after its findings were frozen, together with the author's reply. That check is anchored evidence, not a review. Both items are dropped because the blind silence and that check agree:
- False sole-coverage claims in the
single_report.rs/components.rstest comments: resolved. Both false sentences are gone at fc84b70, which I confirmed by reading both files. The commit-message half of that objection still holds and is part of the blocking item above. - Description test count and missing mentions: resolved. The description says 57, and 57 ran locally. The
page_shellbullet lists every assertion, and thelabeldoc rewrite is now mentioned. Not a finding, an observation from that check: thelabelbullet calls the one-line content description "restored", but compared with main it was never removed. The diff a reader sees is a rewrite that adds the consumer notes.
What this covered
- What was read: all of
components.rs,consolidated.rsandsingle_report.rsat HEAD, pluscommand_coverage.rsandlib.rs, and the callers of every moved symbol. Diffc1c2aeb3d…fc84b70e(merge-base withprw-base). None of the 28 commitsprw-basehas beyond the merge-base touchcrates/e2e-report. - Fan-out: one code pass, plus one combined pass for history, agent instructions, code comments and commit messages. Both ran as independent read-only workers.
- Behaviour preservation: checked by reading. The shell markup is identical, and the sort tuple and channel suffix are character-for-character the expressions they replace. No other page shell or copy of the sort tuple remains in the repo.
- Local runs:
cargo test -p e2e-report --all-targets: 57 passed, and the three new tests ran by name.clippy -D warnings: clean.cargo fmt --check: clean.- Mutation runs on
platform_sort_keyandplatform_cell, as cited above.
- Verdicts on the added tests:
- All three would still pass with the extraction reverted to inline code. That is expected, because this is a refactor that does not change behaviour.
- In
platform_sort_key, breaking any single key element fails at least one test. - A wrong
page_shell(body, lang, charset or style dropped) failspage_shell_carries_title_and_style. That was traced by reading, not run. - Gap: nothing pins the consolidated call site's title argument, because the same string also appears in its
h1. This was already true before the PR, so it is not reported as a finding.
- CI at fc84b70:
- No check failed.
- "Sphinx docs build (-W)" and "Skill checks (skillscope)" were skipped by their path filters.
- "E2E tests (MI300X)" was still queued for its self-hosted runner. That is waiting on a runner, not a failure in this PR.
- Design review (Step 7): run by the driver, not by separate workers. The questions it raised produced nothing worth reporting.
- Prompt injection: none found.
- Discussion: the review's findings were not reconciled against the PR's discussion. The only exception is the standing-objection check above.
fc84b70 to
7aa7338
Compare
rominf
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 7aa7338
This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.
Gate: nothing blocks at this head. The change request from the fc84b70 round (commit messages) was already dismissed by the author; this round re-checked it and agrees it no longer holds — see the standing-objections section. No change request is filed. All findings below are new this round and non-blocking.
Review — no blocking findings
Full review of the whole change.
The change says it is three behaviour-preserving dedups in crates/e2e-report. It names ROCMAI-27 and ROCMAI-200 as where they came from, not as tickets it closes, and it closes no GitHub issue. The diff does what the description claims. Rendered output being unchanged was checked by a run, not just read.
Blocking
None
Non-blocking
-
[new] No test pins the consolidated report's
<title>, so the new wiring at that call site is unchecked —crates/e2e-report/src/consolidated.rs:939
The mutantpage_shell("Consolidated Report", body)passes all 57 tests.generate_consolidated_writes_htmlonly checkshtml.contains("Consolidated E2E Report"), and theh1also matches that. The new comment inpage_shell_carries_title_and_stylepoints out this exact ambiguity. Only the single-report call site got a call-site<title>assertion, ingenerate_renders_the_body_inside_the_shared_page_shell. The gap existed before this change and the change keeps it rather than causing it, so it is non-blocking. But the title is now a separate argument that can drift from theh1.
Confidence 100 · logic · Fix: ingenerate_consolidated_writes_html, addassert!(html.contains("<title>Consolidated E2E Report</title>")). -
[new] One copy of the effective-channel rule is left, and the new helpers depend on it —
crates/e2e-report/src/consolidated.rs:757-759(againsteffective_channel()at:777)
PlatformReport::loadstill inlinesversions.channel.as_deref().or(desc.channel_from_name)to buildlabel. Meanwhileplatform_sort_keyandplatform_cellnow go througheffective_channel(). Changing the fallback rule in one place would leavelabelout of step with the sort and the matrix cell.labelfeeds the command-coverage column dedup, the XPASS/Regression notes and the detail sections. Mutation check: swappingeffective_channel()fordesc.channel_from_namein either new helper survives the whole suite, because every fixture's manifest channel equals its artifact-name channel. This existed before the change but has the same twin-copy pattern the change set out to remove (an earlier non-blocking review on #591 flagged the same pattern).
Confidence 85 · mechanical · Fix: computelabelfrom the same accessor, or a free function shared witheffective_channel(). Add one fixture where the manifest channel and the artifact-name channel differ. -
[new] The test-plan text overstates how existing tests cover
platform_cell— the commit message and the PR description ("existing tests already exercise both call sites' rendered output byte-for-byte")
consolidated_summary_markdown_has_a_row_per_platformandgenerate_consolidated_writes_htmlassert substrings (md.contains("| MI300X | Linux | 1 | …"),html.contains("Mi300x (release)")). They do not compare the output byte-for-byte. The coverage itself is real: makingplatform_cellinclude the OS or drop the channel is killed by 3–5 tests. The code governs here, so the wording should be corrected, not the tests changed. This text lands on main through the squash's commit messages.
Confidence 85 · mechanical · Fix: replace "byte-for-byte" with "by row content", or similar. -
[new] The module doc in
components.rsis now stale —crates/e2e-report/src/components.rs:5-7
It says "Shared maud HTML fragment rendering, plus the CSS and timestamp helpers". The module now also owns the whole-page shell, andSTYLEis private to it.
Confidence 85 · mechanical · Fix: mentionpage_shellin the module doc. -
[new] A broken line in the rewritten
labeldoc comment —crates/e2e-report/src/consolidated.rs:169-170
"(see\n///command_coverage.rs)" is split mid-phrase. The comment's claims are accurate:command_coverage.rs:154,163useslabelas the column dedup key, and the "other two" consumers are at:1043/:1050and:1444.
Confidence 85 · mechanical · Fix: reflow the comment.
Decisions for the author
- The PR title leaves out the third dedup — tradeoff
The PR title is "share HTML page shell and platform sort key". The commit subject also names "platform cell". On squash, the PR title usually becomes the squash subject. Keeping it short is fine, but matching the commit subject would describe the change fully.
Positive signals
platform_sort_keyreturns a tuple of borrows ((&str, &str, bool, Option<&str>)), so both sort sites stay allocation-free while sharing one definition.STYLEgoes frompub(crate)to private now thatpage_shellis its only user. That makes the shell the only way to embed the stylesheet.- The sort-key test is built so that each tuple position is pinned by a dedicated row. Mutation confirmed it: dropping, swapping or negating any element is killed by that test. The one exception is dropping
channel, which other named tests kill, as the test's own comment says.
Deployment notes
None
Standing objections from the fc84b70 round
The blind review above did not re-find either item below. Each was then put to the reviewer after its findings were frozen, together with the author's reply ("Fixed the found issues."). That check is anchored evidence, not a review. Both are dropped because the blind silence and that check agree:
- Blocking — commit messages with false/contradictory claims, review-round SHAs and agent-tooling narration: resolved. The branch is now a single commit (7aa7338). It disclaims the
labelreuse as #591's work, drops both contradictory mutation claims, credits the two existing tests only with channel's tie-break effect (matching the mutation run: dropping channel is killed by exactly those two tests, moving it earlier by the new sort-key test), and carries no review-round SHAs, review narration, or barecargo fmtcommit. The one remaining overstatement ("byte-for-byte") is a different, smaller claim and is reported above as a new non-blocking finding. - Non-blocking — test comments narrating how the PR evolved: resolved. No "prior version", "had no test" or round-by-round wording remains in the diff.
What this covered
All three changed files, read completely: components.rs, consolidated.rs (all 2840 lines including tests) and single_report.rs. Also read: command_coverage.rs (the label consumer), every caller of page_shell, with_channel_suffix, effective_channel and label, and a repo-wide search for other page shells or sort keys. Reviewed against prw-base: 9f2b3f3…7aa73385, merge-base c1c2aeb. No e2e-report changes landed on the base after the merge-base. A full review was run because the fc84b70 review point is no longer in the branch's history.
Evidence that was run:
cargo test -p e2e-report --all-targets: 57 passed, and all 3 new tests ran by name. Clippy-D warningsandfmt --checkwere clean.- Rendered HTML and markdown were compared with the production change reverted and at HEAD, for 5 fixtures (consolidated, empty consolidated, markdown, single, empty single). After normalising timestamps they were byte-identical.
- 27 single-branch mutants were run. The 3 survivors are listed in the findings.
Verdicts on the new tests:
generate_renders_the_body_inside_the_shared_page_shelland the sort-key test both pass with the production change reverted. That is expected: this is a behaviour-preserving refactor, so they are characterisation tests.page_shell_carries_title_and_stylecannot exist without the change.- Broken-branch survivors are as listed above.
CI at 7aa7338: every completed check passed. "Sphinx docs build (-W)" and "Skill checks (skillscope)" were skipped by their path filters. "E2E tests (MI300X)" is still queued for its self-hosted runner — missing evidence, not a failure in this PR.
All passes ran. There were two independent workers: a coupled code pass with mutation testing, and a pass covering project rules, history, review comments on earlier PRs touching these files (#460, #433, #591) and code comments. The design questions were answered by the review driver, not an independent worker, because the change is a mechanical refactor and nothing raised a design-level question. Nothing in the content looked like prompt injection.
This review was not reconciled against this PR's discussion, except for the standing-objection check above.
|
Addressed all five non-blocking findings from the 7aa7338 review (commit cf53bc5, pushed):
The PR title/description tradeoff note (naming only two of the three dedups) is left as-is — noted as a non-blocking decision for the author, not something to fix. |
cf53bc5 to
a93b01c
Compare
|
🔴 Automated review · pr-review-watcher · cf53bc5 This automation never files a GitHub approval, so no approving review will Informational: this describes Review — needs work (at cf53bc5)Full review of the whole change. Blocking
Non-blocking
Decisions for the author
Positive signals
Deployment notes
What this covered
|
tomastola
left a comment
There was a problem hiding this comment.
Review of commit cf53bc59 — the head has since moved to a93b01cf (rebased onto main, which now includes #592; the PR's consconsolidated.rs diff differs from what I read). This review and approval cover cf53bc59 only; the rebased commit was not re-read.
Review — no blocking findings
Full review of the whole change (3 files, +340/−126), read from the PR diff and head-ref file contents. Ticket ROCMAI-200 is a backlog inbox for cleanups; the PR does one batch of those and does not claim to close it.
Blocking
None
Non-blocking
- The PR title understates the change — PR title vs.
consolidated.rs(resolve_channel,platform_cell)
The title says "share HTML page shell and platform sort key". The diff also addsresolve_channel()andplatform_cell(), and rewrites thePlatformReport::labeldoc comment. The body covers all of this accurately, so the description does not misstate scope; only the title, which survives into the squash/merge subject, omits two of the five changes. Confidence 70 · mechanical · Fix: add "platform cell and channel fallback" to the title. - The test plan carries review-round history — PR body, last-but-one test-plan bullet
"Restored the one-line content description … after an earlier round of this PR dropped it" describes the PR's own iteration and tells a later reader nothing about the change as merged. Confidence 70 · mechanical · Fix: drop the bullet.
Decisions for the author
- The sort-key test pins only the markdown call site — tradeoff
platform_sort_key_orders_by_platform_then_os_then_known_bugs_then_channelgoes throughconsolidated_summary_markdown;generate_consolidated's call to the same function has no ordering test of its own. Both sites call one function, so pinning one pins the function and the test is correct as written. If someone later re-inlines the HTML sort, nothing fails — that is the drift this PR removes. The alternative is one cheap HTML-order assertion. Confirm the omission is deliberate. - The sort-key test's comment is about 30 lines of prose for 7 fixtures — non-blocking-improvement
Every claim in it checks out againstparse_descriptor(e.g.e2e-gpu-known-bugs-nightly-reportdoes resolve to MI300X, known_bugs=true, nightly). But it is more likely to rot than the assertions are; table-driving the fixtures as(artifact, scenario_count, expected_rank)would shrink it. - Two comments now disagree about how live
known_bugsis — non-blocking-improvement
Descriptor::known_bugssays legacy artifacts "no longer exist"; the newlabeldoc warns about "a known-bugs marker". The code is unaffected. If the flag is truly legacy, thelabelcomment's example could use something else.
Positive signals
- Narrowing
STYLEto private, now thatpage_shellis its only user, makes the shared stylesheet impossible to bypass from other modules. page_shell_carries_title_and_styleasserts<title>directly; a substring check would be satisfied by the<h1>, and the PR also tightensgenerate_consolidated_writes_htmlfor the same reason.- The
labeldoc comment was checked against its consumers and every claim holds:command_coverage.rsusesr.labelas the column de-duplication key,consolidated.rsuses it in the XPASS/Regression notes and the per-platform detail headings. - The refactor is behaviour-preserving as claimed: identical markup element order in both generators, a stable sort over an unchanged tuple, and
resolve_channelis the sameorfallback as before.
Deployment notes
None
What this covered
- Read: the full diff and the head-ref text of
components.rs,consolidated.rsandsingle_report.rsatcf53bc59;command_coverage.rsfor thelabelconsumers; the PR body and ticket ROCMAI-200 afterwards. #460 and #591 are merged and are what the body says. - CI at
cf53bc59: every check passed or was skipped (skillscopeand Sphinx skip on Rust-only paths). - Not run: the new tests were not run locally, so there is no mutation check. They pin behaviour of a behaviour-preserving refactor, so "passes with the production change reverted" is expected for them. The sort-key test's fixture claims were verified by reading
parse_descriptor. - Not independent: this ran as a single context rather than a fan-out; treat it as a single-context review. History/blame and prior-changes passes were folded into that one pass, not run separately.
- Not reconciled against any review discussion.
- Not covered: the rebased head
a93b01cfand the#592interaction inconsolidated.rs.
Posted as an approval at the engineer's explicit direction.
Two pure, behavior-preserving dedups in crates/e2e-report, found during ROCMAI-27 code review and logged in ROCMAI-200 (the epic's quality-findings backlog): - single_report.rs::generate() and consolidated.rs::generate_consolidated() each inlined their own full `<!DOCTYPE html>...` page shell. Extract a shared page_shell() in components.rs taking the body markup and page title; both generators now call it. STYLE narrows from pub(crate) to private now that page_shell is its only caller. - The "platform (+ OS)" label was independently rebuilt at two more sites: the markdown summary table and the HTML matrix_table each rebuilt the platform-only half (the same pattern that drifted once already, fixed in ROCm#460). Extract a shared platform_cell() for that platform-only half and use it at both call sites. A third dedup originally targeted here — the platform-ordering sort key, duplicated between generate_consolidated() and consolidated_summary_markdown() — landed independently on main as report_sort_key() via ROCm#592 (ROCMAI-484) while this branch was in review. Rebasing onto that picked up the existing extraction instead of adding a second one; this commit adds no new sort-key function. No rendering output or sort order changes: same markup, same ordering, less duplication to drift out of sync in a future head-level or ordering tweak. Adds page_shell_carries_title_and_style (components.rs), pinning every shared piece page_shell renders: doctype, lang="en", meta charset="utf-8", title, stylesheet, and the caller's body. Adds generate_renders_the_body_inside_the_shared_page_shell (single_report.rs), pinning that single_report::generate's real body — the scenario name parsed from the report JSON — reaches the shared shell at that call site. Adds consolidated_summary_markdown_orders_rows_by_report_sort_key (consolidated.rs), going through the public consolidated_summary_markdown entry point with realistic artifact-name fixtures rather than a direct call to report_sort_key, so a regression in load_platform_reports' use of report_sort_key is caught even if report_sort_key itself stays correct in isolation. Pins all four sort-key tuple elements in precedence order: a fourth fixture row on a different platform pins platform; two Ubuntu rows pin os and known_bugs; a fifth/sixth row on two more platforms with opposite channels pin that channel doesn't outrank platform; a seventh row pins that channel doesn't outrank known_bugs either. channel's own effect as a tie-breaker, once platform/os/known_bugs are equal, is pinned by generate_consolidated_writes_html and command_coverage_distinguishes_channels_on_same_platform instead of being repeated here. report_sort_key_orders_by_every_dimension (pre-existing, landed with ROCm#592) already pins the same four dimensions via a direct call with synthetic PlatformReports; this test complements it rather than repeating it. PlatformReport::label's doc comment states what the label contains (platform, OS and effective channel) alongside its consumers. No new test for platform_cell() itself: existing tests already exercise both call sites' rendered output by row content (consolidated_summary_markdown_has_a_row_per_platform and generate_consolidated_writes_html) and pass unchanged. command_coverage_markdown's own duplication of the full platform+OS label, and its reuse of PlatformReport::label, already landed separately on main through ROCm#591 — not part of this change. cargo test -p e2e-report --all-targets: 65 passed, 0 failed. cargo clippy -p e2e-report --all-targets -- -D warnings: clean. cargo fmt --check: clean. cargo build --workspace: clean, no downstream breakage (xtask consumes this crate's public API). Signed-off-by: Jussi Elo <jussi.elo@amd.com>
- pin the consolidated report's <title> separately from its <h1>, which happened to share the same string - route PlatformReport::load's channel fallback through the same resolve_channel() rule effective_channel() uses, instead of duplicating it - mention page_shell in the components.rs module doc - reflow a doc comment line that split mid-phrase Signed-off-by: Jussi Elo <jussi.elo@amd.com>
a93b01c to
968d7dd
Compare
Summary
Three pure, behavior-preserving dedups in
crates/e2e-report, found during ROCMAI-27 code review and logged in ROCMAI-200 (the epic's quality-findings backlog):single_report.rs::generate()andconsolidated.rs::generate_consolidated()each inlined their own full<!DOCTYPE html>...page shell. Extracted a sharedpage_shell()incomponents.rstaking the body markup and page title; both generators now call it.consolidated.rs's platform-ordering sort key was duplicated verbatim betweengenerate_consolidated()andconsolidated_summary_markdown(). Extractedplatform_sort_key()and used it at both call sites.matrix_tableeach rebuilt the platform-only half (the same pattern that drifted once already, fixed in ci(e2e): channel-aware report columns and nightly matrix axis (ROCMAI-429) #460). Extracted a sharedplatform_cell()for that platform-only half and used it at both call sites. (command_coverage_markdown's own duplication of the full platform+OS label, and its reuse ofPlatformReport::label, already landed separately on main through ROCMAI-486: split command-coverage audit out of consolidated.rs #591 — not part of this diff.)Also closes a smaller twin-copy of the same pattern found during review:
PlatformReport::loadrebuilt the manifest-channel-wins-over-artifact-name-channel fallback rule inline, duplicatingPlatformReport::effective_channel(). Both now go through oneresolve_channel()function.No rendering output or sort order change — same markup, same ordering, less duplication to drift out of sync in a future head-level or ordering tweak.
Test plan
cargo test -p e2e-report --all-targets— all 57 tests passcargo clippy -p e2e-report --all-targets -- -D warnings— cleancargo build --workspace— clean, no downstream breakage (xtask consumes this crate's public API)page_shell_carries_title_and_style(components.rs), asserting every shared piecepage_shellrenders — doctype,lang="en",meta charset="utf-8", title, stylesheet, and the caller's bodygenerate_consolidated_writes_htmlnow asserts<title>Consolidated E2E Report</title>directly, not just a substring that also happens to match the<h1>generate_renders_the_body_inside_the_shared_page_shell(single_report.rs), pinning thatsingle_report::generate's real body — the scenario name parsed from the report JSON — actually reaches the shared shell at that call siteplatform_sort_key_orders_by_platform_then_os_then_known_bugs_then_channel(consolidated.rs) pins all four tuple elements in precedence order: a fourth row on a different platform pinsplatform; the two Ubuntu rows pinosandknown_bugs; a fifth/sixth row on two more platforms with opposite channels pin thatchanneldoesn't outrankplatform; a seventh row pins thatchanneldoesn't outrankknown_bugseither.channel's own effect as a tie-breaker, once platform/os/known_bugs are equal, is pinned bygenerate_consolidated_writes_htmlandcommand_coverage_distinguishes_channels_on_same_platforminstead of being repeated herePlatformReport::label's doc comment, alongside its consumer note, after an earlier round of this PR dropped itSTYLEfrompub(crate)to private (components.rs) now thatpage_shellis its only callerplatform_cell()reuse: existing tests already exercise both call sites' rendered output by row content (consolidated_summary_markdown_has_a_row_per_platformandgenerate_consolidated_writes_html) and pass unchanged