Repository navigation
fix(xtask): harden arch-doc checker's possessive-owner narrowing - #493
jussielo-amd wants to merge 7 commits into
Conversation
8e893c6 to
6d9a61b
Compare
|
🔴 Automated review · pr-review-watcher · 14be823 This automation never files a GitHub approval, so no approving review will SummaryThe PR loosens the architecture-doc checker's possessive-owner narrowing: a clause now continues through any text until the sentence ends, and non-path asides no longer break it. It also rejects prose slash-pairs such as Reviewed: the whole change, Verified:
Blocking: 0 · Non-blocking: 10. Since the previous round (b939036)
🚫 Blocking (must fix before merge)None. Non-blocking
|
rominf
left a comment
There was a problem hiding this comment.
Automated review · pr-review-watcher · 6d9a61b
Requesting changes for two defects; details are in the review comment on this PR.
xtask/src/architecture_doc.rs(aside branch for non-path code spans): it now clears onlyinter_text, so a stalelast_code_spansurvives and a'safter a type name (`AgentClient`'s `lib.rs`) narrows the file to an earlier, unrelated directory. That is the over-narrowing direction the checker exists to prevent. Resolve by clearinglast_code_spanin that branch and adding the two reproductions from the report as regression tests.- New behaviour that tests do not pin: removing the possessive reset for hyphenated owners, switching
alltoanyinis_prose_slash_phrase, or dropping the end-of-text sentence-boundary case each leaves every test green, andslash_path_with_only_one_prose_side_is_still_a_candidatecannot fail as written. Resolve by giving that test a mixed input (e.g.docs/and) and adding tests forcontains_a_sentence_boundaryat end of text and for a hyphenated owner after an earlier sentence.
rominf
left a comment
There was a problem hiding this comment.
Automated review · pr-review-watcher · f612531
Requesting changes for one remaining defect. The details are in the review comment on this PR. Both objections from the previous change request are now fixed and verified.
xtask/src/architecture_doc.rs (the pending-hyphenated-span branch, which clears inter_text): commit 443d36d says a sentence-ending period must survive two clears, the type-name aside and the pending hyphenated span. Only the aside case is tested. If you add sentence_boundary_seen = false; after this branch's inter_text.clear(), all 60 architecture_doc tests still pass, and the masking leak this flag exists to prevent comes back.
Example: under heading `crates/rocm-dash-core`, `rocm-dash-daemon`, `rocm-dash-tui`, the input "`rocm-dash-tui`'s `a.rs` is done. See `rocm-dash-daemon` docs and `b.rs`." then narrows b.rs to rocm-dash-tui, so a stale b.rs elsewhere would pass CI silently.
To resolve, add that input as a regression test asserting that b.rs keeps the full heading list. The test passes at f612531 and fails under the mutation.
Both objections from this review are fixed at f612531: the aside branch now clears last_code_span, and the four untested behaviours are now pinned by tests. Each was verified by mutation. Superseded by a new, narrower change request for one remaining test gap.
|
Checked the failing check: E2E tests (MI300X). 4 unexpected failures, all in GPU-serving scenarios: This PR only touches Confirmed pre-existing: the same |
|
Added the regression test in 67c7e68: `extract_path_citations_does_not_leak_an_owner_past_a_sentence_boundary_hidden_by_a_discarded_hyphenated_span`. It reproduces the exact case from the review (a sentence boundary before a bare hyphenated word that's discarded as prose, followed by a scoped citation) and asserts `b.rs` falls back to the full heading list rather than narrowing to the stale owner. Confirmed it fails if `sentence_boundary_seen = false;` is added after the discard branch's `inter_text.clear()`, and passes with current behavior (61/61 `architecture_doc` tests green). |
|
Addressed 3 of the 10 non-blocking items from the full review in b73e99a:
Left the rest as-is:
64/64 `architecture_doc` tests pass, `cargo fmt`/`clippy -D warnings` clean, `cargo xtask check-architecture-doc` passes on the real doc. Each new test verified to fail under its corresponding single-line mutation. |
rominf
left a comment
There was a problem hiding this comment.
Automated review · pr-review-watcher · b73e99a
Requesting changes for one defect. The details are in the review comment on this PR. The earlier objection, at f612531, is fixed and verified by mutation, so that change request is being withdrawn separately.
docs/architecture.md:42: this branch is behind main. Since it forked, #476 (b1cbc19) rewrote this same paragraph and split app/mod.rs into app/types.rs, app/event_loop.rs, app/scrollbar.rs and app/actions.rs. This PR's version says "app/mod.rs itself is not yet modularized further", which is false on main. The PR now conflicts with main in this paragraph. If that conflict is resolved by keeping this side, check-architecture-doc still passes, because every file it cites exists. CI would then let the false claim land and drop #476's description.
To resolve: rebase onto current main and keep #476's paragraph. This PR's checker passes it unchanged.
b73e99a to
b939036
Compare
|
Rebased onto current Verified post-rebase: |
rominf
left a comment
There was a problem hiding this comment.
Automated review · pr-review-watcher · b939036
Requesting changes for one defect. The details are in the report comment on this PR.
The earlier objection, filed at b73e99a about docs/architecture.md:42, is resolved: the rebase keeps #476's paragraph. That change request is being withdrawn separately.
The new defect is the PR description and commit 9e625e0.
- Both still say this PR reverts the
rocm-dash-tuiparagraph indocs/architecture.mdto natural prose "as the end-to-end proof". - No commit in the range touches that file.
- The repo squash-merges with commit messages, so that false claim would land in
main's history. Fixes #492would also close #492 with that last step of the issue undone.
To resolve, do one of these:
extract_path_citations's possessive-owner narrowing only recognized exactly 's/slash/and as literal connector text between two Code spans, and fully reset all chain state on any intervening non-path backtick span (a type name). Real prose describing a multi-file split embeds exactly those things -- explanatory asides and free phrases like "was split into" -- so every citation after the first lost its narrowing, as happened when ROCm#456 merged into main while ROCm#432 was in review, breaking CI on both the architecture-doc gate and the Windows run_passes_against_the_real_doc test. Make the clause continue through any intervening text within the same sentence instead of requiring a literal connector before every citation; a sentence-ending period still hard-stops it unconditionally -- the guard against a later, unrelated sentence silently inheriting an earlier owner, which is the one masking risk this scoping feature exists to prevent. A naive "any period" check would misfire on a de-backticked filename's own extension dot (the de-backticked agent.rs sits inside the very clause it must not break), so sentence-ending is specifically a period followed by whitespace or end-of-text, not one glued to surrounding characters. Part of ROCm#492; the docs/architecture.md revert that proves this against the real doc is a separate commit later in this branch. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
is_path_candidate accepted any slash-containing span unconditionally, so a future doc edit using a common shorthand pair (`read/write`, `and/or`, `GPU/CPU`) would false-fail CI. Reject a slash-path whose every segment is a known prose word, while still accepting a real path with only one ordinary-word segment (`apps/rocm`). Also fix continues_possessive_clause: it inferred a sentence boundary from inter_text, but a non-candidate aside span (a type name) or a pending hyphenated span clears inter_text first, discarding an already-seen sentence-ending period before the check ever ran. A period right before such an aside was silently lost, letting a possessive owner wrongly narrow a later, unrelated citation past its sentence boundary. Track the boundary with a persistent flag instead, reset only where a clause's owner is actually (re)confirmed. Fixes ROCm#440. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…te parity contains_a_sentence_boundary (renamed from ends_with_a_sentence_boundary, which checked anywhere in the text, not just the end) treated any "period followed by whitespace" as a sentence end, so "e.g. "/"i.e. "/ "etc. " wrongly ended a possessive clause early. The real doc already uses "e.g." near the checker's own module doc, so a future doc edit using it inside a possessive clause would fall back to the heading's full crate list instead of narrowing - a false CI failure on an accurate edit, the exact class of bug this scoping feature exists to prevent. Guard against a small, curated list of abbreviations (e.g, i.e, etc), matched against the alphanumerics-and-periods token the candidate period ends, so surrounding punctuation like a leading `(` doesn't need separate stripping. Also closes a gap the ROCm#440 fix left open: is_path_candidate rejects a slash-joined prose pair (read/write, and/or, GPU/CPU), but is_directory_shaped didn't get the same guard, so a heading citing such a span would still be collected as a directory. Both now share the is_prose_slash_phrase check. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
A type-name aside (e.g. an identifier) right before a possessive `'s` used to leave `last_code_span` pointing at whatever directory-shaped span preceded it, so `'s` could pick up that stale, unrelated owner instead of correctly finding none. Clear `last_code_span` in the aside branch alongside `inter_text`. Also pins two gaps the review found in the existing test suite: a `is_prose_slash_phrase` regression test whose input couldn't actually distinguish `all` from `any`, and missing coverage for the hyphenated-owner `sentence_boundary_seen` reset when a new clause starts right after an earlier sentence in the same paragraph. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Mirrors the existing aside-branch regression test for the pending- hyphenated-span discard path: sentence_boundary_seen must survive that inter_text.clear() too, or a later citation narrows to a stale owner instead of falling back to the heading's full list. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
… gap Non-blocking findings from the last automated review: - contains_a_sentence_boundary only checked '.', so a clause ending in '?' or '!' kept carrying its owner into the next sentence. - The sentence_boundary_seen reset at the end of the main citation Code arm had no dedicated unit test, only indirect coverage via run_passes_against_the_real_doc. - is_path_candidate's doc comment claimed one blind spot remained, undercounting the PROSE_SLASH_WORDS gap described two lines below. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
ROCm#432 shipped a lead-in sentence chaining every rocm-dash-tui filename with literal 's/slash/and connectors, to unblock its own CI the moment ROCm#456's natural-prose version of this paragraph broke the then-stricter possessive-owner narrowing. With that narrowing now hardened to continue through ordinary prose, the workaround is no longer needed -- and leaving it in place meant the real doc never exercised the new continuation logic at all, since every citation still resolved through the old literal-connector chain. Revert to natural prose: each sentence that needs narrowing opens with its own explicit `rocm-dash-tui's` anchor directly before its first citation (the exact-match possessive step), and later citations in the same sentence rely on the new continue-through-any-text behavior, including through explanatory asides like `` `AgentClient` `` and `` `MouseEvent` `` -- exactly the shape ROCm#456's original prose had, and exactly what broke CI under the old narrower checker. This is the end-to-end proof ROCm#492 asked for: `cargo xtask check-architecture-doc` now passes this paragraph via the new continuation logic against the real doc, not just the unit tests. Fixes ROCm#492. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
b939036 to
14be823
Compare
|
Addressed the blocking defect from the review at b939036 (PR comment 5415245022): the description and commit Went with option (a) — actually did the revert, rather than just rewording the claim away, since #492's own proposal names it as the closing step and it's the only way the real doc exercises the new continuation logic (dropping the lead slash-chain sentence, each remaining What changed, pushed as
Verified: |
|
CI: |
|
Re-reviewed at 14be823: no blocking findings, 10 non-blocking. The earlier blocker is resolved and the change request is withdrawn. The full report is in the updated review comment: #493 (comment) |
rominf
left a comment
There was a problem hiding this comment.
Requesting changes on the approach, not the code. The automated rounds above checked this PR's logic, and I have no new defect to report in it. My question is whether we should keep building this checker at all.
The cost. architecture_doc.rs is about 1,500 lines on main and grows to about 2,070 with this PR. Almost all of that infers two things from prose: which backtick spans are paths, and which crate a bare agent.rs belongs to. Each round of that inference has needed another heuristic: possessive-owner chains, sentence boundaries, an abbreviation list (e.g./i.e./etc.), a prose-slash word list for #440, and a persistent flag to survive buffer clears. The description already lists the next gaps. Clause continuation isn't connector-aware ("but"), and five interdependent mutable locals need a refactor "before the next heuristic is added". This is a natural-language parser for one document, and it will keep growing.
What to do instead: make citations explicit, then use an off-the-shelf link checker. If the doc writes a citation as a relative markdown link, nothing has to be inferred:
[`agent/clients.rs`](../crates/rocm-dash-tui/src/agent/clients.rs)lychee (Rust, MIT/Apache-2.0) checks these offline. I tried it locally (v0.24.2, lychee --offline --include-fragments docs/) on a sample doc in this shape:
- it flagged a renamed file, a missing directory, and a broken
#anchor, and exited 2 - it ignored prose like
read/write,e.g.,and/or, andAgentClientby construction, because only links are candidates - the whole check took about 3 ms
This also removes the bare-main.rs scoping problem (17 files share that name). A link carries its full path, so no heading context or owner narrowing is needed. Readers also get clickable paths on GitHub, which helps a doc whose banner tells them to go verify file boundaries.
What it would take:
- Rewrite the path citations in
docs/architecture.mdas relative links. It's mechanical, and the paragraph at line 42 is a good showcase. - Add a lychee step to CI.
lycheeverse/lychee-actionexists; pin it to a full SHA per our action-pinning rule. Running it repo-wide ('./**/*.md') is almost free: today it finds 25 OK and 2 errors across our markdown, and both errors areSECURITY.md's root-relative/security/advisories/newGitHub URL, which needs an--exclude(or--root-dir). - Optionally, keep a tiny xtask lint that fails on a bare
`*.rs`citation outside a link, so new citations don't drift back to plain backticks. That's a regex, not a parser. - Delete
architecture_doc.rsand its CI job. #492 and #440 are then fixed by removal rather than by more heuristics.
A stronger variant for per-module detail is rustdoc intra-doc links ([crate::therock]) under -D rustdoc::broken_intra_doc_links, which also catches renamed modules and items, not just files. That suits the module notes better than the top-level map, so I'd treat it as a follow-up, not part of this change.
If you'd rather land this PR as a stop-gap to unblock #492, I'm open to that. Please open a tracking issue for the lychee migration and link it from the description, so the deferred refactor doesn't become the next heuristic layer instead.
|
Before going further on replacing this checker's approach with lychee, I want to flag something so we can align on scope. lychee is a link/URL checker — it extracts hyperlinks ( More importantly, the problem this checker solves isn't just "does a path exist" — it's "does this specific crate's file still exist," because the repo has 17 files literally named A straight swap would mean either:
Did you have a narrower scope in mind — e.g. lychee for actual hyperlinks/external URLs elsewhere in the docs, as a separate and additive check, rather than replacing this checker's path-citation logic? Want to make sure we're solving the same problem before planning any changes here. |
|
Answering the scope question directly: no, I meant the full replacement — step 4 of the proposal was "delete I think we're reading the proposal differently on one point. You're right that lychee wouldn't see these citations as the doc is written today — but rewriting them is step 1, not a side effect. The code block in my review was the proposed citation shape: [`agent/clients.rs`](../crates/rocm-dash-tui/src/agent/clients.rs)So "lychee can't reproduce heading-scoping" is true and, I'd argue, not a loss. Heading-scoping exists because a bare On the one new point — that the author "would just be hand-writing the correct path," so lychee only confirms an already-correct path resolves. That's fair, but it separates two failure modes:
Trading automatic detection of born-wrong for deleting ~2,000 lines of prose inference seems clearly worth it to me, especially as the born-wrong case is the one review is good at. On readability: the doc already hand-writes full repo-relative paths in running prose in four places today — Scope is 51 distinct The offer from my review stands: if you'd rather land this as a stop-gap to unblock #492, I'm fine with that — just open a tracking issue for the migration and link it from the description, so the deferred refactor doesn't turn into the next heuristic layer. |
|
Opened the tracking issue for the lychee migration: #578. It covers the proposal, the migration steps, and the scope discussion from this thread (full replacement, the drift-vs-born-wrong distinction, and the rustdoc intra-doc-links follow-up as a separate, later issue). Linked it from this PR's description as well. Going with your offer to land this PR as the stop-gap — further lychee discussion can continue on #578 rather than here. |
Pull request was closed
* ci: add lychee gate for markdown links (ROCm#578) docs/architecture.md's path citations are about to move to explicit markdown links instead of bare backtick filenames, checked by an off-the-shelf link checker instead of a hand-rolled parser (see ROCm#578). This adds that checker now, additively: it only looks at links that already exist, so it can land independently of the citation rewrite and doesn't conflict with ROCm#493/ROCm#564, which are still editing docs/architecture.md's prose. Scoped offline (no network) to avoid external-host flakiness, and excludes docs/rocm-docs/ (Sphinx/MyST source whose cross-doc anchors are resolved by the docs-build job, not plain markdown). Signed-off-by: Jussi Elo <jussi.elo@amd.com> * fix(docs-links): correct overclaim, document the new CI gate lychee.toml's comment said it replaces architecture_doc.rs's checker; that checker is untouched and still runs, so say "first step toward replacing" instead. Also add the docs-links gate to CONTRIBUTING.md's list of enforced CI invariants, next to check-architecture-doc. Signed-off-by: Jussi Elo <jussi.elo@amd.com> * fix(docs-links): narrow CONTRIBUTING.md's link-coverage claim The docs-links line said every link in the repo is checked, but lychee.toml runs offline (no external https:// links) and excludes docs/rocm-docs/ (Sphinx/MyST source, covered by docs-build instead). Say what the job actually covers and point at lychee.toml for the exclusion list, instead of overclaiming. Signed-off-by: Jussi Elo <jussi.elo@amd.com> --------- Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Summary
extract_path_citations's possessive-owner narrowing (added in feat(xtask): guard docs/architecture.md path citations in CI #432) only recognized exactly's///andas literal connector text between two Code spans, and fully reset all chain state on any intervening non-path backtick span (a type name). Real prose describing a multi-file split embeds exactly those things — explanatory asides and free phrases like "was split into" — so every citation after the first lost its narrowing.mainwhile feat(xtask): guard docs/architecture.md path citations in CI #432 was in review: it broke CI on both the architecture-doc gate and the Windowsrun_passes_against_the_real_doctest the moment the two PRs' content combined. feat(xtask): guard docs/architecture.md path citations in CI #432 shipped an immediate workaround (an awkward slash-chained lead-in sentence) to unblock its own CI.agent.rssits inside the very clause it must not break), so "sentence-ending" specifically means a period followed by whitespace or end-of-text, not one glued to surrounding characters.docs/architecture.md'srocm-dash-tuiparagraph from feat(xtask): guard docs/architecture.md path citations in CI #432's slash-chain workaround back to natural prose: each sentence that needs narrowing opens with its own explicitrocm-dash-tui'sanchor directly before its first citation, and later citations in that sentence rely on the new continue-through-any-text behavior, including through explanatory asides (`AgentClient`,`MouseEvent`) — this is the end-to-end proof that the new logic handles the real doc, not just synthetic unit tests.extract_path_citations_does_not_leak_an_owner_across_a_sentence_boundaryregression test.is_path_candidateaccepted any slash-containing span unconditionally, so a future doc edit using a common shorthand pair (`read/write`,`and/or`,`GPU/CPU`) would false-fail CI. Now rejects a slash-path whose every segment is a known prose word (a small curated list, same bounded-tradeoff shape asBARE_FILE_EXTENSIONS), while still accepting a real path with only one ordinary-word segment (apps/rocm).continues_possessive_clauseinferred "has a sentence ended" from theinter_textbuffer, but a non-candidate aside span (a type name) or a pending hyphenated span clears that buffer first — silently discarding an already-seen sentence-ending period before the check ever ran. A period immediately followed by an aside (`rocm-dash-tui`'s `agent.rs` is done. `AgentClient` tracks `metrics.rs`.) let a possessive owner wrongly narrow a later, unrelated citation. Fixed by tracking the boundary with a persistent flag that survives those clears, reset only where a clause's owner is actually (re)confirmed.`rocm-dash-tui`'s `agent.rs` stayed put, but `metrics.rs` moved elsewherewould still narrowmetrics.rstorocm-dash-tui). This is the same deliberate "a single sentence is a safe enough unit to trust" tradeoff described above, just a reminder that it isn't semantic — only a new sentence or heading reliably resets the owner.contains_a_sentence_boundary, since it scans anywhere in the text, not just its end) treated any "period + whitespace" as a sentence end, so an abbreviation like`e.g.`/`i.e.`/`etc.`— which the doc's own intro paragraph already uses nearby — would wrongly end a clause early and fall an otherwise-narrowable citation back to the heading's full crate list. Fixed with a small curated abbreviation list, matched against the alphanumerics-and-periods token the candidate period ends (so a leading(doesn't need separate stripping). (2)is_path_candidate's xtask check-architecture-doc: guard against slash-containing prose false-failing CI #440 fix wasn't mirrored inis_directory_shaped, so a heading citing a prose slash-phrase (`read/write`) would still be collected as a directory even though the same span is rejected as a citation elsewhere — both now share theis_prose_slash_phrasecheck.extract_path_citationsthreads five interdependent mutable locals (inter_text,last_code_span,possessive_owner,sentence_boundary_seen,pending_hyphenated) through one loop via areset_inline!macro, and this PR's own fixes are two bugs from exactly that shape (a clear silently dropping a flag it shouldn't have). Consolidating this into a struct that makes "what survives an aside vs. a hard reset" checkable in one place is worth doing before the next heuristic is added, but is a larger refactor than this PR's scope.Fixes #492, #440.
This lands as a stop-gap within the current heuristic approach. Per review (#493 (review)), the longer-term plan is to replace
architecture_doc.rs's prose inference with explicit markdown links checked by lychee — tracked in #578.Test plan
cargo test -p xtask architecture_doc— 64/64 pass, including regression tests for aside-tolerant narrowing, natural comma/and lists, the sentence-boundary masking guard, prose-slash-word rejection for xtask check-architecture-doc: guard against slash-containing prose false-failing CI #440, the aside-hidden sentence-boundary fix, the abbreviation guard,is_directory_shaped/is_path_candidateparity, and a leak case across a discarded hyphenated span.cargo xtask check-architecture-doc— passes against the real doc, now exercising the new continuation logic (not just the old literal-connector path) on the revertedrocm-dash-tuiparagraph.cargo test -p xtask— 271/271 pass.cargo clippy -p xtask --all-targets -- -D warnings/cargo fmt -p xtask -- --check— clean.