Skip to content

fix(xtask): harden arch-doc checker's possessive-owner narrowing - #493

Closed
jussielo-amd wants to merge 7 commits into
ROCm:mainfrom
jussielo-amd:issue-492-arch-doc-connector-robustness
Closed

jussielo-amd wants to merge 7 commits into
ROCm:mainfrom
jussielo-amd:issue-492-arch-doc-connector-robustness

Conversation

@jussielo-amd

@jussielo-amd jussielo-amd commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • extract_path_citations's possessive-owner narrowing (added in feat(xtask): guard docs/architecture.md path citations in CI #432) only recognized exactly 's///and as literal connector text between two Code spans, and fully reset all chain state on any intervening non-path backtick span (a type name). Real prose describing a multi-file split embeds exactly those things — explanatory asides and free phrases like "was split into" — so every citation after the first lost its narrowing.
  • That's exactly what happened when ROCMAI-51: Modularize crates/rocm-dash-tui/src/agent.rs #456 merged into main while feat(xtask): guard docs/architecture.md path citations in CI #432 was in review: it broke CI on both the architecture-doc gate and the Windows run_passes_against_the_real_doc test the moment the two PRs' content combined. feat(xtask): guard docs/architecture.md path citations in CI #432 shipped an immediate workaround (an awkward slash-chained lead-in sentence) to unblock its own CI.
  • This PR fixes the checker itself: a possessive clause now continues through any intervening text within the same sentence, rather than requiring a literal connector directly before every citation. A sentence-ending period still hard-stops the clause unconditionally — the guard against a later, unrelated sentence silently inheriting an earlier owner (the masking failure mode this scoping feature exists to prevent).
  • A naive "any period ends the clause" check would misfire on a de-backticked filename's own extension dot (agent.rs sits inside the very clause it must not break), so "sentence-ending" specifically means a period followed by whitespace or end-of-text, not one glued to surrounding characters.
  • Reverts docs/architecture.md's rocm-dash-tui paragraph from feat(xtask): guard docs/architecture.md path citations in CI #432's slash-chain workaround back to natural prose: each sentence that needs narrowing opens with its own explicit rocm-dash-tui's anchor directly before its first citation, and later citations in that sentence rely on the new continue-through-any-text behavior, including through explanatory asides (`AgentClient`, `MouseEvent`) — this is the end-to-end proof that the new logic handles the real doc, not just synthetic unit tests.
  • Considered and rejected a simpler "sticky owner for the rest of the heading" design: it would reintroduce the exact masking risk this checker exists to prevent (an unrelated file from a different crate in the same heading silently inheriting the wrong owner). Sentence-scoped persistence is the safe middle ground — see the new extract_path_citations_does_not_leak_an_owner_across_a_sentence_boundary regression test.
  • Also fixes xtask check-architecture-doc: guard against slash-containing prose false-failing CI #440: is_path_candidate accepted any slash-containing span unconditionally, so a future doc edit using a common shorthand pair (`read/write`, `and/or`, `GPU/CPU`) would false-fail CI. Now rejects a slash-path whose every segment is a known prose word (a small curated list, same bounded-tradeoff shape as BARE_FILE_EXTENSIONS), while still accepting a real path with only one ordinary-word segment (apps/rocm).
  • A follow-up review caught a correctness bug in the sentence-boundary guard itself: continues_possessive_clause inferred "has a sentence ended" from the inter_text buffer, but a non-candidate aside span (a type name) or a pending hyphenated span clears that buffer first — silently discarding an already-seen sentence-ending period before the check ever ran. A period immediately followed by an aside (`rocm-dash-tui`'s `agent.rs` is done. `AgentClient` tracks `metrics.rs`.) let a possessive owner wrongly narrow a later, unrelated citation. Fixed by tracking the boundary with a persistent flag that survives those clears, reset only where a clause's owner is actually (re)confirmed.
  • Known, accepted limitation (flagged in review, not fixed here): clause continuation isn't connector-aware — any text within the same sentence keeps the owner alive, even a contrastive word like "but" (e.g. `rocm-dash-tui`'s `agent.rs` stayed put, but `metrics.rs` moved elsewhere would still narrow metrics.rs to rocm-dash-tui). This is the same deliberate "a single sentence is a safe enough unit to trust" tradeoff described above, just a reminder that it isn't semantic — only a new sentence or heading reliably resets the owner.
  • A second review round caught two more correctness gaps: (1) the sentence-boundary check (renamed contains_a_sentence_boundary, since it scans anywhere in the text, not just its end) treated any "period + whitespace" as a sentence end, so an abbreviation like `e.g.`/`i.e.`/`etc.` — which the doc's own intro paragraph already uses nearby — would wrongly end a clause early and fall an otherwise-narrowable citation back to the heading's full crate list. Fixed with a small curated abbreviation list, matched against the alphanumerics-and-periods token the candidate period ends (so a leading ( doesn't need separate stripping). (2) is_path_candidate's xtask check-architecture-doc: guard against slash-containing prose false-failing CI #440 fix wasn't mirrored in is_directory_shaped, so a heading citing a prose slash-phrase (`read/write`) would still be collected as a directory even though the same span is rejected as a citation elsewhere — both now share the is_prose_slash_phrase check.
  • Deferred design note (flagged in the same review round, not addressed here): extract_path_citations threads five interdependent mutable locals (inter_text, last_code_span, possessive_owner, sentence_boundary_seen, pending_hyphenated) through one loop via a reset_inline! macro, and this PR's own fixes are two bugs from exactly that shape (a clear silently dropping a flag it shouldn't have). Consolidating this into a struct that makes "what survives an aside vs. a hard reset" checkable in one place is worth doing before the next heuristic is added, but is a larger refactor than this PR's scope.

Fixes #492, #440.

This lands as a stop-gap within the current heuristic approach. Per review (#493 (review)), the longer-term plan is to replace architecture_doc.rs's prose inference with explicit markdown links checked by lychee — tracked in #578.

Test plan

  • cargo test -p xtask architecture_doc — 64/64 pass, including regression tests for aside-tolerant narrowing, natural comma/and lists, the sentence-boundary masking guard, prose-slash-word rejection for xtask check-architecture-doc: guard against slash-containing prose false-failing CI #440, the aside-hidden sentence-boundary fix, the abbreviation guard, is_directory_shaped/is_path_candidate parity, and a leak case across a discarded hyphenated span.
  • cargo xtask check-architecture-doc — passes against the real doc, now exercising the new continuation logic (not just the old literal-connector path) on the reverted rocm-dash-tui paragraph.
  • cargo test -p xtask — 271/271 pass.
  • cargo clippy -p xtask --all-targets -- -D warnings / cargo fmt -p xtask -- --check — clean.

@jussielo-amd
jussielo-amd requested a review from a team as a code owner October 2, 2026 07:16
@jussielo-amd
jussielo-amd force-pushed the issue-492-arch-doc-connector-robustness branch from 8e893c6 to 6d9a61b Compare October 2, 2026 08:45
@rominf

rominf commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

🔴 Automated review · pr-review-watcher · 14be823

This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.

Summary

The PR loosens the architecture-doc checker's possessive-owner narrowing: a clause now continues through any text until the sentence ends, and non-path asides no longer break it. It also rejects prose slash-pairs such as read/write (#440), and rewrites the rocm-dash-tui paragraph in docs/architecture.md back into natural prose. Outcome: No blocking findings. The sentence-boundary gap below is the one I would most like fixed before merge.

Reviewed: the whole change, prw-base...HEAD (7 commits): xtask/src/architecture_doc.rs and docs/architecture.md, plus their callers, the docs that describe the checker, and every commit message. This was a full review, because the rebase removed the previous review point.

Verified:

  • cargo test -p xtask passes (271 tests), and cargo xtask check-architecture-doc passes against the real doc.
  • The base and head extractors were compiled side by side and run on both versions of the doc:
    • On the base doc they give identical citations.
    • On the new doc the base checker fails on 10 citations, which fall back to the full four-crate list. The head checker narrows them correctly, so the "end-to-end proof" claim holds.

Blocking: 0 · Non-blocking: 10.

Since the previous round (b939036)

  • The previous blocker is resolved, as your reply says. That blocker was that the PR text and commit 9e625e0 described a docs/architecture.md revert that was not in the diff. At this head:

    • commit 14be823b makes the doc change, and it is the one carrying Fixes #492;
    • the first commit (bdb025de) now says the revert is a separate, later commit;
    • the description matches the diff, including the 64/64 count.

    The change request filed at b939036 is being withdrawn.

  • A finding kept non-blocking: the review agent rated the sentence-boundary gap (first item below) as blocking. Earlier rounds already reported the .) / ." part of it as non-blocking. xtask/src/architecture_doc.rs is byte-identical to the file at b939036, so nothing has changed there to make it a defect now. The tight-list-bullet case is a new observation on that same unchanged code, and it is listed alongside.

  • The Windows CI re-run you mentioned needs nothing from this side.

🚫 Blocking (must fix before merge)

None.

Non-blocking

  • xtask/src/architecture_doc.rs:340-366, 492-497 — the comment at :299-301 says a sentence-ending ./?/! "always" ends the clause, but the code misses several sentence and block endings. In each of these cases prw-base gives the full crate list and HEAD wrongly narrows to rocm-dash-tui. This is the silent, masking direction, so I recommend fixing it here.

    • A period followed by a closing " or ), as in `a.rs` is "done." Then `c.rs`. The real doc already has this shape at "reducer's reason to exist."; it is harmless there today only because no owner is active at that point.
    • Tight list bullets: Tag::Item never calls reset_inline!, so an owner carries from one bullet into the next.

    The fix: skip closing )/"/'/]/*/_ before the whitespace test, reset on Start/End(Tag::Item), and add a test for each shape.

  • xtask/src/architecture_doc.rs:616 — sentence_boundary_seen = false runs for every path candidate, but the comments at :319-324 and :436-440 say it is reset "only where a clause's owner is actually (re)confirmed". This is safe, because possessive_owner is reset in the same step, so the comment is what should change.

  • xtask/src/architecture_doc.rs:340 — etc. is never treated as a sentence end, so in …`b.rs`, etc. Then `c.rs`… the owner carries into the next sentence. The doc comment only admits the opposite gap.

  • xtask/src/architecture_doc.rs:347-352 — "end of text" is the end of a pulldown Text chunk, not of the sentence. A period glued to a following inline span (v1.`x`) ends the clause early. This fails loudly rather than masking anything, but the comment presents end-of-text as end of sentence.

  • xtask/src/architecture_doc.rs:577-583 — a non-.rs path citation mid-clause ((see `docs/x.md`) and `c.rs`) still drops the owner, while a type-name aside does not. This contradicts "continues through any text" and is not documented.

  • xtask/src/architecture_doc.rs:326 — continues_possessive_clause is a const fn that returns !flag under a roughly 40-line comment. The explanation belongs on the flag.

  • xtask/src/architecture_doc.rs tests — i.e. and etc. are never exercised; only e.g. is. Removing either list entry would leave the suite green.

  • docs/architecture.md:42 — "agent/mod.rs keeps the AgentClient seam (AgentError, …) that the old agent.rs was split into" has three problems:

    • it reads as though agent.rs was split into the seam;
    • it recasts AgentError/StateSnapshot/InferenceParams/REQUEST_TIMEOUT as parts of the seam;
    • "into an app/ directory … into app/types.rs" repeats "into".
  • docs/architecture.md:42 — the rewrite drops the "are every file this phase's dashboard-TUI split touches" summary, and the commit message does not mention that.

  • Commit 14be823b message — it says each sentence that needs narrowing opens with its own anchor. The app/mod.rs keeps only… and …re-exports from app/mod.rs sentences have no anchor; they pass through the remembered narrowed_owners. It also calls the change a "revert", but the text is new prose rather than ROCMAI-51: Modularize crates/rocm-dash-tui/src/agent.rs #456's paragraph.

rominf
rominf previously requested changes Oct 2, 2026

@rominf rominf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review · pr-review-watcher · 6d9a61b

Requesting changes for two defects; details are in the review comment on this PR.

  1. xtask/src/architecture_doc.rs (aside branch for non-path code spans): it now clears only inter_text, so a stale last_code_span survives and a 's after a type name (`AgentClient`'s `lib.rs`) narrows the file to an earlier, unrelated directory. That is the over-narrowing direction the checker exists to prevent. Resolve by clearing last_code_span in that branch and adding the two reproductions from the report as regression tests.
  2. New behaviour that tests do not pin: removing the possessive reset for hyphenated owners, switching all to any in is_prose_slash_phrase, or dropping the end-of-text sentence-boundary case each leaves every test green, and slash_path_with_only_one_prose_side_is_still_a_candidate cannot fail as written. Resolve by giving that test a mixed input (e.g. docs/and) and adding tests for contains_a_sentence_boundary at end of text and for a hyphenated owner after an earlier sentence.

rominf
rominf previously requested changes Oct 5, 2026

@rominf rominf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review · pr-review-watcher · f612531

Requesting changes for one remaining defect. The details are in the review comment on this PR. Both objections from the previous change request are now fixed and verified.

xtask/src/architecture_doc.rs (the pending-hyphenated-span branch, which clears inter_text): commit 443d36d says a sentence-ending period must survive two clears, the type-name aside and the pending hyphenated span. Only the aside case is tested. If you add sentence_boundary_seen = false; after this branch's inter_text.clear(), all 60 architecture_doc tests still pass, and the masking leak this flag exists to prevent comes back.

Example: under heading `crates/rocm-dash-core`, `rocm-dash-daemon`, `rocm-dash-tui`, the input "`rocm-dash-tui`'s `a.rs` is done. See `rocm-dash-daemon` docs and `b.rs`." then narrows b.rs to rocm-dash-tui, so a stale b.rs elsewhere would pass CI silently.

To resolve, add that input as a regression test asserting that b.rs keeps the full heading list. The test passes at f612531 and fails under the mutation.

@rominf
rominf dismissed their stale review October 5, 2026 07:14

Both objections from this review are fixed at f612531: the aside branch now clears last_code_span, and the four untested behaviours are now pinned by tests. Each was verified by mutation. Superseded by a new, narrower change request for one remaining test gap.

@jussielo-amd

Copy link
Copy Markdown
Collaborator Author

Checked the failing check: E2E tests (MI300X).

4 unexpected failures, all in GPU-serving scenarios: bench-load-real-serve, chat-end-to-end-local-model, serve-vllm-inference, serve-vllm-default-on-instinct.

This PR only touches docs/architecture.md and xtask/src/architecture_doc.rs (the arch-doc checker) — nothing related to serving or vLLM — so these can't be caused by this change.

Confirmed pre-existing: the same bench-load-real-serve / chat-end-to-end-local-model / serve-vllm-inference scenarios failed on the MI300X runner 3 days ago on an unrelated docs-only PR (run 37052848311), so this looks like an ongoing hardware/runner issue rather than something introduced here. No action taken on this PR.

@jussielo-amd

Copy link
Copy Markdown
Collaborator Author

Added the regression test in 67c7e68: `extract_path_citations_does_not_leak_an_owner_past_a_sentence_boundary_hidden_by_a_discarded_hyphenated_span`.

It reproduces the exact case from the review (a sentence boundary before a bare hyphenated word that's discarded as prose, followed by a scoped citation) and asserts `b.rs` falls back to the full heading list rather than narrowing to the stale owner. Confirmed it fails if `sentence_boundary_seen = false;` is added after the discard branch's `inter_text.clear()`, and passes with current behavior (61/61 `architecture_doc` tests green).

@jussielo-amd

Copy link
Copy Markdown
Collaborator Author

Addressed 3 of the 10 non-blocking items from the full review in b73e99a:

  • `:358` — `contains_a_sentence_boundary` now also recognizes `?`/`!` as sentence boundaries, not just `.`. Added a direct unit test plus an end-to-end regression (`"... is done! `state.rs` other."` no longer inherits the prior owner).
  • `:608` — added a dedicated unit test for the `sentence_boundary_seen` reset at the end of the main citation `Code` arm (previously only indirectly covered by `run_passes_against_the_real_doc`), using a slash-qualified owner confirmed right after an earlier sentence. Confirmed it fails if that reset is removed.
  • `:179` — reworded `is_path_candidate`'s doc comment from "one deliberate blind spot remains" to acknowledge the second one (unlisted `PROSE_SLASH_WORDS` pairs) described two lines below it.

Left the rest as-is:

  • `:306`, `:457` — already-accepted tradeoffs / harmless dead code per your own notes, no action needed.
  • `docs/architecture.md:42` — a commit-message wording nit on an already-pushed commit; not rewriting published history for it.
  • `:341`, `:605`, `:567`, `:486` — real inconsistencies but each is a logic/design call (etc.-abbreviation ambiguity, aside-transparency parity, owner-shape parity, SoftBreak semantics) rather than a quick fix, so leaving those for a follow-up discussion rather than bundling them into this PR.

64/64 `architecture_doc` tests pass, `cargo fmt`/`clippy -D warnings` clean, `cargo xtask check-architecture-doc` passes on the real doc. Each new test verified to fail under its corresponding single-line mutation.

rominf
rominf previously requested changes Oct 5, 2026

@rominf rominf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review · pr-review-watcher · b73e99a

Requesting changes for one defect. The details are in the review comment on this PR. The earlier objection, at f612531, is fixed and verified by mutation, so that change request is being withdrawn separately.

docs/architecture.md:42: this branch is behind main. Since it forked, #476 (b1cbc19) rewrote this same paragraph and split app/mod.rs into app/types.rs, app/event_loop.rs, app/scrollbar.rs and app/actions.rs. This PR's version says "app/mod.rs itself is not yet modularized further", which is false on main. The PR now conflicts with main in this paragraph. If that conflict is resolved by keeping this side, check-architecture-doc still passes, because every file it cites exists. CI would then let the false claim land and drop #476's description.

To resolve: rebase onto current main and keep #476's paragraph. This PR's checker passes it unchanged.

@rominf
rominf dismissed their stale review October 5, 2026 10:54

Resolved at b73e99a: the regression test added in 67c7e68 covers the pending-hyphenated-span case and fails under the mutation named in this review. A separate change request covers the stale doc paragraph.

@jussielo-amd
jussielo-amd force-pushed the issue-492-arch-doc-connector-robustness branch from b73e99a to b939036 Compare October 5, 2026 12:19
@jussielo-amd

Copy link
Copy Markdown
Collaborator Author

Rebased onto current main (b73e99a → b939036 replayed on 5c4f401d). Resolved the docs/architecture.md:42 conflict by keeping #476's paragraph (the already-modularized app/types.rs/app/event_loop.rs/app/scrollbar.rs/app/actions.rs split) — this PR's checker passes it unchanged, as predicted. No other conflicts.

Verified post-rebase: cargo test -p xtask (271 passed), cargo clippy -p xtask --all-targets -- -D warnings clean, cargo fmt -p xtask -- --check clean, cargo xtask check-architecture-doc passes.

@jussielo-amd
jussielo-amd enabled auto-merge October 5, 2026 12:20
rominf
rominf previously requested changes Oct 5, 2026

@rominf rominf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review · pr-review-watcher · b939036

Requesting changes for one defect. The details are in the report comment on this PR.

The earlier objection, filed at b73e99a about docs/architecture.md:42, is resolved: the rebase keeps #476's paragraph. That change request is being withdrawn separately.

The new defect is the PR description and commit 9e625e0.

  • Both still say this PR reverts the rocm-dash-tui paragraph in docs/architecture.md to natural prose "as the end-to-end proof".
  • No commit in the range touches that file.
  • The repo squash-merges with commit messages, so that false claim would land in main's history.
  • Fixes #492 would also close #492 with that last step of the issue undone.

To resolve, do one of these:

  • Re-apply the doc change on top of #476's paragraph, with an owner anchor in each sentence that needs one.
  • Or reword 9e625e0's message and the description so they no longer claim the revert, and stop Fixes #492 from auto-closing the issue, or note on #492 where the doc follow-up is tracked.

@rominf
rominf dismissed their stale review October 5, 2026 13:31

Resolved at b939036: the rebase keeps #476's docs/architecture.md paragraph and the PR no longer edits that file. A separate change request (5415245022) covers a new, unrelated item.

extract_path_citations's possessive-owner narrowing only recognized
exactly 's/slash/and as literal connector text between two Code spans,
and fully reset all chain state on any intervening non-path backtick
span (a type name). Real prose describing a multi-file split embeds
exactly those things -- explanatory asides and free phrases like "was
split into" -- so every citation after the first lost its narrowing,
as happened when ROCm#456 merged into main while ROCm#432 was in review,
breaking CI on both the architecture-doc gate and the Windows
run_passes_against_the_real_doc test.

Make the clause continue through any intervening text within the same
sentence instead of requiring a literal connector before every
citation; a sentence-ending period still hard-stops it unconditionally
-- the guard against a later, unrelated sentence silently inheriting
an earlier owner, which is the one masking risk this scoping feature
exists to prevent. A naive "any period" check would misfire on a
de-backticked filename's own extension dot (the de-backticked agent.rs
sits inside the very clause it must not break), so sentence-ending is
specifically a period followed by whitespace or end-of-text, not one
glued to surrounding characters.

Part of ROCm#492; the docs/architecture.md revert that proves this against
the real doc is a separate commit later in this branch.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>
is_path_candidate accepted any slash-containing span unconditionally,
so a future doc edit using a common shorthand pair (`read/write`,
`and/or`, `GPU/CPU`) would false-fail CI. Reject a slash-path whose
every segment is a known prose word, while still accepting a real
path with only one ordinary-word segment (`apps/rocm`).

Also fix continues_possessive_clause: it inferred a sentence boundary
from inter_text, but a non-candidate aside span (a type name) or a
pending hyphenated span clears inter_text first, discarding an
already-seen sentence-ending period before the check ever ran. A
period right before such an aside was silently lost, letting a
possessive owner wrongly narrow a later, unrelated citation past its
sentence boundary. Track the boundary with a persistent flag instead,
reset only where a clause's owner is actually (re)confirmed.

Fixes ROCm#440.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…te parity

contains_a_sentence_boundary (renamed from ends_with_a_sentence_boundary,
which checked anywhere in the text, not just the end) treated any
"period followed by whitespace" as a sentence end, so "e.g. "/"i.e. "/
"etc. " wrongly ended a possessive clause early. The real doc already
uses "e.g." near the checker's own module doc, so a future doc edit
using it inside a possessive clause would fall back to the heading's
full crate list instead of narrowing - a false CI failure on an
accurate edit, the exact class of bug this scoping feature exists to
prevent. Guard against a small, curated list of abbreviations (e.g,
i.e, etc), matched against the alphanumerics-and-periods token the
candidate period ends, so surrounding punctuation like a leading `(`
doesn't need separate stripping.

Also closes a gap the ROCm#440 fix left open: is_path_candidate rejects a
slash-joined prose pair (read/write, and/or, GPU/CPU), but
is_directory_shaped didn't get the same guard, so a heading citing such
a span would still be collected as a directory. Both now share the
is_prose_slash_phrase check.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>
A type-name aside (e.g. an identifier) right before a possessive `'s`
used to leave `last_code_span` pointing at whatever directory-shaped
span preceded it, so `'s` could pick up that stale, unrelated owner
instead of correctly finding none. Clear `last_code_span` in the aside
branch alongside `inter_text`.

Also pins two gaps the review found in the existing test suite: a
`is_prose_slash_phrase` regression test whose input couldn't actually
distinguish `all` from `any`, and missing coverage for the
hyphenated-owner `sentence_boundary_seen` reset when a new clause
starts right after an earlier sentence in the same paragraph.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Mirrors the existing aside-branch regression test for the pending-
hyphenated-span discard path: sentence_boundary_seen must survive
that inter_text.clear() too, or a later citation narrows to a stale
owner instead of falling back to the heading's full list.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>
… gap

Non-blocking findings from the last automated review:

- contains_a_sentence_boundary only checked '.', so a clause ending in
  '?' or '!' kept carrying its owner into the next sentence.
- The sentence_boundary_seen reset at the end of the main citation
  Code arm had no dedicated unit test, only indirect coverage via
  run_passes_against_the_real_doc.
- is_path_candidate's doc comment claimed one blind spot remained,
  undercounting the PROSE_SLASH_WORDS gap described two lines below.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>
ROCm#432 shipped a lead-in sentence chaining every rocm-dash-tui filename
with literal 's/slash/and connectors, to unblock its own CI the moment
ROCm#456's natural-prose version of this paragraph broke the then-stricter
possessive-owner narrowing. With that narrowing now hardened to
continue through ordinary prose, the workaround is no longer needed --
and leaving it in place meant the real doc never exercised the new
continuation logic at all, since every citation still resolved through
the old literal-connector chain.

Revert to natural prose: each sentence that needs narrowing opens with
its own explicit `rocm-dash-tui's` anchor directly before its first
citation (the exact-match possessive step), and later citations in the
same sentence rely on the new continue-through-any-text behavior,
including through explanatory asides like `` `AgentClient` `` and
`` `MouseEvent` `` -- exactly the shape ROCm#456's original prose had, and
exactly what broke CI under the old narrower checker.

This is the end-to-end proof ROCm#492 asked for: `cargo xtask
check-architecture-doc` now passes this paragraph via the new
continuation logic against the real doc, not just the unit tests.

Fixes ROCm#492.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>
@jussielo-amd
jussielo-amd force-pushed the issue-492-arch-doc-connector-robustness branch from b939036 to 14be823 Compare October 6, 2026 05:55
@jussielo-amd

Copy link
Copy Markdown
Collaborator Author

Addressed the blocking defect from the review at b939036 (PR comment 5415245022): the description and commit 9e625e0 claimed a docs/architecture.md revert that the rebase had dropped from the diff.

Went with option (a) — actually did the revert, rather than just rewording the claim away, since #492's own proposal names it as the closing step and it's the only way the real doc exercises the new continuation logic (dropping the lead slash-chain sentence, each remaining .rs citation now resolves either via its own `rocm-dash-tui's` anchor or by continuing an active clause through ordinary prose/asides, not through the old literal-connector chain).

What changed, pushed as 14be823 (rebuilt on 5c4f401, replacing b9390367):

  • Reworded 9e625e0 to drop the false revert claim and move Fixes #492 to where the revert actually lands.
  • Replayed the five unchanged follow-up commits as-is.
  • Added a new commit that does the actual docs/architecture.md revert, with Fixes #492.
  • Updated the PR description to match (and fixed the stale "56/56" test count to the actual 64).

Verified: cargo test -p xtask (271/271), cargo test -p xtask architecture_doc (64/64), cargo xtask check-architecture-doc passes the real doc, cargo clippy -p xtask --all-targets -- -D warnings and cargo fmt -p xtask -- --check clean. Confirmed the rewritten paragraph exercises the new continuation path by tracing the parser state through it (several .rs citations now continue past non-connector asides like `AgentClient` and `MouseEvent` that the old literal-connector check would have rejected).

@jussielo-amd

Copy link
Copy Markdown
Collaborator Author

CI: windows-build-and-test failed on the first run after the push (tests::restart_refuses_a_public_service_without_a_key_before_stopping_it, a Windows temp-dir creation error — os error 3 — in apps/rocm/src/main.rs, a file this PR never touches). Re-ran the job and it passed; all 24 checks are green.

@jussielo-amd
jussielo-amd requested a review from rominf October 6, 2026 07:50
@rominf
rominf dismissed their stale review October 6, 2026 08:11

Objection resolved at 14be823: the docs/architecture.md change is now in the diff (commit 14be823, which carries Fixes #492), and the PR description and commit messages match it. See the updated report comment.

@rominf

rominf commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator

Re-reviewed at 14be823: no blocking findings, 10 non-blocking. The earlier blocker is resolved and the change request is withdrawn. The full report is in the updated review comment: #493 (comment)

@rominf rominf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on the approach, not the code. The automated rounds above checked this PR's logic, and I have no new defect to report in it. My question is whether we should keep building this checker at all.

The cost. architecture_doc.rs is about 1,500 lines on main and grows to about 2,070 with this PR. Almost all of that infers two things from prose: which backtick spans are paths, and which crate a bare agent.rs belongs to. Each round of that inference has needed another heuristic: possessive-owner chains, sentence boundaries, an abbreviation list (e.g./i.e./etc.), a prose-slash word list for #440, and a persistent flag to survive buffer clears. The description already lists the next gaps. Clause continuation isn't connector-aware ("but"), and five interdependent mutable locals need a refactor "before the next heuristic is added". This is a natural-language parser for one document, and it will keep growing.

What to do instead: make citations explicit, then use an off-the-shelf link checker. If the doc writes a citation as a relative markdown link, nothing has to be inferred:

[`agent/clients.rs`](../crates/rocm-dash-tui/src/agent/clients.rs)

lychee (Rust, MIT/Apache-2.0) checks these offline. I tried it locally (v0.24.2, lychee --offline --include-fragments docs/) on a sample doc in this shape:

  • it flagged a renamed file, a missing directory, and a broken #anchor, and exited 2
  • it ignored prose like read/write, e.g., and/or, and AgentClient by construction, because only links are candidates
  • the whole check took about 3 ms

This also removes the bare-main.rs scoping problem (17 files share that name). A link carries its full path, so no heading context or owner narrowing is needed. Readers also get clickable paths on GitHub, which helps a doc whose banner tells them to go verify file boundaries.

What it would take:

  1. Rewrite the path citations in docs/architecture.md as relative links. It's mechanical, and the paragraph at line 42 is a good showcase.
  2. Add a lychee step to CI. lycheeverse/lychee-action exists; pin it to a full SHA per our action-pinning rule. Running it repo-wide ('./**/*.md') is almost free: today it finds 25 OK and 2 errors across our markdown, and both errors are SECURITY.md's root-relative /security/advisories/new GitHub URL, which needs an --exclude (or --root-dir).
  3. Optionally, keep a tiny xtask lint that fails on a bare `*.rs` citation outside a link, so new citations don't drift back to plain backticks. That's a regex, not a parser.
  4. Delete architecture_doc.rs and its CI job. #492 and #440 are then fixed by removal rather than by more heuristics.

A stronger variant for per-module detail is rustdoc intra-doc links ([crate::therock]) under -D rustdoc::broken_intra_doc_links, which also catches renamed modules and items, not just files. That suits the module notes better than the top-level map, so I'd treat it as a follow-up, not part of this change.

If you'd rather land this PR as a stop-gap to unblock #492, I'm open to that. Please open a tracking issue for the lychee migration and link it from the description, so the deferred refactor doesn't become the next heuristic layer instead.

@jussielo-amd

Copy link
Copy Markdown
Collaborator Author

Before going further on replacing this checker's approach with lychee, I want to flag something so we can align on scope.

lychee is a link/URL checker — it extracts hyperlinks ([text](url)-style links or bare URLs) and checks they resolve. The citations this checker validates are backtick-quoted plain text (e.g. `main.rs`), not markdown links, so lychee wouldn't pick them up at all as the doc is written today.

More importantly, the problem this checker solves isn't just "does a path exist" — it's "does this specific crate's file still exist," because the repo has 17 files literally named main.rs/lib.rs (5 main.rs, 12 lib.rs). citation_exists's heading-scoping (section_dirs) and this PR's possessive-owner narrowing exist specifically so a stale citation under one crate's heading can't hide behind some other crate's same-named file still existing. A generic link checker has no notion of "which heading is this citation under," so it can't reproduce that.

A straight swap would mean either:

Did you have a narrower scope in mind — e.g. lychee for actual hyperlinks/external URLs elsewhere in the docs, as a separate and additive check, rather than replacing this checker's path-citation logic? Want to make sure we're solving the same problem before planning any changes here.

@rominf

rominf commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator

Answering the scope question directly: no, I meant the full replacement — step 4 of the proposal was "delete architecture_doc.rs and its CI job," not an additive URL check alongside it.

I think we're reading the proposal differently on one point. You're right that lychee wouldn't see these citations as the doc is written today — but rewriting them is step 1, not a side effect. The code block in my review was the proposed citation shape:

[`agent/clients.rs`](../crates/rocm-dash-tui/src/agent/clients.rs)

So "lychee can't reproduce heading-scoping" is true and, I'd argue, not a loss. Heading-scoping exists because a bare `agent.rs` doesn't say which crate it means — it's machinery to recover information the citation threw away. A link carries the full path, so there's nothing left to infer. The 17-files-named-main.rs problem is the strongest argument for explicit paths: that's exactly the ambiguity a path resolves for free, and the one the possessive-owner chains, sentence boundaries, and abbreviation list are all there to resolve by inference instead.

On the one new point — that the author "would just be hand-writing the correct path," so lychee only confirms an already-correct path resolves. That's fair, but it separates two failure modes:

  • Drift — a file moves or is deleted as the code evolves. This is what the gate exists for, and an explicit link catches it exactly, with no inference.
  • Born-wrong — someone writes a path that resolves but belongs to a different crate than the heading claims. Today's checker catches this; a link checker wouldn't. But a born-wrong citation is a literal path sitting in the diff for review, and it never becomes wrong — it starts wrong.

Trading automatic detection of born-wrong for deleting ~2,000 lines of prose inference seems clearly worth it to me, especially as the born-wrong case is the one review is good at.

On readability: the doc already hand-writes full repo-relative paths in running prose in four places today — apps/rocm/src/therock.rs, apps/rocm/src/automations.rs, crates/rocm-dash-tui/src/ui/approval.rs, xtask/src/architecture_doc.rs — and nobody has flagged those as a prose cost. The link form adds the target but keeps the rendered text identical to the backtick it replaces, and readers get clickable paths on a page whose own banner tells them to go verify file boundaries. The prose has also already paid a readability cost in the other direction: #432 shipped the slash-chained lead-in sentence to satisfy the parser, and part of this PR is undoing it. A checker you rewrite sentences to please isn't readability-neutral either.

Scope is 51 distinct .rs citations (35 bare, 16 already slash-qualified) — mechanical, and the current checker can generate most of the rewrite, since it already resolves each citation to a concrete file.

The offer from my review stands: if you'd rather land this as a stop-gap to unblock #492, I'm fine with that — just open a tracking issue for the migration and link it from the description, so the deferred refactor doesn't turn into the next heuristic layer.

@jussielo-amd

Copy link
Copy Markdown
Collaborator Author

Opened the tracking issue for the lychee migration: #578. It covers the proposal, the migration steps, and the scope discussion from this thread (full replacement, the drift-vs-born-wrong distinction, and the rustdoc intra-doc-links follow-up as a separate, later issue). Linked it from this PR's description as well.

Going with your offer to land this PR as the stop-gap — further lychee discussion can continue on #578 rather than here.

auto-merge was automatically disabled October 7, 2026 05:24

Pull request was closed

jussielo-amd added a commit to jussielo-amd/rocm-cli that referenced this pull request Oct 7, 2026
* ci: add lychee gate for markdown links (ROCm#578)

docs/architecture.md's path citations are about to move to explicit
markdown links instead of bare backtick filenames, checked by an
off-the-shelf link checker instead of a hand-rolled parser (see ROCm#578).
This adds that checker now, additively: it only looks at links that
already exist, so it can land independently of the citation rewrite
and doesn't conflict with ROCm#493/ROCm#564, which are still editing
docs/architecture.md's prose.

Scoped offline (no network) to avoid external-host flakiness, and
excludes docs/rocm-docs/ (Sphinx/MyST source whose cross-doc anchors
are resolved by the docs-build job, not plain markdown).

Signed-off-by: Jussi Elo <jussi.elo@amd.com>

* fix(docs-links): correct overclaim, document the new CI gate

lychee.toml's comment said it replaces architecture_doc.rs's checker;
that checker is untouched and still runs, so say "first step toward
replacing" instead. Also add the docs-links gate to CONTRIBUTING.md's
list of enforced CI invariants, next to check-architecture-doc.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>

* fix(docs-links): narrow CONTRIBUTING.md's link-coverage claim

The docs-links line said every link in the repo is checked, but
lychee.toml runs offline (no external https:// links) and excludes
docs/rocm-docs/ (Sphinx/MyST source, covered by docs-build instead).
Say what the job actually covers and point at lychee.toml for the
exclusion list, instead of overclaiming.

Signed-off-by: Jussi Elo <jussi.elo@amd.com>

---------

Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants