Repository navigation
refactor(rocm): split therock.rs into child modules, part 1 of 3: registry, version, io, process, fetch - #824
Conversation
Create a crate-level runtime_registry module holding the installed-runtime manifest model, the registry path helpers, the manifest loaders and writer, the wheel-composition readers and the rocm_sdk probe validator. therock re-exports the names other modules reach as therock::X, so no consumer changes. The probe validator moves here rather than to a therock child so that the manifest validators can call it without a new crate-level cycle. Pure move: the serde shape of the registry files is unchanged, and the tests that exercise only the moved items move with them. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Move runtime selection, usability and activation validation out of runtimes_cmd into runtime_registry, and delete runtimes_cmd's duplicate registry path helpers (their bodies were identical to the ones that moved there in the previous commit). runtimes_cmd re-exports the names other modules import from it, so those importers are unchanged. therock's consent code now reaches the registry through crate::runtime_registry, which ends the therock to runtimes_cmd dependency. Pure move; the selector test moves with the code. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Move the version parsing, PEP 440 ordering, host-version reader and build-date helpers out of therock.rs into a leaf child module. therock re-exports runtime_version_display for the modules that render versions; the version property tests now import the moved items from the new module. Pure move; the one test that exercises only the moved items moves with them. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
…therock/process.rs Move the scratch-directory and atomic-write helpers into a leaf io module, and the command-running, bounded-wait, progress-line and tarball-extraction helpers into a process module that depends on it. Extraction lives in process so that io and process do not depend on each other. therock re-exports write_file_atomically for its other callers. The process-wide environment lock the tests share moves out of the therock test module into test_support as a single static, so the tests that stay in therock and the moved ones still serialize on the same lock. The archive property tests move under process. Pure move; the tests that exercise only the moved items move with them. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Move the metadata GET with its etag cache and detached-signature verification, the streaming artifact download, and the tarball size and free-space preflight into a child module that depends on the io module. therock re-exports truthy_env, which the driver install code reads. The pinned metadata key constant moves with it, so the path the verify-pinned-keys check reads (and the two doc lines that name it) now point at the new file. Pure move; the tests that exercise only the moved items move with them. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Comment and documentation fixes for statements in the five split commits that were inaccurate or stale. No code changes. - runtime_registry module doc and architecture entry: say the module also holds registry selection and activation validation, that it depends on neither therock nor runtimes_cmd, and that runtimes_cmd still reaches the manifest model through therock's re-exports while therock no longer depends on runtimes_cmd in production code. They used to say the two modules no longer depend on each other. - therock/io.rs module doc and architecture entry: the atomic writes serve the metadata cache entries, the staged metadata public key, the startup update-check record and the active-runtime marker, not the runtime registry manifests, which are written with a plain fs::write. - architecture entry for therock/fetch.rs: list the pub(super) items, including CachedHttpText and its text field, instead of saying everything else is private. - architecture entry for therock/version.rs: the helpers behind the pub(super) items are private, not all items pub(super). - test_support: the PROCESS_ENV_TEST_LOCK doc names the therock tests and the variables it covers, and notes that the main.rs test env guard and the engine_ipc production lock are separate. - VersionOrderKey doc: release and stage are pub(super) because the property tests read them, so it no longer claims no fields are reachable. - Comments in tests that stayed in therock.rs now point at the guard tests in therock::process and the HTTP servers in the fetch tests, and say the shared env lock lives in test_support. - The proptests module doc and the comparator doc say where the comparators live now; an e2e harness comment names therock/process.rs as the extraction site. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
There was a problem hiding this comment.
🟡 Changes recommended
The new header mention causes the extractor to read Remaining Owner Step instead of the pinned PEM, so verification will fail once the canonical metadata key is published.
1 open issue
What changed in this PR
This first stage of the TheRock refactor separates runtime registry responsibilities and install helpers, removing the production dependency from therock to runtimes_cmd while preserving existing caller interfaces and manifest formats.
Changes:
- Extract runtime manifests, selection, and validation into
runtime_registry. - Move version, filesystem, subprocess, and fetch helpers into child modules, retaining tests and a shared environment lock.
- Update architecture documentation, source-reference comments, and the pinned-key checker’s source path.
| File | Description |
|---|---|
xtask/src/verify_pinned_keys.rs |
Points metadata-key verification to therock/fetch.rs. |
xtask/src/main.rs |
Updates the key-verification help comment. |
xtask/src/env_mutation_contract.rs |
Updates the shared-lock location in documentation. |
xtask/src/e2e_prewarm.rs |
Updates the version-helper reference comment. |
tests/e2e-cucumber/tests/e2e/comfyui_steps.rs |
Updates the probe-validator reference comment. |
tests/e2e-cucumber/tests/e2e.rs |
Updates the extraction reference comment. |
docs/architecture.md |
Documents module ownership and dependencies. |
apps/rocm/src/therock/version.rs |
Houses version parsing, ordering, and date helpers. |
apps/rocm/src/therock/process.rs |
Houses subprocess, progress, and extraction helpers and tests. |
apps/rocm/src/therock/io.rs |
Houses temporary-directory and atomic-write helpers and tests. |
apps/rocm/src/therock/fetch.rs |
Houses downloads, signed metadata caching, and related tests. |
apps/rocm/src/therock_version_proptests.rs |
Adjusts imports and documentation for relocated helpers. |
apps/rocm/src/test_support.rs |
Centralizes the existing TheRock environment-test lock. |
apps/rocm/src/runtimes_cmd.rs |
Re-exports relocated registry helpers and removes duplicate paths. |
apps/rocm/src/runtime_registry.rs |
Owns manifest models, persistence, selection, and validation. |
apps/rocm/src/main.rs |
Declares the registry module and removes an unused import. |
Review effort: Balanced
Remove redundant text from documentation comments. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Jussi Elo <jussi.elo@amd.com>
5aa28ca to
2119ee1
Compare
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review (Claude Code) · 2119ee1
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — no blocking findings
Full review of the whole change (7 commits, 17 files, +4179/-3871).
Delivers ROCMAI-799, ROCMAI-800, ROCMAI-801, ROCMAI-802, ROCMAI-803 (Jira sub-tasks for Phase 8.1-A steps 1-5, all currently "In Progress"). The PR explicitly scopes itself as part 1 of 3 of a larger therock.rs split; the description's claimed scope matches the actual diff, and none of the ticket summaries are claimed as closed.
Blocking
None
Non-blocking
None
Decisions for the author
None
Positive signals
- The seven commits are genuinely atomic and reviewable in isolation (one extraction step per commit, plus a separate doc-correction commit) — this let each independent review pass diff a moved function against its pre-move source byte-for-byte rather than trusting the "pure move" framing, and every slice held up under that check.
- Consolidating
PROCESS_ENV_TEST_LOCKinto a singlepub(crate) staticintest_support.rs(apps/rocm/src/test_support.rs:33) is the correct fix for the real hazard a naive split would create: every env-mutating/bare-spawn test acrosstherock.rs,therock/io.rs, andtherock/process.rswas confirmed to still serialize on the one lock instance, so the split introduces no test-isolation race. - The final commit (
9127618) demonstrably fixed real doc/comment inaccuracies left by earlier commits (confirmed by diffing intermediate commit states against final head) — evidence the author re-verified doc claims against code rather than writing them once and moving on.
Deployment notes
- No user-observable behavior change is claimed or found; the registry manifest structs' serde shape was independently confirmed byte-identical (field names,
#[serde(default...)]attributes, variant names all unchanged). - At review time, CI's
E2E testsandwindows-build-and-testjobs were stillin_progress; every check that had completed (clippy, build-and-test, Test (affected crates), Coverage, license/notices, docs-links, commit-signatures, CodeQL, etc.) passed. Thecfg(windows)code that moved (publish_temp_file/replace_file_windowsinio.rs, Windows temp-handling inprocess.rs) is only exercised by the Windows lane, which the PR text itself flags as not run locally — worth confirmingwindows-build-and-testlands green before merge. - This PR is explicitly the foundation for two follow-up PRs; nothing in it should need to be re-opened once merged, since the remaining
therock.rsclusters are untouched here.
What this covered
Every one of the 5 new/changed modules (runtime_registry.rs, therock/version.rs, therock/io.rs, therock/process.rs, therock/fetch.rs) read completely at pr-824-head, each diffed line-by-line against its pre-move source in therock.rs on origin/main (11a6069…2119ee1), via 5 independent fan-out workers grouped by commit/coupling: (1) registry extraction (e38d25a, 5bd8fce), (2) version extraction (38bec5d), (3) io/process extraction (d5f5d58), (4) fetch + pinned-key extraction (4ab2cf1, 2119ee1), (5) the doc-correction commit (9127618) plus docs/architecture.md and cross-module wiring in the surviving therock.rs. Each worker also checked the small incidental comment-only diffs in xtask/ and tests/e2e-cucumber/. I additionally: read the repo's agent-instruction file (AGENTS.md) in full and checked the diff against its doc-link and architecture-guardrail conventions (no violations found); independently re-derived the pinned-key-extraction logic in xtask/src/verify_pinned_keys.rs myself to resolve a conflict with an earlier automated bot comment flagging a token-ordering risk there — confirmed no such risk exists in the code as written (only one occurrence of the token in fetch.rs, at the real const declaration), consistent with that same bot's own follow-up review at this head reporting the issue resolved; and retrieved the 5 named Jira tickets to confirm the description's scope claim.
Did not run: a dedicated "prior review comments" pass beyond what the mandatory pre-review claim-check surfaced (an automated bot's comments, read only for that check and independently re-verified rather than trusted, per method); a local cargo build/test/clippy run of my own (relied on the PR's disclosed verification matrix plus live CI results instead, consistent with reading what CI already reports rather than re-deriving it). Initial attempts to launch the fan-out hit a shared environment-wide concurrency limit and had to be retried in sequence; all 5 workers eventually launched and completed with full independent coverage, so this delayed but did not reduce coverage. Not reconciled against any PR discussion beyond the mandatory head-commit claim-check.
juhovainio
left a comment
There was a problem hiding this comment.
Looks good. I compared the removed and added lines across the whole diff, and the move is verbatim. The only differences are visibility changes (pub(super)/pub(crate)), path re-points to the new modules, and doc/comment re-pointing. No logic changed, and the test count is the same before and after (43 test attributes removed, 43 added).
- The five steps are all there with no scope creep, and the module docs and
docs/architecture.mdmatch the new layout. - CI is green, including the Windows build/test, E2E and clippy.
- The visibility widening is minimal and disclosed in the description.
- The
test_pathsfixture is duplicated across the new test modules. That reads as deliberate, so I'm leaving it as is.
This is a diff-level review; I didn't run the build locally.

Why
apps/rocm/src/therock.rsis the largest file in the repository, about 12,600 lines with roughly 7,000 of production code in sixteen clusters that change for different reasons, and it sits in a module cycle:therockcalledruntimes_cmdfor the active-runtime lookup whileruntimes_cmdimportedtherock. This is the first of three PRs that split it into child modules behind a parent that re-exports what other files use. It lands the part every later move depends on: a crate-levelruntime_registry.rsthat ends the production dependency oftherockonruntimes_cmd, and the four leaf modules (version,io,process,fetch) that the remaining clusters build on. Delivers ROCMAI-799, ROCMAI-800, ROCMAI-801, ROCMAI-802 and ROCMAI-803.What changes
Five commits, one per step, in dependency order so the branch can be reviewed commit by commit, plus a final commit that corrects documentation:
runtime_registry.rstakes the runtime manifest model, its loaders and the SDK probe validators out oftherock.rs.runtimes_cmd.rs, the duplicate path helpers inruntimes_cmd.rsare deleted, andtherock's consent code now reaches the registry throughcrate::runtime_registry. That removes the production edge fromtherocktoruntimes_cmd.therock/version.rstakes the version grammar and build-date helpers.therock/io.rstakes the scratch-directory and atomic-write helpers andtherock/process.rstakes the subprocess, progress and tarball-extraction helpers.PROCESS_ENV_TEST_LOCKmoves totest_support.rsas one static, so tests in every child still serialise on the same lock.therock/fetch.rstakes the signed-metadata cache, downloads and the pinned key constant, and theverify_pinned_keyspath pin follows it.The 14 files that import from
therockare unchanged exceptmain.rs, which gains themod runtime_registry;line and drops one import that became unused. The serde shape of the registry manifest structs is byte-identical to the original, so the on-disk format does not change.Besides the relocations, the commits make these edits, which the first five commit messages understate: helpers that other modules use are widened to
pub(super)orpub(crate)(including the fields ofParsedVersionandVersionOrderKeythat the property tests read);runtime_version_build_dateis narrowed frompub(crate)topub(super);mod ioshadowsstd::iointherock.rs, so threeio::paths are nowstd::io::; the moved tests use small copies of thetest_pathsfixture in each new test module; two doc links are nowsuper::paths; comment pins inxtaskand the e2e harness name the new files; and each new file carries a module doc and an entry indocs/architecture.md. The final commit corrects statements in those new docs that the review found inaccurate (which module depends on which, what the atomic writers are used for, and what ispub(super)).Verification
cargo fmt --checkis clean, andcargo clippy -D warningsis clean onrocmat every commit and onxtaskat the final head.cargo test -p rocm --bin rocm -- --listgives the same 1,374 tests and 7 ignored tests before and after, with identical leaf names, including the 25version_proptestsand the twoarchive_propertiescases.cargo test -p rocm --bin rocm -- therock:: runtime_registry:: runtimes_cmd::passes 257 tests, and the xtaskenv_mutation_contract,ascii_output_contractandverify_pinned_keystests pass.cargo xtask manifest --checkandcargo xtask check-crate-edgespass.verify_pinned_keysentry is dormant today (no canonical metadata key is published), so a stale path would not fail CI. The new path was checked separately with a temporary key file, which reported the constant found intherock/fetch.rs.tests/e2e-cucumbersuite and the Windows lane. Thecfg(windows)code that moved (publish_temp_fileandreplace_file_windowsinio.rs, the Windows temp-directory handling inprocess.rs) is exercised only on the Windows lane; a compile check with thecfggates forced showed no unused or missing imports, but that is not a Windows build.Scope
This is an internal refactor with no user-observable change, so no end-to-end scenario is added; the unmodified
tests/e2e-cucumbersuite covers the unchanged behaviour. The remaining clusters (source and family resolution, managed Python, probes, tarball and wheel resolution, the consent gate and update planning) follow in the next two PRs and are not part of this one.Template checklist: not a bug fix, so there are no stale xfail rows to narrow; no new subcommand or subsystem; no user-facing message is added or changed.