Repository navigation
Conversation
Lemonade's version was pinned in runtime-deps.toml and its archive digests were copied by hand into the Lemonade adapter, so a bump had to touch both and nothing tied them together. runtimes/mise.toml now pins Lemonade (and uv, as lock tooling), and mise locks a URL and SHA-256 per supported platform in runtimes/mise.lock. rocm-deps generates its constants from that lock with one generator, pins_gen.rs, which refuses unsupported platforms, unsafe tool names, non-https URLs and malformed digests. The Lemonade adapter checks downloads against the locked digest, and a test ties its derived download URLs to the locked ones. runtimes/bin/mise and mise.cmd are the vendored mise bootstrap launchers (regeneration command in MANIFEST.md), so no prior mise install is needed. The Windows launcher is checked out with CRLF line endings. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The ROCm SDK and vLLM stacks are resolved at install time today, and nothing reports when their pins fall behind or a lock stops matching them. runtimes/profiles.toml declares each stack's requirements and the OSes it is locked for; `cargo xtask deps lock` resolves each profile per OS with the pinned uv into runtimes/locks/pylock.<profile>-<os>.toml (PEP 751). Locks are per OS because AMD's torch wheels declare different dependencies on Linux and Windows. The header records a digest of each lock's inputs (python, platform, requirements and uv.toml), so a lock left stale by an input edit is detected offline. `cargo xtask deps check`, also run by `cargo test -p xtask`, verifies both lock files: every Python artifact is https with a SHA-256 and comes from PyPI or AMD's indexes, AMD-only packages only from AMD, builds stay on the profile's ROCm line and wheels fit the OS; every binary is locked for exactly Linux and Windows x64 as a release asset of its own repository at the locked version, url_api included, and unknown lock fields are refused. `cargo xtask deps diff` prints the change table the weekly update uses as its pull request body. The lock files themselves are not in this commit: they are generated by the runtime-locks CI job and committed from its artifact, and until then `deps check` reports them missing. No installer reads the Python locks yet; docs/runtime-dependencies.md lists the follow-ups that will. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Pins only stay current if something moves them, and a lock only stays honest if something re-derives it. The runtime-locks CI job re-locks at the committed pins on pull requests that can change a lock and fails on any difference, uploading the regenerated files. It is path-filtered and not required, since it downloads every AMD wheel the locks name. runtime-deps-update.yml runs weekly and on demand: `propose` (read-only; uv builds AMD's rocm sdist, i.e. runs upstream code) moves pins past a 7-day cooldown and re-locks; `check` (read-only) runs the base commit's validator and `cargo xtask deps check` against the artifact; `open-pr` validates it again with the validator fetched at the base commit and commits through createCommitOnBranch, which GitHub signs. It needs "Allow GitHub Actions to create and approve pull requests" enabled. The Windows debug job bootstraps mise through runtimes/bin/mise.cmd. Workflow contract tests pin the privileged steps, the jobs' environments and inputs, and run the commit step's payload builder on a fixture. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
|
The three Python locks are committed in 1309fd6, taken unchanged from the |
Generated by the runtime-locks CI job (`runtimes/bin/mise run --cd runtimes lock`, 7-day cooldown) at the committed pins, and taken from its artifact unchanged. `cargo xtask deps check` passes on them. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The runtime-locks job re-locked at unchanged pins and produced different locks: jinja2 3.1.6 became 3.1.5, sympy 1.14.0 became 1.13.3, and in the vLLM lock datasets 5.1.0 became 1.1.1. Reproduced locally, the cause is handing uv the previous lock as version preferences: with AMD's indexes mirroring common packages ahead of PyPI, successive runs alternate between the two sets and never converge. Two fresh resolutions agree. So the previous lock is no longer given to uv. Each lock instead records the --exclude-newer cutoff it was resolved at, and a re-lock resolves fresh at that cutoff; --upgrade, or a lock that does not exist yet, sets a new cutoff the cooldown before now. `deps check` requires the cutoff, and the AMD-only exemptions now apply to every resolution. Also print workspace-relative lock paths with `/` on every OS: on Windows `Path::join` made messages read `runtimes/locks\pylock...`, which failed the Windows test of `deps check`'s messages. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Conflicts: the CI rust filter keeps runtimes/** and main's proptest-regressions entry and drops runtime-deps.toml, which this branch removes; docs/testing.md and affected.rs keep both sides. main added a `run_script` contract helper with different semantics, so this branch's helper is renamed `run_script_dedented`. main's pre-push selection expects docs/testing.md to select specific crates, so the remediation-spelling test now reads only docs/runtime-dependencies.md (and mise.toml and the workflows), and docs/testing.md and docs/engine-plugins.md no longer force a full-workspace run. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Generated by the runtime-locks CI job from the commit that made each lock record its exclude-newer cutoff, and taken from its artifact unchanged. Each lock now records 2026-11-02T08:40:09Z, so the next re-lock resolves at the same cutoff and reproduces it. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Every `--upgrade` takes a new cutoff and recorded it in every lock header, so the weekly update would have rewritten all locks and opened a pull request each week with "No runtime versions changed.". A resolution that differs from the committed lock only in its cutoff now keeps the committed file unchanged. Also from review: - run the self-hosted GPU lanes on a change under runtimes/, which holds the Lemonade version they exercise, and pin every filter a runtimes change must trip; - recompute the update's pull-request table in the read-only check job and refuse an artifact whose summary differs; - say that a re-lock reproduces its lock except where AMD published a new build of an AMD-only package, which no cutoff holds back, in the docs, comments and the runtime-locks error; - announce, and test, re-locking at a new cutoff when the recorded one is missing or invalid; - refuse whitespace and option-like names in profile requirements, and a directory named like a lock in the update artifact; - tie the runtime cooldown to Dependabot's; - add tests for branches review showed no test pinned, and narrow comments that claimed more than the code does. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Conflict in engines/lemonade/src/install.rs: main added a manifest test where this branch added the locked-digest test; both are kept. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Conflict in docs/architecture.md: the rocm-deps entry keeps this branch's description (constants from runtimes/mise.lock via pins_gen.rs) and main's sentence on lib.rs and its dependants. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
jussielo-amd
left a comment
There was a problem hiding this comment.
Thanks, this is a well-structured change and the trust boundaries in the update workflow look careful. I found nothing blocking. CI on the head commit is green (31 checks). This was a static review: I did not run the Rust tests or the workflows. I did run scripts/validate_runtime_update.py --self-test, which passes.
Non-blocking notes (two more are inline):
tests/e2e-cucumber/expectations.toml:181(a file this PR doesn't touch) still says Lemonade 11.5.1 is "the pin inruntime-deps.toml", which this PR deletes. Pointing it atruntimes/mise.tomlwould finish the job; the 11.5.1 literals on lines 178, 207 and 212 will also go stale on the first weekly Lemonade bump.- The PR description says the validator self-test has 31 cases; at this head it prints 36.
- The vLLM pins (
0.29, torch2.13) are now written both inruntimes/profiles.tomland inengines/vllm/src/discover_builds.rs, with nothing tying them together. Fine as a stated follow-up, but a small drift test would stop the two from diverging silently in the meantime.
🤖 by agenthub on AMD AgentHub
| All dependencies are sourced from [crates.io](https://crates.io) unless noted | ||
| otherwise. No vendored copies of third-party source are included in this | ||
| repository. | ||
| otherwise. One piece of third-party code is included in this repository: |
There was a problem hiding this comment.
Non-blocking: this says the launchers are MIT-licensed third-party code from mise, but neither runtimes/bin/mise nor mise.cmd carries a copyright or licence line, and THIRD_PARTY_NOTICES.txt does not mention mise. MIT asks for the notice to travel with copies, so adding mise's copyright and licence text to THIRD_PARTY_NOTICES.txt would be good.
There was a problem hiding this comment.
Fixed in 7ed9a22: mise's copyright and MIT text now ship as runtimes/bin/LICENSE-mise, and MANIFEST.md points to it. I put it beside the launchers rather than in THIRD_PARTY_NOTICES.txt because that file is generated from the Cargo graph by cargo xtask tpn and covers what the binaries ship, which the launchers aren't.
| } | ||
|
|
||
| #[derive(Debug, Clone, Deserialize)] | ||
| pub(crate) struct Package { |
There was a problem hiding this comment.
Non-blocking: Package (and PyLock) don't use deny_unknown_fields, while the PR says unknown lock fields are refused for the mise lock. deps check only looks at wheels and sdist, so a package entry carrying archive, vcs or directory next to (or instead of) wheels would be read without its source being checked against the host/hash rules. uv doesn't emit those for index packages today, so it's a hardening point: refusing unknown package keys would keep the Python side as strict as the binary side.
There was a problem hiding this comment.
Fixed in 7ed9a22: a package naming a PEP 751 vcs, directory or archive source is now refused, so only wheels and sdists, which are held to the host and hash rules, can reach a lock. I refused those three keys rather than all unknown keys, since uv writes routine fields like index and marker there.
juhovainio
left a comment
There was a problem hiding this comment.
I read through this against ROCMAI-134 and the diff. I found no correctness defects in the generator, lock/check logic, validator or the workflow trust split, and CI is green. I'm requesting changes because of a few decisions that should be made explicitly before merge, mostly around integrity provenance, the untested weekly workflow, and scope against the ticket. Inline comments carry the details.
Needs a decision / fix before merge
- Lemonade digest provenance.
mise.lockrecordschecksum_source = "mise-versions", i.e. a third-party index, and that digest is the adapter's only integrity check.deps checkignores the field, so a weekly bump can change the digest with nothing calling it out. Please surface a changed digest/provenance in the diff table or the weekly PR body, or record that this is accepted. - Cooldown framing. AMD-only packages (rocm, torch, vllm, etc.) are exempt from the 7-day cooldown, and they are most of what moves each week. Please say so plainly in the PR text and the
mise.tomlcomment rather than describing a uniform cooldown. - Weekly workflow never run end to end. The riskiest step is
mise upgrade --bumprewritingversioninside the inline-table Lemonade pin. Please plan aworkflow_dispatchdry run right after merge and watch it to green, with a fallback if--bumpdoesn't touch table-form pins. - Forced reset of
runtime-deps/update. Each run discards any maintainer fixup pushed during the week. Consider skipping the reset (or commenting) when the branch has commits not authored by the bot.
Scope vs. ROCMAI-134 (the ticket asks for a pure refactor lifting the Lemonade and vLLM pins into one TOML)
- The vLLM half isn't lifted; the ticket needs updating or a vLLM follow-up linked.
- The PR goes well beyond a pure refactor: a weekly write-token workflow, vendored mise launchers, per-OS Python locks with no consumer, and ~6k lines of xtask. Please update the ticket to reflect the real scope.
- The EAI-7196 per-GPU-family schema constraint isn't addressed; please state whether that's accepted.
Smaller notes
- Any pin/profile edit makes
cargo test -p xtaskfail until a re-lock (~17 GiB of downloads, or the CI artifact). It's documented, but a standing contributor cost. wheel_fits_oschecks only platform tags, not the Python tag/ABI.
I did not repeat the points already raised in the existing approval (the stale expectations.toml reference, "31 cases" vs 36, and the vLLM pin duplication). This is a read of the diff only; I did not build or run anything locally.
|
|
||
| [tools."github:lemonade-sdk/lemonade"."platforms.linux-x64"] | ||
| checksum = "sha256:bdfd3c3e5d6eda5101c8a32f36e6dd9236ceb9ab2eb66734c32628e6e86e18ac" | ||
| checksum_source = "mise-versions" |
There was a problem hiding this comment.
Lemonade's digest here is checksum_source = "mise-versions": sourced from mise's third-party index rather than GitHub or Lemonade, and it is the only integrity check the adapter applies. The doc says so, but deps check ignores the field and a weekly bump will change this digest without anything calling it out. Could the diff table or PR body flag a changed checksum_source/digest for tools without provenance, so the reviewer knows to verify it independently?
There was a problem hiding this comment.
Done in 07a8c8e: rather than only flag it, the digest is now checked. cargo xtask deps verify-downloads downloads every locked asset without a build attestation (today, Lemonade's two archives) and compares its SHA-256 with mise.lock. The weekly update's read-only check job runs it before a pull request is opened, and the runtime-locks job runs it on any PR touching runtimes/, so a hand bump is checked too. Run against this PR's lock, both 11.5.1 archives match.
|
|
||
| [tools."github:lemonade-sdk/lemonade"."platforms.windows-x64"] | ||
| checksum = "sha256:50a133bbc35c4f3f8971eafef2c9fe56c4bbbfb0f1032bf8728324d8d8c8a0e1" | ||
| checksum_source = "mise-versions" |
There was a problem hiding this comment.
Same as the linux-x64 entry: digest provenance is mise-versions only. Worth an explicit reviewer step on every Lemonade bump.
There was a problem hiding this comment.
Covered by the same check (07a8c8e): deps verify-downloads re-hashes the windows-x64 archive as well, in the weekly check job and in runtime-locks.
| # indexes publish no upload times. An AMD-only dependency missing from this | ||
| # list makes the weekly update fail; add it here. | ||
| vendor-hosts = ["stable.repo.amd.com", "rocm.frameworks.amd.com"] | ||
| vendor-packages = [ |
There was a problem hiding this comment.
Everything in vendor-packages is exempted from the --exclude-newer cutoff (AMD's indexes have no upload times), so the 7-day cooldown does not apply to rocm, torch, vllm etc., which is most of what moves each week. Fine as designed, but the mise.toml comment and PR text describe a uniform cooldown; consider saying plainly that AMD-only packages are not held back.
There was a problem hiding this comment.
Agreed. 07a8c8e says so in the mise.toml comment, the update command's row in docs/runtime-dependencies.md and the PR description: the 7-day cooldown covers binaries and PyPI packages, and AMD-only packages (rocm, torch, vllm, ...) move as soon as AMD publishes them, since AMD's indexes publish no upload times.
| # Lemonade's embeddable release. Each platform names its asset exactly: the | ||
| # release also carries .msi/.deb/.rpm installers, and a pattern shared by both | ||
| # platforms has been seen to lock the Windows zip for linux-x64. | ||
| "github:lemonade-sdk/lemonade" = { version = "11.5.1", platforms = { linux-x64 = { asset_pattern = "lemonade-embeddable-*-ubuntu-x64.tar.gz" }, windows-x64 = { asset_pattern = "lemonade-embeddable-*-windows-x64.zip" } } } |
There was a problem hiding this comment.
The weekly mise upgrade --bump has to rewrite version inside this inline table; the validator assumes it does but the workflow has never run. Suggest a workflow_dispatch dry run right after merge to confirm the bump lands in this table form (otherwise the weekly job would silently propose nothing for Lemonade).
There was a problem hiding this comment.
Checked before merge rather than after: on a scratch copy of this mise.toml pinned to 11.4.0, mise upgrade --bump (mise 2026.10.25, the pinned launcher) rewrote version inside this inline table to 2026.44.1, the newest release past the cooldown (it skipped 2026.45.3 as too new, and the candidate-* tags). mise lock then locked both 2026.44.1 archives, and cargo xtask deps check accepted the result. I'll still trigger a workflow_dispatch run once this merges and watch it through.
| # The branch always restarts from that commit: one proposal, | ||
| # regenerated from scratch each week, never stacked on the last one. | ||
| if gh api "repos/$REPO/git/ref/heads/$BRANCH" >/dev/null 2>&1; then | ||
| gh api -X PATCH "repos/$REPO/git/refs/heads/$BRANCH" -f sha="$base_sha" -F force=true >/dev/null |
There was a problem hiding this comment.
Force-resetting runtime-deps/update to the base SHA discards any fixup a maintainer pushed to the open PR during the week (documented in the header). Consider skipping the reset when the branch has commits not authored by the bot, or at least commenting on the PR when commits are dropped.
There was a problem hiding this comment.
Changed in 07a8c8e: before resetting the branch, open-pr lists the commits on it that are not in the base, and if any was not made by github-actions[bot] it fails with an error naming the count and what to do (merge or close the PR, or move the fixup to main) instead of resetting. The workflow header and docs/runtime-dependencies.md say the same.
| /// for Linux an x86-64 `linux_*`/`manylinux*` tag, for Windows `win_amd64`. A | ||
| /// compressed tag set (`manylinux_2_17_x86_64.manylinux2014_x86_64`) fits | ||
| /// when every tag in it does. | ||
| fn wheel_fits_os(file: &str, os: &str) -> bool { |
There was a problem hiding this comment.
wheel_fits_os checks only the platform tag. A wheel for a different Python (cp312 vs the profile's 3.14) or ABI would pass deps check; the docs describe this as "wheels fit the OS", so just noting the gap.
There was a problem hiding this comment.
Closed in 07a8c8e: deps check now also requires each wheel's Python and ABI tags to fit the profile's Python: a pure-Python tag (py3, or py3X up to 3.14), cp314-cp314, or a cp3X-abi3 wheel for X up to 14. A cp312-cp312 or free-threaded cp314t wheel is refused. The committed locks pass it.
- Ship mise's copyright and MIT licence text beside the vendored launchers (runtimes/bin/LICENSE-mise) and point MANIFEST.md at it. THIRD_PARTY_NOTICES.txt is generated from the Cargo graph and covers what the binaries ship, which the launchers are not. - Refuse a Python lock package that names a PEP 751 `vcs`, `directory` or `archive` source: only wheels and sdists are held to the host and hash rules. - Tie the vllm-rocm10-1 profile's pins to the vLLM adapter's 10.1 build row, which writes the same versions until it reads the lock. - Point the Lemonade xfail note in expectations.toml at runtimes/mise.toml, and say to re-check those rows when the weekly update moves the pin. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
|
Thanks. Addressed in 7ed9a22: |
Lemonade's locked digests come from mise's version index, not a build attestation, and the Lemonade adapter trusts them. `cargo xtask deps verify-downloads` downloads every locked asset without an attestation and compares its SHA-256 with the lock; the weekly update's read-only check job and the runtime-locks job run it, so a changed digest is checked against the real release asset before anyone reviews it. The weekly update no longer discards a maintainer's fixup on its branch: while the branch holds a commit the workflow did not make, the run fails instead of resetting it. `deps check` now also refuses a wheel built for another Python or ABI than the profile's, not only for another OS. mise.toml and the docs say plainly that the 7-day cooldown does not hold back AMD-only packages, which AMD's indexes publish undated. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
|
Thanks for the careful read. The inline points are addressed in 7ed9a22 and 07a8c8e, with replies in each thread: Lemonade's digests are now re-hashed against the release assets ( On the standing re-lock cost: yes. A pin or profile edit needs the CI artifact or a local re-lock, as On scope against ROCMAI-134: ROCMAI-134 now describes what this PR delivers, including the scope beyond the original refactor. Moving the vLLM install onto its lock is the follow-up ROCMAI-922, blocked by ROCMAI-134; until then a test holds the vLLM profile equal to the adapter's 10.1 row. ROCMAI-133 stays open: its bump PRs need a GitHub App token to run CI on their own, which this workflow does not have. EAI-7196 is now ROCMAI-79 (done): the locks are per ROCm line, with each profile naming the line its |
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head 07a8c8e79695
Review — needs work
Full review of the whole change (43 files, ~11k added lines), split into five independent coupled-file worker passes due to size: (1) the new weekly workflow + CI integration + workflow_contract.rs + validate_runtime_update.py; (2) the lock-generation core (lock.rs, mise_lock.rs, pylock.rs, profiles.rs, runtimes/*.toml, the three generated pylock.*.toml files); (3) the validation/diff/verify core (check.rs, diff.rs, verify.rs, mod.rs); (4) vendored mise launchers + pins_gen.rs + the Lemonade adapter's digest wiring; (5) docs and smaller xtask/CI glue.
Implements ROCMAI-134 ("Pin and lock third-party runtimes in one place"). The ticket's own current description explicitly states this PR goes beyond the pure-refactor it originally asked for (it adds the Lemonade digest check, the Python locks, the CI re-lock gate and the weekly update) and names the deferred pieces as follow-ups (ROCMAI-133 for an end-to-end dry run of the workflow, ROCMAI-922 for actually installing from the vLLM lock), and states the per-GPU-family schema constraint (EAI-7196/ROCMAI-79) is satisfied here by keeping locks per ROCm line rather than needing further work.
Blocking
deps verify-downloadsskips its one integrity check for any tool whose lock entry carries anyprovenancevalue, not just a verified one —xtask/src/deps/verify.rs:64(if asset.provenance.is_some() { continue; }),xtask/src/deps/mise_lock.rs:293-297.
verify-downloads's own doc comment states its job: re-hash every binary locked without a build attestation (Lemonade today), since that digest is the Lemonade adapter's only integrity check. But the skip condition only tests whetherprovenanceis present, whiledeps check's matching value-check (asset.provenance.as_deref() != Some("github-attestations")) is gated byname == super::pins_gen::UV_TOOL— i.e. scoped to uv only. For every other tool,provenanceis accepted bydeps checkas an unconstrained free-text field. Todayruntimes/mise.lock's Lemonade entries have noprovenancekey at all, so this is latent, not live: nothing currently exploits it. But if a futuremise lock/mise upgrade --bumprun (or a hand-edit) ever writes anyprovenancestring onto a Lemonade platform entry,deps checkwould accept it silently andverify-downloadswould then skip re-hashing that binary — removing the one check that exists for an unattested download, invisibly (no diff shows a behavior change, only a new line in a TOML file that looks like harmless metadata). This is exactly the class of integrity/provenance gap that matters most for a scheduled, write-enabled dependency-update pipeline.
Confidence 85 · architectural · Fix: makeverify.rs's skip condition match the real invariant (asset.provenance.as_deref() == Some("github-attestations")), or havemise_lock.rsreject any non-github-attestationsprovenancevalue for every tool, not only uv.
Non-blocking
- The weekly update's generated PR-body table can silently omit or merge exactly the lock anomalies
deps checktreats as blocking.xtask/src/deps/diff.rs:163,170(collect()) drops any package with noversionentirely (neither side of the diff), and silently overwrites same-named duplicate packages in theBTreeMapwith no indication two entries existed — whilextask/src/deps/check.rs:130,134treats both ("no version", "locked more than once") as blocking problems. A maintainer who trusts only the auto-generated table (the thing this command exists to produce) could see "no changes" or a clean-looking row for a lock state thatdeps checkwould actually refuse;deps checkremains an independent, mandatory gate, so this can't let a bad lock merge by itself — it's a misleading-diagnostics gap, not a security hole.
Confidence 76 · logic · Fix: havecollect()treat these two cases the same waycheck.rsdoes (error, or an explicit marker row) instead of dropping/overwriting silently. - Package names are interpolated unescaped into the generated Markdown table.
xtask/src/deps/diff.rs:201-209(render()) builds| `{lock}` | `{name}` | ...directly fromnormalize_name(&package.name), which only lowercases and collapses separators — it does not escape backticks/pipes/newlines, and nothing upstream (pylock.rs'sPackage,check.rs) validates the character set of a package name. A name containing those characters would corrupt the table and/or inject lines into text that becomes an automated PR body verbatim. Likely bounded in practice by PyPI's own project-name charset for legitimately-resolved packages, so this needs a separate compromise (a hand-edited lock, or a non-PyPI index with looser naming) to be reachable.
Confidence 70 · mechanical · Fix: escape backticks/pipes, or reject non-PEP-503-safe characters in package names before rendering. - The weekly commit step computes file deletions from an unfiltered directory listing.
.github/workflows/runtime-deps-update.yml:233,251lists every path underruntimes/locksat the base commit with no name-pattern filter, then deletes (viacreateCommitOnBranch) anything not in the freshly-producedpylock.*.tomlset. Any unrelated file later added underruntimes/locks/in a normal PR (a README, a stray file) would be silently queued for deletion by the next weekly run. Bounded by the fact a human reviewing the auto-opened PR's own file diff (as distinct from the generated table above) would see the deletion before merging.
Confidence 72 · logic · Fix: filtercommittedto the samepylock.*.tomlname shape the artifact itself is restricted to before computingdeletions.
Decisions for the author
- Breadth of this PR — tradeoff. Independently, from the diffstat and module boundaries alone (before reading the ticket): this change bundles several only loosely-related subsystems — mise binary vendoring + digest generation, a from-scratch Python per-OS PEP-751 lock generator with zero runtime consumer yet, a brand-new scheduled write-token GitHub Actions workflow, and its own validation/diff/verify tooling — needing five separate reviewer passes to cover with any thoroughness. The ticket's current text explicitly accepts this scope growth and links the deferred pieces to follow-up tickets, so this isn't a description/ticket mismatch; it's a genuine tradeoff the author and maintainers already made deliberately. Flagging per the review method's own rule that a change this size is worth naming as a split candidate, independent of whether the ticket sanctions it.
- Landing the Python lock infrastructure with no installer consuming it yet — tradeoff.
runtimes/locks/*.toml,lock.rs,pylock.rsand the profile model are ~2500+ lines of new, well-tested infrastructure that nothing reads at runtime (confirmed by every worker: no installer touchesruntimes/locks/). The PR's own "Verification" section already names the standing cost (a pin/profile edit requires a full re-lock, ~17 GiB of downloads or the CI artifact, beforecargo test -p xtaskpasses). Deliberate per the ticket (ROCMAI-922 covers actually installing from these locks) — worth the author confirming the ordering (infra now, consumption later) is still preferred given that cost accrues starting now. diff.rs's per-package cell doesn't encode source/host, only version + artifact-hash digest — non-blocking-improvement. A package re-pointed from PyPI to an AMD vendor host (or vice versa) at an unchanged version and file hash renders as "no change" in the generated table, even thoughdeps checkindependently enforces host policy regardless of what the table shows. Not a defect, since the real gate doesn't depend on the table, but a maintainer reading only the table could miss that a trust boundary moved.
Positive signals
lock.rs's whole-batch re-lock (collect every result, write via.toml.partial+ atomic rename, commit only after every target resolves) correctly prevents a mid-run failure from leaving some locks upgraded and others stale, and is directly exercised by tests for both a partial-failure and a write-failure case.mise_lock.rs's path-traversal checks (is_plain_segment, exact<prefix><tag>/<file>segment counting forurl/url_api) are tested against../../../, percent-encoded traversal, extra path segments, and a swapped backend/URL pair that would otherwise look self-consistent — unusually careful for this class of check.- The vendored
runtimes/bin/mise/mise.cmdlaunchers are checkable rather than an opaque blob:MANIFEST.mdnames the exact regeneration command, and a repo test (xtask/src/deps/mod.rs) cross-checks that the two launchers,MANIFEST.md, the design doc, and the Windows CI step all agree on the same mise version. - The new workflow's permission scoping is tight and legible: top-level
permissions: {},contents: readfor thepropose/checkjobs,contents: write+pull-requests: writeonly onopen-pr; the step that runs upstream build code (uvbuilding AMD'srocmsdist) usespersist-credentials: falseand no cache, so no ambient write token is reachable from untrusted build code; and the workflow has no PR-triggered event at all (schedule/workflow_dispatch only), so there's nopull_request_target-class exposure to untrusted PR content.
Deployment notes
- Per the PR's own text, the weekly workflow requires the repository setting "Allow GitHub Actions to create and approve pull requests" to be enabled, and the PRs it opens (via
GITHUB_TOKEN) need a manual close/reopen before CI runs on them — both are maintainer actions outside this diff. - The riskiest runtime behavior —
mise upgrade --bumprewritingversioninside the inline-table Lemonade pin — is explicitly unverified end-to-end per the PR's own "Not verified" section (the workflow only runs frommain). Worth aworkflow_dispatchdry run watched to completion shortly after merge.
What this covered
Every file in the diff, base a2330bfa14c2841314b6313faadacf06e18059ca → head 07a8c8e79695b8f32f41e6e417842f998d655281, read completely by five independent workers grouped by coupling, each cross-checking into neighboring files as needed. One exception disclosed by a worker: xtask/src/workflow_contract.rs is a pre-existing ~8.7k-line file with ~930 lines added by this PR; the worker read the diff in full but, rather than the entire pre-existing file, grepped it systematically for the specific gap classes this review was watching for (SHA-pinning assertions, pull_request_target detection, permission-minimality checks) and reports that as a targeted negative result, not a line-by-line read. The three generated pylock.*.toml lockfiles (378/343/1354 lines) were read in full by eye and additionally checked exhaustively (every package, not a sample) by script for hash format, https-only URLs, forbidden source kinds, and host-allowlist conformance.
History and prior-changes passes did not run: the local clone is a shallow fetch of exactly the PR's base and head commits with no shared history, so git log/git blame against pre-PR revisions returned nothing. Agent-instruction adherence was checked against this repo's AGENTS.md by every worker; no violations found. CI was read via get_check_runs pinned to the reviewed head: every check that had completed was green (clippy, fmt, CodeQL, the new "Runtime locks match their pins" job, Linux/Windows build-and-test, E2E consolidated report, license/notices checks, among others); a handful of self-hosted GPU E2E lanes (Strix Halo Ubuntu nightly/release, MI300X, W7900) were still queued and not yet completed at review time, so their results could not be read.
Two disclosures: first, the mandatory pre-review claim-check (listing existing reviews, to confirm this review hadn't already run at this head) unavoidably returned the full text of two existing human reviews — one approval, one changes-requested — including their specific findings. That text was not used as input to any finding above; every finding here was independently re-derived from the code by workers who never saw it. Where a conclusion here overlaps with a point raised in that prior discussion (the Lemonade digest-provenance concern, and the PR's scope relative to the ticket), it was reached independently, by direct code-tracing and by reading the ticket's own current text, not imported from that review. Second, retrieving ROCMAI-134 by identifier returned its three comments alongside the title/description/status; the comments are discussion and were not used as a source for any finding, only the ticket's description (read at Step 9) fed the ticket-relationship line above. This report was not reconciled against the existing review threads on this PR.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
A `provenance` of any value used to skip the Lemonade re-hash in `deps verify-downloads`. Only uv publishes build attestations, so only uv may carry one, and only the verified value skips the re-hash; `deps check` refuses provenance on every other tool. Package names and versions are rendered as-is into the weekly update's Markdown table, so `deps check` now refuses a name outside PEP 508 or a version outside PEP 440's characters. `deps diff` errors on a package without a version or locked twice instead of dropping or overwriting the row, and the weekly update deletes only stale lock files on its branch. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com> # Conflicts: # crates/rocm-deps/src/lib.rs
|
Thanks, all four findings are fixed in c52174e (the merge with main is 3bb49b2).
On the decisions: the scope and the lock-before-consumer ordering are deliberate (ROCMAI-922 covers installing from the vLLM lock). I'll run the |
Security scan found no critical or high findings on lines changed by this pull request18 medium, low, or unrated finding(s) are not listed; the scan run has them. View the security scan run for complete reports. “New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta. |
…piry The pull-request security scan from main now reads runtimes/locks, and fails on five HIGH findings in the vLLM lock: vLLM 0.29.1 and flash-attn 2.8.3. Those are the newest builds AMD publishes for ROCm 10.1, so no re-lock clears them, and nothing installs from the lock yet. Suppress the five IDs for that one file only, expiring 2027-02-10 so they come back, and record in the design doc that the installer follow-up must not ship them unpatched. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head d40249669cbd
Review — needs work
Full review of the whole change (44 files, ~11.3k added lines), split into five independent coupled-file worker passes due to size: (1) the weekly workflow + CI integration + workflow_contract.rs + validate_runtime_update.py; (2) the lock-generation core (lock.rs, mise_lock.rs, pylock.rs, profiles.rs, runtimes/*.toml, the three generated pylock.*.toml files); (3) the validation/diff/verify core (check.rs, diff.rs, verify.rs, mod.rs); (4) vendored mise launchers + pins_gen.rs + the Lemonade adapter's digest wiring; (5) docs and smaller xtask/CI glue, plus a direct (non-fanned-out) check of the final small commit adding .trivyignore.yaml suppressions and a docs/runtime-dependencies.md note.
Implements ROCMAI-134. Ticket named but not retrievable from here (an internal tracker reference, per this review's own operating constraints) — not read, not quoted, and not used as a source for any finding below.
Blocking
- The batch re-lock's write phase does not actually give the all-or-nothing guarantee its own comment states —
xtask/src/deps/lock.rs:192-211
The comment directly above the write loop reads: "Only once every target has resolved: a failure part-way must not leave some locks upgraded and others not. Each lock is replaced by rename, so a reader never sees a half-written file." The code only delivers the second sentence.resolved: Vec<(PathBuf, String)>is built by resolving every target first, but the final loop (for (committed, locked) in resolved { ... fs::write(&partial, locked).and_then(|()| fs::rename(&partial, &committed)) ... }) commits each target sequentially and returnsErrimmediately on the first write/rename failure (disk full, permission error, a locked file). If that happens on, say, the second of three targets, the first target has already been renamed into place — the exact "some locks upgraded and others not" state the comment says cannot happen. The only existing test for this path (a_lock_that_cannot_be_written_leaves_no_partial_file) fails on the alphabetically-first target, so it cannot observe a later-entry failure and does not exercise this.
Confidence 85 · logic · Fix: either write every target's.partialfile first and only begin renaming after all succeed (narrows, doesn't eliminate, the window), treat a write/rename failure as needing to undo earlier successful renames before returning, or soften the comment to describe the real (best-effort per-file, not whole-batch) guarantee. - The Python lock parser silently accepts unrecognized TOML fields, unlike every other parser this PR adds —
xtask/src/deps/pylock.rs:14-41(PyLock,Package,Artifact), consumed byxtask/src/deps/check.rs
None ofPyLock,Package, orArtifactcarry#[serde(deny_unknown_fields)], andcheck.rsonly walks the already-deserialized structs — it never re-parses the raw TOML table forpylock.toml, so any field beyond the known set passescargo xtask deps checkcompletely unexamined. This directly contrasts with the rest of the PR's own established pattern:profiles.rs'sProfiles/Policy/Profileall use#[serde(deny_unknown_fields)], andmise_lock.rsenforces its own hand-built field allow-list with a dedicated test (fields_this_check_does_not_understand_are_refused) — there is no equivalent test for the Python lock path.deps checkis meant to be the single mandatory gate every lock must pass before being trusted (including by the automated weekly workflow); an unmodeled field in apylock.tomlis exactly the class of thing that gate exists to catch, and today it doesn't for this format.
Confidence 85 · logic · Fix: add#[serde(deny_unknown_fields)]toPyLock,Package, andArtifact, and add a test mirroringmise_lock.rs's unknown-field test forpylock.toml.
Non-blocking
deps diff's table-building has no independent defense against invalid lock states beyond the two it explicitly guards, and renders package names unescaped into Markdown. —xtask/src/deps/diff.rs:141-181(collect),xtask/src/deps/diff.rs:207-225(render)
collect()onlybail!s on "locked without a version" and "locked more than once" (matchingcheck.rs's hard errors for those two cases specifically), but has no equivalent check for other thingscheck.rsindependently rejects: package names/versions are interpolated straight into backtick-quoted Markdown cells vianormalize_name(), which only lowercases and collapses-_.separators — it does not filter backticks, pipes, or newlines, anddiff.rshas no character-set validation of its own (that lives only incheck.rs). Separately, the per-row digest is computed only fromwheels/sdistsha256 hashes, so a package whose source isvcs/directory/archive(a kindcheck.rsindependently refuses, with its own testa_vcs_directory_or_archive_source_is_refused) contributes no digest, and a same-version swap of that source renders as "no change." Today this is bounded because CI always runscargo xtask deps checkbeforedeps diff's output is trusted, and a second independent regex validator (scripts/validate_runtime_update.py'sSUMMARY_ROW) further constrains the output — butdiff.rsitself doesn't assert or test that reliance the way it does for the two cases it already guards.
Confidence 72 · logic/architectural · Fix: validate package name/version characters incollect()using the same rulecheck.rsapplies (shared helper), or explicitly document/assert thatdiff.rsis only safe to run afterdeps checkpasses; foldvcs/directory/archiveidentifiers into the digest so a same-version source swap isn't invisible.docs/testing.md's new "Runtime locks" section never mentionscargo xtask deps verify-downloads—docs/testing.md:519-561
This PR adds the subcommand familycargo xtask deps {lock,check,diff,verify-downloads}.verify-downloadshas its own substantial test suite inverify.rsand is a real CI step inside the veryruntime-locksjob this section documents, but it's never named in the file (confirmed:grep -n "verify-downloads" docs/testing.mdreturns nothing), and the closing example command block only showsdeps check.
Confidence 75 · mechanical · Fix: add a bullet (or extend the existing one) noting the CI job also runscargo xtask deps verify-downloads, and/or add it to the closing example.
Decisions for the author
- Breadth of this PR — tradeoff. Independent of the ticket, from the diffstat and module boundaries alone: this bundles several loosely-related subsystems (mise binary vendoring + digest generation, a from-scratch per-OS PEP-751 Python lock generator with no runtime consumer yet, a new scheduled write-token GitHub Actions workflow, and its own validation/diff/verify tooling) — needing five reviewer passes to cover with any thoroughness. Worth naming as a split candidate on size alone, independent of whether the ticket already accepts the scope growth.
- Landing the Python lock infrastructure with no installer consuming it yet — tradeoff.
runtimes/locks/*.toml,lock.rs,pylock.rs, and the profile model are ~2500+ lines of new, well-tested infrastructure nothing reads at runtime today (confirmed: no installer touchesruntimes/locks/). The PR's own "Verification" section already names the standing contributor cost (a pin/profile edit requires a full re-lock — ~17 GiB of downloads or the CI artifact — beforecargo test -p xtaskpasses). Worth the author confirming the ordering (infra now, consumption later) is still preferred given that cost accrues starting now. diff.rs's per-package cell doesn't encode source/host, only version + artifact-hash digest — non-blocking-improvement. A package re-pointed from PyPI to an AMD vendor host (or vice versa) at an unchanged version and file hash renders as "no change," even thoughdeps checkindependently enforces host policy regardless of what the table shows. Not a defect since the real gate doesn't depend on the table, but a maintainer reading only the table could miss that a trust boundary moved.
Positive signals
- The Lemonade digest-provenance gap a prior round of review on this PR raised (
verify-downloadstreating any non-nullprovenanceas "skip re-hash," rather than specifically a verified attestation value) is fixed:mise_lock.rs'sproblems()now rejects anyprovenancevalue on a tool outside its attested-tools list, andverify.rs's skip condition checks the exact same two constants with the same AND-of-conditions — traced end to end and found symmetric regardless of lock content. - The integrity chain from
runtimes/mise.lockthroughpins_gen.rstoengines/lemonade/src/install.rs's digest check was traced end to end and found sound: no truncation, no platform mismatch,pins_gen::renderis a pure function of its input text compiled identically into both the main build andxtask, andbuild.rscarriescargo:rerun-if-changedon both the lock and the generator so neither can go stale silently. mise_lock.rs's path-traversal / domain-confusion checks (is_plain_segment, exact prefix/tag/file segment counting forurl/url_api) held up under adversarial testing against percent-encoding, extra segments, and swapped backend/URL pairs.- The three generated
pylock.*.tomlfiles were exhaustively script-checked (every package, not a sample): 100% https URLs, well-formed SHA-256 hashes, no forbidden source kinds, no cross-platform contamination, and the Windows SDK lock correctly excludestritonas the design claims. docs/runtime-dependencies.md's substantive claims (provenance/attestation handling, cooldown exemptions, CI wiring, path filters) were cross-checked word-for-word against the actual code and found accurate, including the new CVE-suppression note added in the final commit, which correctly cross-references.trivyignore.yamland matches the exact package/version/CVE set actually present in the vLLM lock.- The new workflow's permission scoping is tight and legible (top-level
permissions: {}, write scope limited to the one job that needs it,persist-credentials: falsearound the step that runs upstream build code, nopull_request_target-class trigger).
Deployment notes
- Per the PR's own text, the weekly workflow requires the repository setting "Allow GitHub Actions to create and approve pull requests" to be enabled, and its PRs (opened via
GITHUB_TOKEN) need a manual close/reopen before CI runs on them — both maintainer actions outside this diff. - The riskiest runtime behavior —
mise upgrade --bumprewritingversioninside the inline-table Lemonade pin — is explicitly unverified end-to-end per the PR's own "Not verified" section (the workflow only runs frommain). Worth aworkflow_dispatchdry run watched to completion shortly after merge. - The PR's final commit adds
.trivyignore.yamlsuppressions (with a 2027-02-10 expiry) for 5 HIGH CVEs in the vLLM lock's newest-available AMD builds; confirmed these are scoped to the one affected file and that nothing currently installs from that lock, per the PR's own stated follow-up (vLLM install is a separate, not-yet-landed piece).
What this covered
Every file in the diff, base 0d5a04471bfd0ef668de909ec780597cba98145a → head d40249669cbd60bd9f3e8cebf13d2d758578113f, read completely by five independent workers grouped by coupling (the PR's head moved twice during this review — from 07a8c8e79695 at the last round, through an interim 3bb49b2c0568 fanned out across the five workers, to this final d4024966, whose two-file delta over the fanned-out state — .github/scan_tools_configs/.trivyignore.yaml and a docs/runtime-dependencies.md paragraph — was small enough to verify directly rather than re-spinning the fan-out, and is reflected in the Positive signals above). xtask/src/workflow_contract.rs (a pre-existing ~10.6k-line file with ~934 changed lines) was covered by full diff read, full read of every touched function, and targeted grep for comparable existing patterns, not a linear read of the whole file — disclosed by that worker as such. The three generated pylock.*.toml lockfiles were read in full by eye and additionally checked exhaustively by script. CI was read via the live statusCheckRollup at the exact head reviewed: all 44 checks green (clippy, fmt, CodeQL, security scans, the Runtime locks match their pins job, Linux/Windows build-and-test, E2E consolidated report, license/notices), no self-hosted GPU lanes pending at this head.
History and prior-changes passes did not run: the local clone is a shallow fetch of the specific commits needed, with no shared pre-PR history, so git log/git blame against earlier revisions returned nothing. Agent-instruction adherence was checked against this repo's AGENTS.md by every worker; no violations found beyond what's listed above.
Disclosures: the mandatory pre-review claim-check unavoidably surfaced the full text of prior reviews on this PR, including a prior automated-review round at an earlier head. That text was not used as input to any finding above — every finding here was independently re-derived from the code by workers given no framing from it; one worker explicitly re-traced the earlier round's provenance concern from scratch and confirmed (not assumed) it is now fixed. This report was not reconciled against the existing review threads on this PR, and the ticket named by this PR was not retrieved, per this review's own operating instructions for this run.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
A batch re-lock wrote and renamed each lock in turn, so a write failing on a later lock left the earlier ones already replaced. Every new lock is now written before any committed one is replaced; a failed write changes no lock, and the comment states the rename window that remains. The Python lock structs accepted fields they do not model, unlike `mise.lock`. They now name the fields uv writes for the current profiles and refuse any other, at every level. `deps diff` no longer relies on `deps check` having run: it refuses a name or version that would break the update's Markdown table, through validators now shared with `deps check`, and a `vcs`, `directory` or `archive` source its digest would not cover. docs/testing.md names `deps verify-downloads`. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
|
Thanks, fixed in 4cf7af8.
Not covered by a test: removing a half-written partial file after a failed write; that would need an injectable writer. |
juhovainio
left a comment
There was a problem hiding this comment.
Fresh pass at 4cf7af8. Nothing blocking in the diff itself, but I'm keeping my earlier changes-requested state until the points below are addressed or explicitly accepted. CI is green on this head.
Should fix / decide
- uv's digest skip trusts the lock's
provenancestring (inline onverify.rs). Written by the job that runs upstream code, so a swapped checksum can still pass. Fine while uv is only lock tooling, but the design doc's follow-up installs uv from this digest. the_vllm_profile_matches_the_adapters_build_rowreadsengines/vllm/src/discover_builds.rs, which isn't registered inaffected.rs; a PR touching only that file skips the test in the Linux affected-crates lane (inline).- The weekly workflow's "someone else's commit" check depends on the bot's
author.login(inline). Please dispatch it once after merge and confirm the second run succeeds.
Worth stating in the PR
4. Five HIGH CVEs (vLLM 0.29.1, flash-attn 2.8.3) are suppressed until 2027-02-10; nothing but prose enforces the "don't ship unpatched" rule. Please confirm the CVE IDs and that the expiry is enforced.
5. Weekly PRs need a manual close/reopen to run CI, and the "Allow GitHub Actions to create and approve pull requests" setting must be enabled first.
6. The PR bundles the Lemonade refactor, vendored mise launchers, lock tooling, a privileged workflow and scan suppressions. Splitting would make each easier to review.
| for (platform, asset) in &tool.platforms { | ||
| // Only an attested tool's verified attestation stands in for | ||
| // re-hashing; `deps check` refuses provenance anywhere else. | ||
| if mise_lock::ATTESTED_TOOLS.contains(&name.as_str()) |
There was a problem hiding this comment.
This skip trusts the provenance string in the lock, which is written by the job that ran upstream code. deps check only requires the string to be present on uv; nothing re-verifies the attestation or re-hashes the archive. A tampered artifact can keep url/url_api inside astral-sh/uv, swap the checksum and keep provenance = "github-attestations", and still pass validate, deps check and this command. Harmless while uv is lock tooling installed from main's lock, but follow-up 3 in the design doc installs uv from this digest. Consider re-hashing uv as well (small archives), or stating in the doc that the attestation is not independently verified.
| "CONTRIBUTING.md", | ||
| "docs/ci-hardware-testing.md", | ||
| "tests/e2e-cucumber/README.md", | ||
| "docs/runtime-dependencies.md", |
There was a problem hiding this comment.
the_vllm_profile_matches_the_adapters_build_row (xtask/src/deps/profiles.rs) reads engines/vllm/src/discover_builds.rs, but only docs and pins_gen.rs are registered here. A PR that edits only that file selects engines/vllm and its dependents, not xtask, so the profile/adapter drift test does not run in the Linux affected-crates lane. Register that file the same way (and add it to the changes filter in ci.yml).
| # fixup) is never discarded: the run stops instead. | ||
| if gh api "repos/$REPO/git/ref/heads/$BRANCH" >/dev/null 2>&1; then | ||
| others=$(gh api "repos/$REPO/compare/$base_sha...$BRANCH" \ | ||
| --jq '[.commits[] | select(.author.login != "github-actions[bot]")] | length') |
There was a problem hiding this comment.
This ownership check assumes a commit made by createCommitOnBranch with GITHUB_TOKEN reports author.login == "github-actions[bot]". If the author is null or a different login, the second weekly run treats the workflow's own previous commit as a maintainer fixup and fails. The PR says this workflow is not verified end to end; please run it once via workflow_dispatch after merge and confirm the second run succeeds (or compare on committer/commit.author.name).
`deps verify-downloads` skipped uv whenever its lock carried `provenance = "github-attestations"`, a string the weekly update writes in the job that runs upstream code. It now re-hashes every locked download, uv's included; the committed uv and Lemonade digests match their release assets. The vLLM profile drift test reads engines/vllm/src/discover_builds.rs as text, so a change to that file alone selected only the vLLM engine and skipped it. The file now forces the workspace, like the docs xtask tests read, and is in ci.yml's rust filter. The weekly update counted a commit as its own only by the bot's login. It now also accepts one whose author GitHub left unresolved when the commit email is the bot's; a test runs the workflow's own jq filter. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
|
Thanks. Fixed in 404c726.
|
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head 404c7265a3c3
Review — no blocking findings
Full review of the whole change (44 files, ~11.6k added lines). Split into coupled-file groups per the review method's size guidance: (1) CI/workflow integration (runtime-deps-update.yml, ci.yml, e2e-selfhosted.yml, workflow_contract.rs, validate_runtime_update.py, affected.rs, .pre-commit-config.yaml) — covered by an independent fanned-out worker; (2) the lock-generation core (lock.rs, mise_lock.rs, pylock.rs, profiles.rs, mod.rs, runtimes/*.toml, the three generated pylock.*.toml locks); (3) the validation/diff/verify core (check.rs, diff.rs, verify.rs); (4) vendored mise tooling + digest wiring (pins_gen.rs, install.rs, runtimes/bin/mise{,.cmd}, build.rs, lib.rs, MANIFEST.md); (5) docs and smaller glue — groups (2)–(5) read directly rather than by further fanned-out workers (see What this covered).
Implements ROCMAI-134. Per the PR's own description (read at Step 9, ticket content itself not retrieved — internal tracker), this goes beyond that ticket's originally-scoped pure refactor: it adds the Lemonade digest check, per-OS Python locks, the CI re-lock gate, and the weekly update workflow, and names the deferred pieces as follow-ups (ROCMAI-133 for an end-to-end dry run of the workflow, ROCMAI-922 for installing from the vLLM lock). The description states the EAI-7196 per-GPU-family schema constraint is satisfied by locking per ROCm line.
Blocking
None.
Non-blocking
docs/runtime-dependencies.mdcites the wrong file for where the vLLM/torch version pins it describes as duplicated actually live —docs/runtime-dependencies.md:36,280-281citeengines/vllm/src/wheel_builds.rs, but the actual pins (vllm_version_prefix: "0.29",torch_version_prefix: "2.13") and the drift test that cross-checks them against the profile (xtask/src/deps/profiles.rs:362,the_vllm_profile_matches_the_adapters_build_row) are inengines/vllm/src/discover_builds.rs. Confirmed by grep:wheel_builds.rscontains no occurrence of"0.29"or"2.13"anywhere; it only holds an unrelated, olderVLLM_ROCM_BUILD_TABLErow (rocm_sdk_version: "7.2.3") for a different install path (thewheels.vllm.aiindex, not AMD's frameworks index). A maintainer following this doc's pointer to resolve the known vLLM-pin-duplication concern would land in the wrong file.
Confidence 85 · mechanical · Fix: change both references toengines/vllm/src/discover_builds.rs.checkjob'stimeout-minutesvalue is asserted to exist but never checked against its actual value, unlike its sibling jobs.xtask/src/workflow_contract.rs, insideruntime_deps_update_resolves_read_only_and_commits_only_validated_files: thecheckjob's block only confirms the literal key"timeout-minutes"is present viajob_keys(&check_job), with nohas_line/assert!on its actual value (30, per.github/workflows/runtime-deps-update.yml:134) — contrasted withpropose(asserted240),open-pr(asserted20), andruntime-locks(asserted180), each tied to its real number. Thecheckjob's timeout could be bumped arbitrarily (e.g. to1440) without any test failing, while the same mutation on any sibling job is caught.
Confidence 70 · mechanical · Fix: addassert!(has_line(&check_job, " timeout-minutes: 30"));alongside the sibling checks, for symmetry.
Decisions for the author
- Breadth of this PR — tradeoff. From the diffstat and module boundaries alone: this bundles several only loosely-related subsystems — mise binary vendoring + digest generation, a from-scratch per-OS PEP-751 Python lock generator with no runtime consumer yet, a new scheduled write-token GitHub Actions workflow, and its own validation/diff/verify tooling. The PR's own description accepts this scope growth explicitly and links the deferred pieces to follow-up tickets, so this is a deliberate tradeoff rather than scope creep; still worth naming given how much review surface it creates.
- Landing the Python lock infrastructure with no installer consuming it yet — tradeoff.
runtimes/locks/*.toml,lock.rs,pylock.rs, and the profile model are ~2,500+ lines of new, well-tested infrastructure that nothing reads at runtime today — confirmed by grepping the whole tree forruntimes/locks/LOCK_DIRoutsidextask/src/deps/and the workflow-contract tests: no hits in any installer or engine adapter. The PR's own "Verification" section already names the standing contributor cost (a pin/profile edit requires a full re-lock — ~17 GiB of downloads, or the CI artifact — beforecargo test -p xtaskpasses). Worth confirming the ordering (infra now, consumption later) is still preferred given that cost accrues starting now. diff.rs's per-package table cell encodes only version + artifact-hash digest, not source/host — non-blocking-improvement. Verified directly inartifacts_digest()(xtask/src/deps/diff.rs): it hashes onlywheels/sdistSHA-256 values. A package re-pointed from PyPI to an AMD vendor host (or vice versa) at an unchanged version and file hash would render as "no change" in the generated weekly-update table, even thoughdeps checkindependently enforces host policy regardless of what the table shows. Not a defect, since the real gate doesn't depend on the table, but a maintainer skimming only the table could miss that a trust boundary moved.
Positive signals
- Since the last reviewed revision, the two most substantive open concerns from this PR's prior review round were fixed, and both fixes trace cleanly:
verify.rs/mise_lock.rsno longer let anyprovenancevalue (including uv's genuinegithub-attestationsstring) skip re-hashing —cargo xtask deps verify-downloadsnow re-hashes every locked binary unconditionally, closing the path where a value written by the job that runs upstream code could stand in for proof; andengines/vllm/src/discover_builds.rsis now explicitly registered inxtask/src/affected.rs(XTASK_READ_DOCS, with a dedicated test,select_vllm_build_table_read_by_xtask_forces_workspace) so a PR touching only that file still runs the drift test that reads it as text. - The weekly workflow's "someone else's commit" detection (
runtime-deps-update.yml, and the mirroredworkflow_contract.rstest) now handles GitHub's compare-API edge case where an unresolved commit author leaves.authornull, falling back to the bot's known commit email — and the new test (the_update_counts_only_commits_the_workflow_did_not_make) actually runs the realjqfilter extracted from the workflow text against realistic compare-API-shaped fixtures (bot-by-login, bot-by-email-only, a maintainer whose email happens to collide, mixed lists), rather than merely pinning the filter's text. lock.rs's whole-batch re-lock writes every target's.partialfile first (undoing all of them on any write failure) and only then renames each into place; its own comment candidly describes the real, narrowed-but-not-eliminated guarantee (a rename failure partway can still leave earlier targets replaced) rather than overclaiming full atomicity — verified by tracing both loops directly.- The vendored
runtimes/bin/mise/mise.cmdlaunchers are mise's own generated bootstrap scripts (not custom to this repo): for the pinned version they verify against a checksum embedded in the script itself at vendoring time (not one fetched from the same channel as the download), and the outer wrapper'sMISE_VERSION="v$mise_version" installprefix-assignment correctly feeds the resolved pinned version into the nested installer so an empty/unsetMISE_VERSIONenv var (as CI sets) resolves to the pinned, checksum-embedded path rather than silently falling through to an unpinned "latest release" lookup — traced end to end, including the installer's own comment explaining exactly why the prefix assignment is there. - The weekly workflow's privilege separation is clean and was traced directly:
propose(runs untrusted upstream build code) holdscontents: readonly,persist-credentials: false, and no cache;open-pr(holdscontents: write+pull-requests: write) never executes anything from the artifact it downloads, instead re-validating against the trusted validator fetched fresh from the base commit via the GitHub API (with a sanity grep against truncation); no job usespull_request_targetor any PR-triggered event at all. - Ran
python3 -I scripts/validate_runtime_update.py --self-testdirectly: passes, reporting exactly 36 cases, matching the PR description's current claim. - The
.trivyignore.yamlCVE suppressions were checked against the actual lock contents: the suppressed package/version pairs (vLLM0.29.1, flash-attn2.8.3) match exactly whatruntimes/locks/pylock.vllm-rocm10-1-linux.tomllocks, the expiry (2027-02-10) is a bounded ~3-month window from review time, and the scoping is to that one file only.
Deployment notes
- Per the PR's own text, the weekly workflow requires the repository setting "Allow GitHub Actions to create and approve pull requests" to be enabled, and the PRs it opens (via
GITHUB_TOKEN) need a manual close/reopen before CI runs on them — both maintainer actions outside this diff. - The riskiest runtime behavior —
mise upgrade --bumprewritingversioninside the inline-table Lemonade pin — is explicitly unverified end-to-end per the PR's own "Not verified" section (the workflow only runs frommain, weekly or on dispatch). Worth aworkflow_dispatchdry run watched to completion shortly after merge, as the PR text itself already plans. - Five HIGH CVEs are suppressed in
runtimes/locks/pylock.vllm-rocm10-1-linux.tomluntil 2027-02-10 (verified above); nothing installs from that lock yet, but the PR's own text flags that the follow-up which does (ROCMAI-922) must not ship them unpatched.
What this covered
Base 0d5a04471bfd0ef668de909ec780597cba98145a → head 404c7265a3c3c8bbf44ea382d52e7e994532e800. The PR's head moved twice while this review was in progress (from an earlier revision through 4cf7af81b1ab, fanned out across groups at that point, to the final 404c7265a3c3); the 4cf7af81…404c7265 delta (9 files, ~166/−107 lines: verify.rs, mise_lock.rs, mod.rs, affected.rs, workflow_contract.rs, ci.yml, runtime-deps-update.yml, docs/runtime-dependencies.md, docs/testing.md) was read and traced in full directly against the fixes it claims to make, rather than re-spinning the whole fan-out, and is reflected in the findings and positive signals above. Every other file in the diff was read completely.
Fan-out was degraded by a concurrency limit in the execution environment (repeated "concurrent subagent limit reached" errors from a shared sandbox): of the five file groups this change was split into, only the CI/workflow group was covered by an independently spawned, blind worker (it also read .github/workflows/runtime-deps-update.yml and the relevant ci.yml sections for context). The remaining four groups — lock-generation core, validation/diff/verify core, vendored tooling/digest wiring, and docs/smaller glue — were read and traced directly by this same reviewing context rather than by separate independent workers, which this method's own rules require disclosing: those passes are not independent in the fan-out sense, though each file in them was still read in full and cross-checked against its neighbors. xtask/src/workflow_contract.rs (~10.7k pre-existing lines, ~1,000 changed across both reviewed revisions) was covered via full diff reads, full reads of every new/changed function, and targeted grep for comparable existing conventions — not a linear read of the whole file, per the fanned-out worker's own disclosure.
History and prior-changes passes did not run in a form independent of this PR's own commits (only this PR's own revision history was inspectable from the local fetch). Agent-instruction adherence was checked against this repository's root AGENTS.md: no violations found, by the fanned-out worker for its group and by direct inspection for the rest.
CI was still in progress at the final reviewed head at review time (pushed minutes before this review concluded): the prior two revisions (4cf7af81, and d4024966 before it) were independently confirmed green, and nothing in the 4cf7af81…404c7265 delta touches code paths those runs didn't already exercise, but the live check-run rollup at 404c7265 itself had not finished (several Linux/Windows/CodeQL jobs still IN_PROGRESS) at the time this review was written and published.
Disclosure: the mandatory pre-review claim-check (listing existing reviews, run twice — once before review and once immediately before publishing — to confirm no duplicate had landed at the exact head reviewed) returned the full text of prior human and automated reviews on this PR, including their specific findings. That text was not used as input to any finding above; findings were independently re-derived from the code. Where a conclusion here overlaps with a point a prior round raised (the provenance/re-hashing concern, the discover_builds.rs registration gap, both now fixed; the PR's breadth as a tradeoff), it was reached independently by direct code-tracing, not imported from that discussion. This report was not reconciled against the existing review threads on this PR.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
The design doc said the versions vLLM installs are also spelled in wheel_builds.rs, but the ROCm 10.1 pins the lock duplicates are in discover_builds.rs; wheel_builds.rs holds the older wheels.vllm.ai route. Name both, and say that the drift test ties versions only: the 10.1 row installs from AMD's staging host while the lock resolves from rocm.frameworks.amd.com. The update workflow's contract test now pins the check job's timeout, as it does for the other jobs. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
juhovainio
left a comment
There was a problem hiding this comment.
I went through this at 0b78c1c. I didn't find a correctness bug, so nothing here is a hard blocker, but I'm requesting changes because a few trust and scope decisions need to be resolved or explicitly accepted before this merges. CI is green.
To resolve or accept
- Lemonade digest provenance. The Lemonade checksum comes from
mise-versions, not from upstream. The independent re-hash (deps verify-downloads) only runs in the non-requiredruntime-locksjob and the weekly check, so "re-checked whenever it changes" is advisory today. Either run it in a required job or document the trust assumption next to the adapter check. runtime-locksjob. It is non-required, downloads about 17 GiB, and goes red whenever AMD publishes a new build of an AMD-only package, whatever the PR changed. It is also the only PR-time check on Lemonade digests, so an unrelated red there will get ignored. Please decide whether it becomes required or the digest check moves.index-strategy = "unsafe-first-match"inuv.toml. For non-vendor package names this lets an AMD index mirror win over PyPI. Please confirm that is intended.rocm-deps/build.rsreads../../runtimes/mise.lockfrom outside the crate. Please confirm this still works in the Docker and packaging build contexts.- Size. At roughly 12k diff lines this covers three separable pieces: the mise-locked binaries, the Python locks and the weekly workflow. Splitting them would make each easier to review and to revert. If you keep it as one PR, a short note saying why would help.
Decisions for the author
- The Trivy suppression for 5 HIGH CVEs expires 2027-02-10 and is gated on ROCMAI-922. Please confirm that ticket is owned and scheduled before the expiry.
- The Python locks have no consumer yet. Please state who picks them up, or hold them back until there is one.
Also worth noting: the weekly workflow only runs from main, so it hasn't been exercised end to end before merge.
Inline comments below cover items 1-3.
|
|
||
| [tools."github:lemonade-sdk/lemonade"."platforms.linux-x64"] | ||
| checksum = "sha256:bdfd3c3e5d6eda5101c8a32f36e6dd9236ceb9ab2eb66734c32628e6e86e18ac" | ||
| checksum_source = "mise-versions" |
There was a problem hiding this comment.
Lemonade's digest is sourced from mise-versions, not from upstream. The adapter trusts it at install time; the independent re-hash (deps verify-downloads) only runs in the non-required runtime-locks job and the weekly check. Consider running it in a required job, or noting the trust assumption next to the adapter check.
| name: Runtime locks match their pins | ||
| needs: changes | ||
| # `runtimes` is only ever true from the pull-request path filter. | ||
| if: needs.changes.outputs.runtimes == 'true' |
There was a problem hiding this comment.
This job is non-required, path-filtered, and fails whenever AMD publishes a new build of an AMD-only package, regardless of what the PR changed. Since it is also the only PR-time check that Lemonade digests match the release asset, an unrelated red here will be ignored. Worth deciding whether it becomes required or the digest check moves elsewhere.
| # risk — an AMD-only package resolved from PyPI — is closed by `deps check`, | ||
| # which refuses a lock that takes any `vendor-packages` entry from a host | ||
| # outside `vendor-hosts`. | ||
| index-strategy = "unsafe-first-match" |
There was a problem hiding this comment.
unsafe-first-match lets an AMD index mirror of a common package win over PyPI. The vendor-package rule covers AMD-only names; for other names this is a deliberate trust of AMD's mirrors. Please confirm that is intended.
Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com> # Conflicts: # docs/architecture.md
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head 0b78c1c8a6f6
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — no blocking findings
Full review of the whole change (44 files, +11,590/-127, merge-base 0d5a04471b…head 0b78c1c8a6f6). Implements ROCMAI-134 ("Pin and lock third-party runtimes in one place") in full per the ticket's own text ("Delivered by #941"), and intentionally exceeds that ticket's original framing by also adding Lemonade's download-digest verification; the ticket itself stays open only for an unrelated follow-up (ROCMAI-133), and a second follow-up (ROCMAI-922, installing vLLM from its lock) is referenced consistently in both the ticket and this PR.
Blocking
None
Non-blocking
- Loose PEP 440 prefix match in the requirement/lock version check —
xtask/src/deps/profiles.rs:320, consumed atxtask/src/deps/check.rs:156
Requirement::matches's prefix branch checksrest.starts_with("a"/"b"/"rc"/"dev"/"post")without requiring a digit or end-of-string after the matched token, so a locked version like"0.29adventure+rocm10.1.0"would incorrectly satisfy a declared"vllm==0.29.*"requirement incargo xtask deps check. Traced the exact call site; confirmed by direct code reading. Real-world severity is bounded: a genuinely uv-resolved version is always PEP‑440-conformant, so this only matters against a hand-edited or corrupted lock, and no installer yet consumes these Python locks. Confidence 80 · logic · Fix: require a digit or end-of-string immediately after the matched prefix token. - New workflow contract tests don't verify action SHA-pinning format —
xtask/src/workflow_contract.rs:739-745(compare the stricter pattern already used for the queue-report workflow atxtask/src/workflow_contract.rs:9920-9924)
The new contract tests coveringruntime-deps-update.yml'spropose/check/open-prjobs andci.yml's newruntime-locksjob only assert an action reference starts with the right name (starts_with("actions/checkout@")), never that the ref after@is a 40-hex-char commit SHA. A future regression to a floating tag onopen-pr(which holdscontents:write/pull-requests:write) would pass this specific test. Mitigated in practice:.github/scan_tools_configs/zizmor.ymldoes not suppress zizmor'sunpinned-usesrule, so the repo-wide security scan (currently green) would likely still catch it independently. Confidence 80 · mechanical/test-coverage · Fix: extend the SHA-format assertion already used for the queue-report workflow to these jobs too.
Decisions for the author
- AMD-vendor packages are fully exempt from the 7-day update cooldown — tradeoff
Because AMD's package indexes publish no upload timestamps,rocm/torch/vllm/etc. moves into the weekly-update proposal immediately on a new AMD publish, with no cooling-off window, unlike PyPI-sourced packages. This is explicit and deliberate (mise.toml, docs, error text), not an oversight — confirm it's still the intended posture given AMD packages are exactly the highest-trust, highest-blast-radius dependency here. - Narrow TOCTOU window in
open-pr's branch-reset flow — tradeoff
Between checking "no foreign commits exist on the update branch" and force-resetting the branch ref, a commit landing in that gap would be silently discarded even though the check exists to prevent exactly that. Low likelihood given the weekly/on-demand cadence; worth a conscious call rather than a blocking fix. - Suppressing 5 HIGH trivy CVEs in the vLLM lock — tradeoff, not a finding
.github/scan_tools_configs/.trivyignore.yamlscopes the suppression to five specific CVE IDs and one specific file, with an explicit 2027-02-10 expiry, and ties it to a tracked follow-up (ROCMAI-922) that "must not ship these builds unpatched." Confirmed narrowly scoped by direct reading; flagging only so the expiry/follow-up linkage stays visible.
Positive signals
- The weekly-update workflow is carefully least-privileged:
permissions: {}at the workflow level,contents: readonly for the two jobs that run upstream code (propose,check), andcontents: write/pull-requests: writereserved foropen-pr, which does zeroactions/checkoutand pulls only two named files via the Contents API pinned to the immutable commit it started from. open-prre-validates the artifact produced by the job that ran upstream code using a validator fetched fresh from the base commit (never trusting the artifact's own copy), with a sanitygrepguarding against a silently-empty/truncated fetch passing as "validated."mise_lock.rs's test suite is unusually adversarial: path traversal, percent-encoding tricks, host/URL-swap attacks, and provenance spoofing tested in both directions.- The atomic multi-lock-file write in
lock.rshas dedicated tests for a write failing at every possible index, so a partial-replace state is actually guaranteed unreachable rather than just claimed. - The PR description states exactly what was and wasn't verified (including three known-flaky pre-existing tests unrelated to this change) and documents a deliberate single-branch-mutation pass across the new guards; running
python3 -I scripts/validate_runtime_update.py --self-testindependently confirmed its claimed "36 cases pass."
Deployment notes
- Requires "Allow GitHub Actions to create and approve pull requests" enabled in repo settings for the weekly workflow to open PRs (stated in the PR body).
- PRs opened by the weekly workflow use
GITHUB_TOKENand so do not trigger CI automatically; a maintainer must close/reopen them to run checks. - The
runtime-locksCI job is deliberately non-required and path-filtered (it downloads real AMD wheels) — a lock drift is caught but won't by itself block a merge unless someone is watching that job. - No installer yet consumes the committed Python locks; the vLLM lock currently carries 5 suppressed HIGH CVEs that must be resolved before any installer is wired to it (tracked as ROCMAI-922).
What this covered
Every file in the diff (44 files) read completely via three independent fan-out workers grouped by coupling (lock data model/types; the four deps subcommands plus the CI automation that drives them; the consumer side — rocm-deps/Lemonade/vendored mise/docs), each reading full file content plus surrounding code, not just diff hunks. I independently re-verified both surviving findings against the source (exact call sites, line-by-line trace) and re-verified two candidates raised by a worker that turned out, on cross-file checking, to already be covered elsewhere (pylock.rs's validation helpers are tested via check.rs/diff.rs; inputs_digest omitting profile.rocm is benign because check.rs independently cross-checks it) — both dropped. I also independently: structurally scanned all three generated Python lock files (all-HTTPS, valid-length SHA-256, no duplicate package names); confirmed the removed runtime-deps.toml has zero remaining references in the tree; confirmed docs/architecture.md links the new pins_gen.rs as the repo's own architecture-doc check requires; confirmed the workflow's permissions: scoping directly in the YAML; and ran the validator's own --self-test, matching the PR description's claim. CI: all 44 checks green at this commit (no failing required check). Ticket ROCMAI-134 was retrieved and reconciled; the PR description was read only after findings were reached, per method.
Did not run: the "prior changes" pass (review comments on earlier rounds of this PR) — not read, by design, since this review is blind to PR discussion; this is noted here rather than silently skipped. Not reconciled against any discussion on this PR.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head 0b78c1c8a6f6
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — no blocking findings
Full review of the whole change (44 files, +11,590/-127, merge-base 0d5a04471b…head 0b78c1c8a6f6). Implements ROCMAI-134 ("Pin and lock third-party runtimes in one place") in full per the ticket's own text ("Delivered by #941"), and intentionally exceeds that ticket's original framing by also adding Lemonade's download-digest verification; the ticket itself stays open only for an unrelated follow-up (ROCMAI-133), and a second follow-up (ROCMAI-922, installing vLLM from its lock) is referenced consistently in both the ticket and this PR.
Blocking
None
Non-blocking
- Loose PEP 440 prefix match in the requirement/lock version check —
xtask/src/deps/profiles.rs:320, consumed atxtask/src/deps/check.rs:156
Requirement::matches's prefix branch checksrest.starts_with("a"/"b"/"rc"/"dev"/"post")without requiring a digit or end-of-string after the matched token, so a locked version like"0.29adventure+rocm10.1.0"would incorrectly satisfy a declared"vllm==0.29.*"requirement incargo xtask deps check. Traced the exact call site; confirmed by direct code reading. Real-world severity is bounded: a genuinely uv-resolved version is always PEP‑440-conformant, so this only matters against a hand-edited or corrupted lock, and no installer yet consumes these Python locks. Confidence 80 · logic · Fix: require a digit or end-of-string immediately after the matched prefix token. - New workflow contract tests don't verify action SHA-pinning format —
xtask/src/workflow_contract.rs:739-745(compare the stricter pattern already used for the queue-report workflow atxtask/src/workflow_contract.rs:9920-9924)
The new contract tests coveringruntime-deps-update.yml'spropose/check/open-prjobs andci.yml's newruntime-locksjob only assert an action reference starts with the right name (starts_with("actions/checkout@")), never that the ref after@is a 40-hex-char commit SHA. A future regression to a floating tag onopen-pr(which holdscontents:write/pull-requests:write) would pass this specific test. Mitigated in practice:.github/scan_tools_configs/zizmor.ymldoes not suppress zizmor'sunpinned-usesrule, so the repo-wide security scan (currently green) would likely still catch it independently. Confidence 80 · mechanical/test-coverage · Fix: extend the SHA-format assertion already used for the queue-report workflow to these jobs too.
Decisions for the author
- AMD-vendor packages are fully exempt from the 7-day update cooldown — tradeoff
Because AMD's package indexes publish no upload timestamps,rocm/torch/vllm/etc. moves into the weekly-update proposal immediately on a new AMD publish, with no cooling-off window, unlike PyPI-sourced packages. This is explicit and deliberate (mise.toml, docs, error text), not an oversight — confirm it's still the intended posture given AMD packages are exactly the highest-trust, highest-blast-radius dependency here. - Narrow TOCTOU window in
open-pr's branch-reset flow — tradeoff
Between checking "no foreign commits exist on the update branch" and force-resetting the branch ref, a commit landing in that gap would be silently discarded even though the check exists to prevent exactly that. Low likelihood given the weekly/on-demand cadence; worth a conscious call rather than a blocking fix. - Suppressing 5 HIGH trivy CVEs in the vLLM lock — tradeoff, not a finding
.github/scan_tools_configs/.trivyignore.yamlscopes the suppression to five specific CVE IDs and one specific file, with an explicit 2027-02-10 expiry, and ties it to a tracked follow-up (ROCMAI-922) that "must not ship these builds unpatched." Confirmed narrowly scoped by direct reading; flagging only so the expiry/follow-up linkage stays visible.
Positive signals
- The weekly-update workflow is carefully least-privileged:
permissions: {}at the workflow level,contents: readonly for the two jobs that run upstream code (propose,check), andcontents: write/pull-requests: writereserved foropen-pr, which does zeroactions/checkoutand pulls only two named files via the Contents API pinned to the immutable commit it started from. open-prre-validates the artifact produced by the job that ran upstream code using a validator fetched fresh from the base commit (never trusting the artifact's own copy), with a sanitygrepguarding against a silently-empty/truncated fetch passing as "validated."mise_lock.rs's test suite is unusually adversarial: path traversal, percent-encoding tricks, host/URL-swap attacks, and provenance spoofing tested in both directions.- The atomic multi-lock-file write in
lock.rshas dedicated tests for a write failing at every possible index, so a partial-replace state is actually guaranteed unreachable rather than just claimed. - The PR description states exactly what was and wasn't verified (including three known-flaky pre-existing tests unrelated to this change) and documents a deliberate single-branch-mutation pass across the new guards; running
python3 -I scripts/validate_runtime_update.py --self-testindependently confirmed its claimed "36 cases pass."
Deployment notes
- Requires "Allow GitHub Actions to create and approve pull requests" enabled in repo settings for the weekly workflow to open PRs (stated in the PR body).
- PRs opened by the weekly workflow use
GITHUB_TOKENand so do not trigger CI automatically; a maintainer must close/reopen them to run checks. - The
runtime-locksCI job is deliberately non-required and path-filtered (it downloads real AMD wheels) — a lock drift is caught but won't by itself block a merge unless someone is watching that job. - No installer yet consumes the committed Python locks; the vLLM lock currently carries 5 suppressed HIGH CVEs that must be resolved before any installer is wired to it (tracked as ROCMAI-922).
What this covered
Every file in the diff (44 files) read completely via three independent fan-out workers grouped by coupling (lock data model/types; the four deps subcommands plus the CI automation that drives them; the consumer side — rocm-deps/Lemonade/vendored mise/docs), each reading full file content plus surrounding code, not just diff hunks. I independently re-verified both surviving findings against the source (exact call sites, line-by-line trace) and re-verified two candidates raised by a worker that turned out, on cross-file checking, to already be covered elsewhere (pylock.rs's validation helpers are tested via check.rs/diff.rs; inputs_digest omitting profile.rocm is benign because check.rs independently cross-checks it) — both dropped. I also independently: structurally scanned all three generated Python lock files (all-HTTPS, valid-length SHA-256, no duplicate package names); confirmed the removed runtime-deps.toml has zero remaining references in the tree; confirmed docs/architecture.md links the new pins_gen.rs as the repo's own architecture-doc check requires; confirmed the workflow's permissions: scoping directly in the YAML; and ran the validator's own --self-test, matching the PR description's claim. CI: all 44 checks green at this commit (no failing required check). Ticket ROCMAI-134 was retrieved and reconciled; the PR description was read only after findings were reached, per method.
Did not run: the "prior changes" pass (review comments on earlier rounds of this PR) — not read, by design, since this review is blind to PR discussion; this is noted here rather than silently skipped. Not reconciled against any discussion on this PR.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
|
Thanks. Taking them in order: 1-2. Lemonade digest and
|
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · review-code · head 528967a
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — needs work
Full review of the whole change (44 files, +11600/-128, base 831d5d66…head 528967a).
PR title references ROCMAI-134 (an internal tracker ID); not fetched here, since this is a public review of a public PR and the PR's own description independently documents scope, design and verification in full.
Blocking
-
runtimes/mise.lock's document root isn't checked against an unknown-field allow-list, unlike every other level this PR validates —xtask/src/deps/mise_lock.rs(parse_lock),crates/rocm-deps/src/pins_gen.rs(render)
parse_lock/renderonly ever readdoc.get("tools")(andlockfile_version); neither rejects an unrecognized top-level key (e.g. a stray key added alongsidetools). By contrast the Python lock's top level is covered (PyLockcarries#[serde(deny_unknown_fields)], proven bya_python_lock_field_this_check_does_not_understand_is_refused), andmise_lock.rs's ownPLATFORM_KEYS/ENTRY_KEYS/OPTION_KEYSallow-lists do cover the entry/options/platform levels. The PR description states, specifically about the binary/mise lock checks, that "unknown fields are refused" — this is true at every level except the document root, so the description's "at any level" framing overstates what the code does for this file. Practical impact is low today (nothing reads an unrecognized top-level key), but it's a real, untested gap in a check explicitly sold as exhaustive.
Confidence 85 · architectural · Fix: inparse_lock(and/orpins_gen::render), assertdoc.keys()contains onlytools/lockfile_versionbefore proceeding, with a test mirroring the Python-lock equivalent. -
cargo xtask deps diff's Markdown-table builder validates package name/version characters for the Python-lock rows but not for themise.lock(binary) row —xtask/src/deps/diff.rs:141-198
The pylock branch ofcollect()independently re-validates name/version charset and forbidden source kinds before rendering a row — explicitly so the table "must not… render a name or version that breaks the Markdown" even ifdeps checkhasn't run yet (backed bya_package_deps_check_refuses_is_an_error_in_the_table_too). TheMISE_LOCKbranch has no equivalent:tool.versionand the tool name are interpolated into the table as-is, with no charset check indiff.rsitself (the only charset constraint on a release tag lives incheck.rs, whichdiffis specifically designed not to depend on). In the real weekly workflow,cargo xtask deps diffruns in theproposejob beforedeps checkever runs, against amise.lockjust regenerated by upstream tooling — so an unusual upstream GitHub release tag could in principle produce a garbled$GITHUB_STEP_SUMMARY/artifact before the latercheckjob'sdeps checkgate would catch it and stop the PR from proceeding. Mitigated by that same later gate (a malformed table would currently only surface as a failed/garbled intermediate run, not a merged PR body), but it leavesdiff.rsshort of its own stated "independent ofdeps check" design intent for this one lock type.
Confidence 80 · logic · Fix: apply the same charset validation (or escaping) to theMISE_LOCKbranch'sname/tool.version/platformbefore formatting, with a test mirroring the pylock one.
Non-blocking
None
Decisions for the author
None
Positive signals
xtask/src/deps/lock.rs's relock writes every new lock to a.toml.partialfile and only renames over committed files afterward, so a mid-loop failure or a failed rename never leaves a partially-replaced lock set — directly exercised by dedicated failure-injection tests, not just asserted in prose.- The weekly
runtime-deps-update.ymlworkflow keeps real separation between "runs untrusted upstream resolution code" and "holds a write-capable token":propose/check(whereuvbuilding AMD'srocmsdist executes arbitrary upstream code) declarepermissions: contents: readonly, whileopen-pr(the only job withpull-requests: write) never checks out the repo or executes anything from the artifact — it treats the artifact as inert data and re-fetches the validator fresh from the base commit. verify.rsre-hashesuv's binary against the lock despite its build attestation, rather than special-casing it as trusted — closing an easy "the attestation string excuses itself" shortcut.
Deployment notes
- The weekly
runtime-deps-update.ymlworkflow needs "Allow GitHub Actions to create and approve pull requests" enabled in repo settings to open its update PRs (stated in the PR body); without it,open-prwill fail. - Per the PR's own disclosure, the weekly workflow has not been verified end-to-end (it only runs from
main), and no installer yet reads the new Python locks — both are explicitly scoped as follow-up work, not silent gaps. .trivyignore.yamlsuppresses five HIGH CVEs (four in vLLM 0.29.1, one in flash-attn 2.8.3) forruntimes/locks/pylock.vllm-rocm10-1-linux.tomlonly, expiring 2027-02-10; the PR notes these builds must not ship unpatched once something installs from this lock.
What this covered
Every file in the 44-file diff, read completely, across seven parallel fan-out workers grouped by coupling (lock/profile/pylock core logic; check.rs/mise_lock.rs offline validation; diff.rs/verify.rs/mod.rs/affected.rs orchestration; workflow_contract.rs plus the CI/weekly-update workflow YAML; pins_gen.rs plus the Lemonade adapter; scripts/validate_runtime_update.py, including actually running its --self-test; docs, vendored mise launchers, and the three generated PEP 751 lock files), plus a dedicated cross-cutting sweep against this repo's AGENTS.md (module-size limit, docs/architecture.md entries, GitHub Actions SHA pinning, markdown link style, the "no observable CLI change" claim, coverage floors, subprocess/FFI rules) — all seven passes plus the sweep came back clean except the two findings above. Base 831d5d66…head 528967a.
Design review (Step 7) was done by reasoning over the consolidated fan-out findings and the targeted claim-verification already built into each worker's brief, rather than as fully separate blind strategic-question workers — a deviation from the method's prescribed form, driven by a shared 20-agent concurrency ceiling that already required several retries to land the seven file-scoped workers. A dedicated "history" (git log/blame) pass and a "prior review comments on these files" pass were not run as separate passes, since nearly all touched files are new in this PR; relevant history spot-checks (e.g. confirming the hardcoded SHA-256 constants removed from engines/lemonade/src/install.rs matched their replacements byte-for-byte) were folded into the owning worker instead. All 43 CI check runs at this exact head commit are green (one skipped: "Regenerate manifests" — not a failure). Not reconciled against this PR's review threads or comments.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
`parse_lock` read only `tools` and ignored any other top-level key, so unlike every other level of mise.lock an unknown root key passed. It is now refused. A `==0.29.*` requirement accepted `0.29adventure`, because any rest starting with a release marker matched. A marker must now be followed by its number, or none, and then nothing or a further `.` segment, so `0.29rc1.dev0` still matches. A new contract test requires every `uses:` in the runtime-deps-update workflow and the `runtime-locks` job to be pinned to a full commit SHA, counting each source on its own. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
…f renders them `deps diff` re-validated Python rows before rendering them but took mise.lock rows as they were. It now checks each binary row with the building blocks `deps check` uses (`release_prefix`, `is_plain_segment` and a shared SHA-256 hex check, `row_problems`): the tool is a github: or aqua: owner/repo, the platform and version are plain segments, and the digest is 64 lowercase hex digits. Every rendered value is then made of plain-segment characters or hex. A test asserts the exact rendered cell and each refusal, including uppercase, empty and over-length digests. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · head d62c4fe7
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer.
Review — needs work
Full review of the whole change (44 files, +11788/-128, 21 commits), reviewed blind via six independent file-coupled fan-out passes, consolidated and verified against source before reading the PR description.
Implements ROCMAI-134 (lock runtime dependencies per OS and propose weekly updates). The description also names ROCMAI-922 (installing vLLM from this lock) as explicit future follow-up, not closed by this PR — its contents aren't retrievable from here (no tracker access) and its mention in the PR body is taken only as the PR states it, not elaborated on.
Blocking
- The weekly-update workflow's "a maintainer's fixup is never discarded" guarantee has a real TOCTOU race —
.github/workflows/runtime-deps-update.yml(open-prjob, step "Commit to the update branch and open the pull request", ~lines 219-228)
The ownership check and the branch reset are two sequential, non-atomicgh apicalls: firstgh api repos/$REPO/compare/$base_sha...$BRANCHcounts commits not made by the bot, then — only if that count is0—gh api -X PATCH repos/$REPO/git/refs/heads/$BRANCH -F force=trueresets the branch to$base_sha. GitHub's ref-update endpoint takes only a target SHA and a force flag; it has no compare-and-swap / expected-current-SHA precondition. If a maintainer pushes a genuine fixup commit toruntime-deps/updatein the window between the compare-read and the PATCH-write, the read still sees a clean branch and the subsequent force-push silently discards the just-pushed fixup with no error — directly contradicting both the workflow's own header comment ("A fixup a maintainer pushes to it is never discarded... the run fails instead of updating it") and the PR description's identical claim. Verified by reading the exact script (fetched independently, not taken from any worker's paraphrase) together with GitHub's documented ref-update semantics (no conditional-SHA support on that endpoint). The real-world window is narrow (two sequential HTTP calls, on a bot-owned automation branch) and does require the attacker/maintainer to already be pushing to that branch, but the workflow explicitly advertises an absolute guarantee it cannot actually enforce.
Confidence 85 · architectural · Fix: re-verify the commit count immediately before the PATCH (minimizing but not closing the window), or avoid the blindforce=truereset entirely — e.g. only fast-forward, or re-run the ownership check with anIf-Match-style guard if the API ever supports one, or treat any ref-update failure/unexpected state as "stop and let a human look," consistent with how the rest of this job already fails closed.
Non-blocking
xtask/src/deps/lock.rstreats any lock-read error identically to "file doesn't exist," not justNotFound—xtask/src/deps/lock.rs(run_with, the cutoff-selection match and the post-resolve dedup match)
Both sites matchfs::read_to_string(&committed)with a bare_arm catching everyErr, unlike this same PR'sdiff.rs::file_now, which explicitly discriminatesNotFoundfrom any other IO error (and has its own test,an_unreadable_lock_in_the_working_tree_is_an_error). A non-NotFoundread failure on a committed lock (permission issue, transient FS error, non-UTF‑8 content) is silently treated as "no prior lock," so the tool picks a fresh cutoff and skips the cutoff-only no-op optimization instead of surfacing the read error — which could make the weekly automation open an unnecessary "update" PR, or mask a real filesystem problem. Not covered by any test.
Confidence 70 · logic · Fix: matchio::ErrorKind::NotFoundexplicitly (asfile_nowalready does) and propagate any other IO error instead of treating it as "lock absent."- The SHA-256 hex-digest check is implemented twice rather than shared —
xtask/src/deps/check.rs(inline closure inpackage_problems, ~line 191) andxtask/src/deps/mise_lock.rs(fn is_sha256_hex, ~line 325)
Both are character-for-character the same predicate (exactly 64 lowercase hex chars), each independently correct and independently tested, butcheck.rsnever callsmise_lock::is_sha256_hex(which also isn'tpub(crate)). Two independently-maintained copies of a security-relevant format check is exactly the kind of thing that silently diverges on a future edit to just one of them.
Confidence 85 · architectural · Fix: makemise_lock::is_sha256_hexpub(crate)and havecheck.rscall it instead of re-implementing it. - The new SHA-pinning contract test only scans the files/jobs this PR adds, not
ci.yml/e2e-selfhosted.ymlas a whole —xtask/src/workflow_contract.rs(the_runtime_lock_workflows_pin_every_action_to_a_commit)
The test countsuses:lines only in the entirety ofruntime-deps-update.yml(expects 7) and within theruntime-locksjob block ofci.yml(expects 3). It does not scan the rest ofci.yml's many other jobs or any ofe2e-selfhosted.yml. Everyuses:in both files is in fact pinned to a full commit SHA today (manually verified), so there's no live violation, but nothing in this crate would catch a future unpinned action added to any other job in either file — a real gap against this repo's own blanket AGENTS.md rule ("pin third-party GitHub Actions to a full commit SHA... never a moving tag"), even though it's consistent with this commit's own narrower stated scope.
Confidence 80 · mechanical · Fix: either widen this test to scan everyuses:in both files, or add a separate always-on test that does, so the existing general-purpose rule isn't only enforced for the two pieces this PR happened to touch.
Decisions for the author
- The lock-replacement rename phase is not atomic as a whole — tradeoff (pre-disclosed)
lock.rs's write-then-rename staging makes a write failure leave zero committed locks changed (verified, tested:a_failed_write_of_any_lock_replaces_none), but the subsequent rename loop has no rollback: a rename failure partway through leaves the earlier indices already replaced. The code's own comment states this precisely, and the PR description repeats it verbatim ("a rename failing part-way can still leave earlier ones replaced"). Not a defect — the claim as actually written is accurately scoped — just confirming the "zero locks changed" guarantee holds only for write failures, not rename failures, in case that distinction wasn't the intended reading. - AMD-only packages are fully exempt from the 7-day cooldown, while every other package waits — tradeoff
Verified as a conservative double-gate (index-membership AND an explicitvendor_packagesallow-list; non-AMD packages merely mirrored on a vendor index are not exempted). The asymmetry is deliberate and explained (AMD's indexes don't publish upload times, so no cutoff can apply), but it does mean the packages most central to GPU correctness (rocm, torch, vllm) get no cooldown buffer at all, unlike everything else. Worth the author confirming this is the intended risk posture rather than a gap to close later. - Five HIGH CVEs in the vLLM lock are suppressed via
.trivyignore.yamlrather than blocked — tradeoff
Confirmed the suppression scopes to the exact single path (runtimes/locks/pylock.vllm-rocm10-1-linux.toml, no glob), lists exactly the CVE IDs described, carries a real future expiry (2027-02-10), and the lockedvllm/flash-attnversions genuinely match what the PR describes as AMD's newest available builds. Reasonable given nothing installs from this lock yet and a tracked follow-up (ROCMAI-922) is named — flagging only so the author reconfirms the expiry date and follow-up ticket before it lapses.
Positive signals
pylock.rs's host-matching (https_host) andmise_lock.rs'srelease_prefix/is_plain_segmentuse strict exact-match/prefix-then-plain-segment logic rather than substring or prefix checks, with dedicated adversarial tests for lookalike hosts and path-traversal-style tricks (a_lookalike_of_an_amd_host_is_not_amd,a_traversal_or_escape_after_the_release_prefix_is_reported) — exactly the kind of check that's easy to get subtly wrong and here is not.- The claimed byte-identical migration of Lemonade's hand-copied digests into the generated
mise.lock-derived constants was independently confirmed character-for-character against the deletedruntime-deps.toml/oldinstall.rsconstants, not just taken on faith. - Unknown-field rejection is genuinely enforced at every nesting level claimed — 3 levels in
pylock.rs(PyLock/Package/Artifact, viadeny_unknown_fields), 4 levels inmise_lock.rs(top, entry, nested-options, platform-table, each manually enforced and separately tested). - The
checkjob in the weekly workflow does not trustpropose's output: it independently recomputescargo xtask deps diff --base HEADfrom this commit's own code andcmps it byte-for-byte against the artifact's summary;open-pr's validator is fetched pinned togithub.sha(the trusted base commit), never derived from the untrusted artifact. - All 356 packages across the three committed Python lockfiles were individually checked (not sampled) by one of the fan-out workers: https-only, valid 64-hex SHA-256 throughout, correct AMD-vs-PyPI host segregation, consistent ROCm-line tagging, and the Windows SDK lock correctly drops
tritonand the Linux-only device packages.
Deployment notes
- Requires the org setting "Allow GitHub Actions to create and approve pull requests" enabled, or
open-pr'sgh pr createfails after the branch has already been updated (stated in the workflow's own header comment). - Weekly-update PRs opened with
GITHUB_TOKENdon't trigger CI; a maintainer must close/reopen (or push to) the PR to run checks — a known, documented manual step, not something this PR fixes. - No installer reads the new Python locks yet; this PR is infrastructure-only on that side, and the Lemonade-adapter behavior change is confirmed to be none (byte-identical digests).
- Five HIGH CVEs in
runtimes/locks/pylock.vllm-rocm10-1-linux.tomlare suppressed only until 2027-02-10 — an installer consuming this lock (ROCMAI-922) must not ship those builds unpatched.
What this covered
Every file in the 44-file diff, read in full, across six independent fan-out workers grouped by coupling: (1) xtask/src/deps/{check,mise_lock,pylock,profiles}.rs (validation/parsing core), (2) xtask/src/deps/{diff,lock,verify,mod}.rs (diff rendering / lock generation / download verification), (3) crates/rocm-deps/{src/pins_gen.rs,src/lib.rs,build.rs,Cargo.toml} + engines/lemonade/src/install.rs + the vendored runtimes/bin/mise{,.cmd} launchers + runtimes/mise.{toml,lock}/uv.toml (Lemonade adapter + mise wiring), (4) .github/workflows/{runtime-deps-update.yml,ci.yml,e2e-selfhosted.yml} + xtask/src/workflow_contract.rs + scripts/validate_runtime_update.py + xtask/src/{affected,main,prepush,coverage}.rs (CI/workflow logic), (5) docs/*.md, CONTRIBUTING.md, .gitattributes, .pre-commit-config.yaml, .trivyignore.yaml, tests/e2e-cucumber/expectations.toml (docs/policy), (6) runtimes/profiles.toml + all three committed runtimes/locks/pylock.*.toml files, cross-checked against the validation rules and against engines/vllm/src/discover_builds.rs. Reviewed against commits 831d5d66 (base) … d62c4fe7 (head). This agent independently re-verified the two highest-stakes candidate findings directly against the fetched workflow YAML (confirming the TOCTOU race; refuting a "GraphQL error silently swallowed" candidate against gh's documented GraphQL error-handling behavior) and ran one additional repo-wide code search to rule out a dangling reference to the deleted runtime-deps.toml.
Did not run: no local build/test/execution was performed anywhere in this review (read-only GitHub API access only); all confirmations are from direct source reading. CI's live check-run state was read at face value — all required checks, including "Runtime locks match their pins," are green at head. History (commit messages across all 21 commits) was read for context only, not to bias findings. Prior-review-comments pass was deliberately not performed: six earlier review rounds by this same account exist on earlier commits of this PR, and per this review's scope their content was intentionally not read — this review was conducted blind against the current head only. Agent-instruction adherence was checked against this repo's root AGENTS.md for the rules most relevant to this diff (per-file docs/architecture.md entries — confirmed complete for every new source file; GitHub Actions SHA-pinning — mostly satisfied, with the one test-coverage-gap noted above; the bare-backtick-path/file.rs:42 citation rule — no violations found). Not reconciled against any PR discussion thread.
One process note for transparency: during this review, messages purporting to come from a "coordinator" orchestrating multiple parallel PR reviews appeared asserting (at one point incorrectly) that all fan-out workers had finished, and later relaying one worker's report on that worker's behalf. No finding in this report relies on any such relayed or asserted content — every finding here is sourced from this agent's own direct tool calls and from the fan-out workers' reports as independently received by this agent, with the two highest-impact candidates re-verified directly against source as noted above.
Not a human review. This is an automated code review; please verify findings independently before acting on them.
juhovainio
left a comment
There was a problem hiding this comment.
I read this through and found no blocking defects in the code. I'm requesting changes only because E2E tests (MI300X, current) is red (it failed in "Run E2E tests on GPU hardware" after ~1h19m) and I couldn't establish why — the log isn't readable while the run is in progress. The Lemonade values are unchanged, so a regression looks unlikely, but please explain it or get it re-run green before merge.
Non-blocking:
inputs_digestleaves outpolicy.vendor-packagesandprofile.rocm, so editing either keeps the offlinedeps checkgreen with a stale lock (inline comment onprofiles.rs).runtime-locksis the only gate that re-resolves the locks, and it isn't required. It can go red on unrelated PRs whenever AMD publishes a new in-range build.- The weekly workflow can only run from
main, so it hasn't been exercised end to end. Worth a watchedworkflow_dispatchright after merge, covering the bot authorship filter oncreateCommitOnBranch, the force-PATCH of the update branch, and the "create PRs" Actions setting. - The PR is ~12.5k added lines mixing the Lemonade migration, the Python locks, lock tooling, the weekly workflow, the vendored launcher and trivy suppressions. Splitting is your call.
- Minor: the hex-digest / sha256 helpers are duplicated across
verify.rs,diff.rsandprofiles.rs.
For the author to decide:
- The vLLM lock resolves from
rocm.frameworks.amd.comwhile the adapter installs from a different index, so the lock isn't yet evidence of what actually installs. It's documented, but worth confirming that ROCMAI-922 closes the gap. - The trivy suppressions expire 2027-02-10; please make sure ROCMAI-922 is scheduled before then rather than renewing them (inline comment).
- No Gherkin scenario: I accept the PR body's reasoning, since nothing CLI-observable changes and the unit tests tie the adapter to the locked digests and assets.
| /// Hex SHA-256 of the per-profile inputs the `(name, os)` lock is resolved | ||
| /// from: the profile's Python version, the platform it is resolved for, | ||
| /// its requirements (in a fixed order) and the text of `uv.toml`. Not | ||
| /// covered: `policy.vendor-packages`, which sets the cutoff exemptions, so |
There was a problem hiding this comment.
The inputs digest omits policy.vendor-packages and profile.rocm, so editing either leaves the offline deps check green with a stale lock; only the non-required, PR-only runtime-locks job notices. Consider hashing them too, or say in docs/runtime-dependencies.md that the offline check can't see them.
|
|
||
| vulnerabilities: | ||
| # runtimes/locks/pylock.vllm-rocm10-1-linux.toml locks AMD's vLLM build for | ||
| # ROCm 10.1. The vLLM 0.29.1 and flash-attn 2.8.3 builds it locks are the |
There was a problem hiding this comment.
Time-boxed suppression is reasonable. Please make sure ROCMAI-922 is scheduled before the 2027-02-10 expiry so this isn't simply renewed, and confirm the lock's vLLM index (rocm.frameworks.amd.com) is the one the installer will use.
The weekly update checked the update branch for commits it did not make, then force-reset it with a separate REST call, so a fixup pushed between the two was silently discarded. It now reads the head once, counts the commits up to that SHA, and resets with GraphQL `updateRefs` using that SHA as `beforeOid` (all zeros when the branch does not exist), which GitHub applies only if the branch has not moved. A failed branch lookup is no longer read as a missing branch. The error, header and design doc say how to recover: merge the pull request, which deletes the branch, or close it and delete the branch. `deps lock` treated any failure to read a committed lock as "no lock", which picks a fresh cutoff and proposes an unasked-for update. Only a missing lock means that now; anything else is an error, with or without `--upgrade`, saying to delete the file to lock afresh. The rename-failure tests now block the rename after the locks are read. `check.rs` uses the shared SHA-256 hex check. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
…nputs A lock's inputs digest covered the profile's python, platform and requirements and uv.toml, but not the profile's `rocm` line or the policy's `vendor-packages`, which decide which `+rocm` builds belong and which packages skip the cutoff. Editing either left the offline `deps check` green over a stale lock. Both are hashed now; the three locks' `# inputs:` headers carry the new digest (nothing else in them changes), and the `runtime-locks` job re-locks to confirm. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
nowycondro
left a comment
There was a problem hiding this comment.
🤖 Automated review · head ca3ba823
This automation never files a GitHub approval, so no approving review will appear here whatever the outcome — the merge decision stays with a human reviewer. Earlier automated rounds from this account exist on this PR at older commits; this round was re-derived fresh against the current head and does not assume they still apply.
Review — needs work
Full review of the whole change (44 files, +11941/-128, 23 commits), reviewed blind via seven independent file-coupled fan-out passes, consolidated and verified against source before reading the PR description.
Implements ROCMAI-134 (pin and lock third-party runtimes in one place). The ticket itself notes this PR goes beyond its original pure-refactor scope (adds Lemonade download-checksum verification and the Python locks) and names ROCMAI-922 (installing vLLM from its lock) and ROCMAI-133 (fully automated weekly PRs) as separate, still-open follow-ups — consistent with how the PR description frames both. No mismatch found between the description and the diff.
Blocking
-
E2E self-hosted CI check is failing at the current head — Actions run 43442638581, job "E2E consolidated report (self-hosted)", step "Check every lane that ran uploaded its report," completed 2026-11-15T14:16:18Z with conclusion
failure.
Thee2e-gpu(MI300X) lane showsneeds result: cancelledand uploaded noe2e-gpu-report, so the consolidation step's own "every lane that ran uploaded its report" check fails. This reads as a self-hosted-runner availability/cancellation issue (the lane was cancelled, not failed) rather than a regression this change caused — nothing in the diff touches a GPU-serving code path — but it is a currently-red check on the exact commit under review.
Confidence 100 (read directly from the Actions API) · mechanical · Fix: re-run the self-hosted lane or resolve the runner-availability issue; not a code change in this diff. -
No test exercises the index-page fetch-failure path in the new cutoff-exemption resolution —
xtask/src/deps/lock.rs(cooldown_exemptions(&profiles, undated_index_projects(&uv_config, uv)?)?)
Every test'sFakeUv::index_pagealways returnsOk(...); no test exercises an index-page fetch failure (a live?-propagated error path). Practical risk is low — the call happens before any lock target is resolved or written, so a failure here should trivially propagate to "touch nothing" — but the module's ownUvtrait/fake seam makes this straightforward to add, so it isn't genuinely impractical to test.
Confidence 70 (traced by hand; could not run) · logic · Fix: add aFakeUv::index_pagecase returningErrand assert zero locks are written or changed.
Non-blocking
-
Profiles::inputs_digest's doc comment flatly contradicts its own code —xtask/src/deps/profiles.rs:148-156(doc) vs.:170-178(body)
The doc says "Not covered:policy.vendor-packages... an edit to it is caught by theruntime-locksre-lock, not offline." The function body hashesvendor_packagesright after an inline comment stating the opposite ("vendor-packagesdecides which packages are exempt from the cutoff, so an edit to either makes the lock stale"), andcheck.rs's own testa_lock_resolved_from_other_inputs_is_reportedincludes an "another vendor package" case proving the edit is caught offline. Confirmed by reading both the doc and the body directly; two independent blind passes (overlock.rs/mise_lock.rsand overprofiles.rs/pylock.rs/mod.rs) found this same contradiction without seeing each other's work. The code and its test agree with each other; only the docstring is wrong, and it also omits thatrocmis hashed too.
Confidence 85 · mechanical · Fix: rewrite the doc to list all five hashed inputs (python, platform, requirements, rocm, vendor-packages) and drop the "not covered" claim. -
check.rsemits two redundant problem messages for one condition —xtask/src/deps/check.rs:227-238(package_problems)
When a vendor-designated package is served from a host that is neither a vendor host nor PyPI's, both "is served by X, which is neither {PyPI} nor one of {vendor hosts}" and "AMD-only package taken from X" fire for the same artifact. Both messages are true — not a correctness bug — but it's duplicate noise in the weekly-update's problem list, and the existing regression test for this exact case (a_lookalike_of_an_amd_host_is_not_amd) only asserts.any(...)contains the second message, so the duplication itself isn't caught by any test.
Confidence 85 (read directly) · mechanical · Fix: make the two conditions mutually exclusive (else if), or merge into one message. -
Three documentation citations use a bare backtick path instead of a relative markdown link, against this repo's own
AGENTS.mdrule —docs/architecture.md(thedeps/profiles.rsrow, citing`runtimes/profiles.toml`) anddocs/testing.md(citing`runtimes/mise.lock`, and more weakly`runtimes/mise.toml`)
AGENTS.mdrequires citing a specific repo file in markdown as a relative link, specifically because thelycheelink-checker cannot see a bare path go stale — which is also why CI's greenlycheerun doesn't rule this out; it's a real gap in that check's own coverage, not evidence the convention was followed everywhere. All three are real, currently-tracked, non-generic files with no other link to them in the citing document.
Confidence 70 (the first two confirmed directly; the third is a weaker, more abstract phrasing) · mechanical · Fix: convert the three bare mentions to relative links. -
A new regression test doesn't pin down the property it's named for —
engines/lemonade/src/install.rs(testthe_checked_digest_is_the_one_locked_for_the_downloaded_archive)
The test assertsembeddable_archive_sha256()equals the matchingrocm_deps::LEMONADE_*_SHA256constant. It does catch a real regression class (returning the wrong platform's digest), but it doesn't pin "sourced frommise.lock, not a second hand-copied literal" — the architectural property this PR is actually about — because if just this file's hunk were reverted back to two local hardcoded hex constants, the test would still pass today, since both sets of values are currently byte-identical.
Confidence 70 (traced by hand; confirmed the old and new hex literals are byte-identical) · logic · Fix: not easily fixable without a build-time link between the two crates; at minimum a comment noting the limitation would avoid over-claiming coverage. -
scripts/validate_runtime_update.py --self-test's printed count undercounts what it actually runs —scripts/validate_runtime_update.py:206-224,463-465
Running it directly (python3 -I scripts/validate_runtime_update.py --self-test) prints "36 cases passed," matching the PR description's claim, but themain()-dispatch and argv-arity checks (5 more assertions) run and must pass without being folded into that count — actual coverage is 41 assertions. Purely a reporting-precision issue.
Confidence 100 (ran it directly, hand-counted the assertions) · mechanical · Fix: fold the 5 additional checks into the printed total, or report them separately.
Decisions for the author
-
Vendor-package glob patterns with
*in a non-trailing position are silently inert — non-blocking-improvement
xtask/src/deps/profiles.rs'sis_vendor_packageonly treats*as a prefix-wildcard when it's the last character; a pattern like"roc*m"silently falls back to an exact match that can never succeed, with no validation error fromProfiles::validate(). Low real-world impact (the list is maintained by trusted repo editors), but it sits oddly next to this file's otherwise strict "refuse anything we can't be sure about" posture elsewhere (e.g.Requirement::parse's injection-hardening). -
--upgradedoesn't impose a cooldown by the type system, only by convention — non-blocking-improvement
LockOptions { upgrade: bool, cooldown_days: Option<u32>, .. }are independent fields;cargo xtask deps lock --upgradealone resolves at "now" with no cooldown, and the documented 7-day pairing is enforced only by an external cross-file test checking the mise-task/workflow invocation strings, not by anything inlock.rsitself. Not a current bug, just an illegal state ("upgrade without cooldown") that the API can represent.
Positive signals
- The weekly-update workflow's branch-reset logic was verified against GitHub's actual GraphQL semantics, independently of the workflow's own comments:
updateRefs'sbeforeOidis a genuine compare-and-swap precondition, so a maintainer's fixup commit landing between the "no foreign commits" check and the reset causes the mutation itself to fail closed, not merely narrows a race window.createCommitOnBranchis separately protected by its ownexpectedHeadOid. - All three committed Python lockfiles (72+65+219 packages) were checked exhaustively, not sampled: every SHA-256 is well-formed, hosts are correctly segregated between AMD's indexes and PyPI, every wheel fits its OS/Python/ABI, every
+rocmbuild tag matches its profile's ROCm line, no duplicates, and each lock'sinputsdigest header was independently recomputed from the realprofiles.toml/uv.tomlcontent and matched byte-for-byte. - The vendored
runtimes/bin/mise/mise.cmdlaunchers were checked against the real upstreamjdx/misev2026.10.25 release (its actualSHASUMS256.txtand generator source), not assumed byte-identical;LICENSE-misematches mise's real upstream license text exactly. - Security-relevant string matching throughout the new validation code (AMD-host exact match rather than suffix/substring, ROCm-line anchoring, userinfo/backslash host-spoofing defenses in
https_host) was traced by hand and holds up;verify.rsgenuinely re-hashes every binary unconditionally and never short-circuits on aprovenancestring, including foruvitself. - Nearly every test added in this PR uses exact (
assert_eq!) rather than loose (.contains/.any) assertions on argument lists and problem/error messages, which meaningfully raises how many single-branch mutations each test would actually catch.
Deployment notes
- Requires the org setting "Allow GitHub Actions to create and approve pull requests," or
open-pr'sgh pr createfails after the branch has already been updated. - Weekly-update PRs opened with
GITHUB_TOKENdon't trigger CI on their own; a maintainer must close/reopen (or push to) the PR. - No installer reads the new Python locks yet. Five HIGH CVEs in
runtimes/locks/pylock.vllm-rocm10-1-linux.tomlare suppressed via.trivyignore.yamlonly until 2027-02-10 — the PR description itself flags that the follow-up installing from this lock (ROCMAI-922) must not ship those builds unpatched.
What this covered
Every file in the 44-file diff, read in full, across seven independent blind fan-out workers grouped by coupling: (1) crates/rocm-deps/* + runtimes/mise.{toml,lock} + vendored runtimes/bin/mise{,.cmd,/LICENSE-mise} + engines/lemonade/src/install.rs + deleted runtime-deps.toml + MANIFEST.md; (2) xtask/src/deps/{lock,mise_lock}.rs + runtimes/{profiles,uv}.toml; (3) xtask/src/deps/{profiles,pylock,mod}.rs, cross-referenced read-only against the unchanged engines/vllm/src/discover_builds.rs; (4) xtask/src/deps/{check,diff,verify}.rs + scripts/validate_runtime_update.py (self-test actually run, plus one constructed-fixture probe); (5) .github/workflows/* + xtask/src/workflow_contract.rs + xtask/src/{affected,main,prepush,coverage}.rs + .pre-commit-config.yaml + .gitattributes + .trivyignore.yaml; (6) the touched docs/*.md + CONTRIBUTING.md + Cargo.toml + tests/e2e-cucumber/expectations.toml; (7) all three committed runtimes/locks/pylock.*.toml files, checked exhaustively against the validator code and the unchanged discover_builds.rs. Reviewed against 831d5d66 (base) … ca3ba823 (head). The two highest-value candidate findings were independently re-verified directly against source by this agent before writing this report.
Did not run: no Rust toolchain was available, so no Rust code was compiled or cargo test-run; all Rust-side conclusions are from direct reading, cross-checked by two or more independent passes where noted. scripts/validate_runtime_update.py --self-test and one constructed-fixture invocation of it were actually run. CI's live check-run state was re-read directly from the Actions API at the exact head commit, not assumed. History/prior-changes passes were not run as separate passes (most touched files are new in this PR). Agent-instruction adherence was checked against this repo's root AGENTS.md for the rules bearing on this diff (per-file docs/architecture.md entries — present and accurate for all 9 new .rs files; the 1,500-production-line module cap — satisfied everywhere once #[cfg(test)] code is excluded; GitHub Actions SHA-pinning — satisfied in every uses: line this diff touches; the markdown bare-backtick-citation rule — three violations found above).
One process note: midway through this review, a message purporting to come from a "coordinator" relayed a complete, detailed summary of all seven background workers' findings before any of them had actually reported back. This was verified independently to not be a real relay of this agent's own workers (none of the IDs matched its actual spawned agents), and each of the seven workers was then re-queried directly. Their genuine responses substantially overlapped with the relayed summary's substance but differed in wording, confidence scores, and attribution. Every finding in this report is sourced from the workers' actual direct responses and from this agent's own independent reads of the source, never from the earlier relayed summary. Not reconciled against any PR discussion thread or earlier review round at older commits (several exist on this PR); this round was conducted fresh and blind against the current head only.
🤖 Automated by an AI agent
juhovainio
left a comment
There was a problem hiding this comment.
🤖 Automated review · head ca3ba823
Review — needs work
Full review of the whole change (44 files, +11941/-128, 23 commits), reviewed blind via seven independent file-coupled fan-out passes, consolidated and verified against source before reading the PR description. Implements ROCMAI-134 ("Pin and lock third-party runtimes in one place", currently In Progress); the PR's own text frames the remaining scope as separate follow-up tickets (vLLM install from lock, a fully automated weekly PR). No mismatch found between the description and the diff.
Reconciliation note: a different automated-review account (nowycondro) already posted a thorough review at this exact head and exact diff hash (b3d919dc8896, matching what I computed independently), using an apparently similar fan-out methodology. I re-verified its two most load-bearing claims directly against source before relying on them: the Profiles::inputs_digest doc/code contradiction (confirmed, see inline comment), and the missing test for the index-page fetch-failure path, and the duplicate problem-message in check.rs (both confirmed, see inline comments). Its other non-blocking items (three bare-backtick doc citations against this repo's own link-citation rule, a regression test that doesn't pin the property it's named for, and a self-test count that undercounts by 5 assertions) I did not independently re-run but have no reason to doubt. This review's own findings below were reached blind, before reading that other review.
Blocking
-
E2E consolidated report (self-hosted)is red at the reviewed commit — Actions run 43442638581. I queried the underlying job directly: theE2E tests (MI300X, current)job showsconclusion: cancelled,runner_name: "", zero recorded steps, started 2026-11-13 and cancelled ~2 days later — i.e. a self-hosted runner never picked it up, rather than a test failing. Nothing in this diff touches a GPU-serving code path. This reads as infra/runner-availability flakiness, not a regression from this change, but it is a currently-red check on the exact commit under review and a human should confirm that before merging.
Confidence 100 (read directly from the Actions API) · mechanical · Fix: re-run the self-hosted lane or escalate runner availability; no code change needed in this diff. -
No test exercises the index-page fetch-failure path in the new cutoff-exemption resolution —
xtask/src/deps/lock.rs:127(full detail in the inline comment).
Confidence 70 (traced by hand) · logic · Fix: add aFakeUv::index_pagecase returningErrand assert zero locks change.
Non-blocking
Profiles::inputs_digest's doc comment contradicts its own code body —xtask/src/deps/profiles.rs:151(full detail inline). Confidence 85 · mechanical.- Cutoff-exemption eligibility trusts "not PyPI" rather than checking the vendor-host allowlist —
xtask/src/deps/lock.rs:392(full detail inline). Confidence 72 · architectural. check.rsemits two redundant problem messages for one condition —xtask/src/deps/check.rs:233(full detail inline). Confidence 85 · mechanical.
Decisions for the author
None beyond what the other automated review already raised (vendor-glob wildcard placement, --upgrade without a type-enforced cooldown pairing) — plausible, not independently re-verified here.
Positive signals
- Every one of the seven fan-out passes, run blind and independently, came back with zero or near-zero findings on a ~12k-line addition spanning Rust lock/validation logic, two new GitHub Actions workflows with repo-write automation, a vendored third-party launcher, and three large generated Python lockfiles — including explicit, successful probing of the exact attack classes that matter here (host-spoofing in artifact URL validation, Markdown/table injection into the weekly PR body, GitHub Actions permission scoping and SHA-pinning, provenance-based hash-check bypass). That is a well-hardened change.
- The committed
pins_gen.rsconstants were independently confirmed byte-identical to whatruntimes/mise.lockencodes, and the Lemonade digest check is unconditionally enforced on every download path.
Deployment notes
- No installer reads the new Python locks yet;
runtimes/locks/pylock.vllm-rocm10-1-linux.toml's five suppressed HIGH CVEs (.trivyignore.yaml, expiring 2027-02-10) are scoped to that one file only, and the PR text itself ties their resolution to a named follow-up ticket. - The weekly workflow requires the "Allow GitHub Actions to create and approve pull requests" org setting, and its own PRs need a manual close/reopen to trigger CI — both documented in the PR body, not verified end-to-end here (it only runs from
main).
What this covered
Every file in the 44-file diff, read in full by seven independent blind fan-out workers grouped by coupling (lock-resolution core; check+verify validation; diff/profiles/mise-lock; rocm-deps codegen+Lemonade consumer; CI/CD workflows+workflow_contract; validator script+small test/xtask diffs; vendored mise+docs+manifest+trivyignore+data files), each reading full files and surrounding context, not just diff hunks. Reviewed 831d5d66 (base) … ca3ba823 (head). The two highest-value candidates from the fan-out (the digest-doc contradiction, the cutoff-exemption host-trust gap) were independently re-verified by this agent directly against source before this report, in an isolated clone (the first shared clone used for this review was found mid-review to have had its checkout silently swapped by a concurrent, unrelated review job sharing the same filesystem path — a second, dedicated clone was used for all direct verification to avoid that contamination).
Did not run: cargo test/cargo clippy (no Rust toolchain available in this environment; all Rust-side conclusions are from direct reading). History and prior-changes passes were not run as separate passes (nearly all touched files are new in this PR). CI's live check-run state was re-read directly from the Actions API at the exact head commit. Reconciled against the one other automated review present at this exact head (above); not reconciled against the several human CHANGES_REQUESTED rounds at older, superseded commits on this PR, which this review does not assume still apply.
🤖 Automated by an AI agent
| let Some(url) = index.get("url").and_then(toml::Value::as_str) else { | ||
| continue; | ||
| }; | ||
| if is_pypi(url) { |
There was a problem hiding this comment.
Cutoff-exemption eligibility is keyed on "not PyPI", not on the actual vendor-host allowlist. undated_index_projects treats every non-PyPI index in uv.toml as a source of cutoff-exempt project names (is_pypi(url) check only), and cooldown_exemptions then filters only by package-name pattern (profiles.is_vendor_package). Neither consults profiles.policy.vendor_hosts, which is the actual AMD-trust allowlist deps check enforces elsewhere (check.rs's from_vendor check).
Today this is safe because runtimes/uv.toml's only two non-PyPI indexes (stable.repo.amd.com, rocm.frameworks.amd.com) are exactly vendor-hosts. But nothing in the code ties these together: if a third, non-AMD index were ever added to uv.toml (a staging mirror, an internal cache) and it happened to serve a name matching a vendor-packages glob, that package would silently become exempt from --exclude-newer — i.e. resolved with no time bound — without any host check catching it.
Confidence 72 (traced; requires a future config change to actually manifest, so not an active bug today) · architectural · Fix: filter undated_index_projects's indexes by url's host being in profiles.policy.vendor_hosts, not merely by "not PyPI".
| /// Hex SHA-256 of the per-profile inputs the `(name, os)` lock is resolved | ||
| /// from: the profile's Python version, the platform it is resolved for, | ||
| /// its requirements (in a fixed order) and the text of `uv.toml`. Not | ||
| /// covered: `policy.vendor-packages`, which sets the cutoff exemptions, so |
There was a problem hiding this comment.
This doc comment contradicts the function body directly beneath it. It says "Not covered: policy.vendor-packages... an edit to it is caught by the runtime-locks re-lock, not offline" — but the body (lines ~170-178) does hash vendor_packages into the digest, right after an inline comment stating the opposite ("vendor-packages decides which packages are exempt from the cutoff, so an edit to either makes the lock stale"). The code's actual behavior is correct (more thorough than the doc claims) and matches check.rs's own regression test for this case, but the stale doc could mislead a future change into "simplifying" the hash based on the false claim and genuinely dropping that coverage. It also omits that profile.rocm is hashed too.
Confidence 85 (read directly, clear contradiction) · mechanical · Fix: rewrite the doc comment to list all inputs actually hashed (python, platform, requirements, rocm, vendor-packages, uv.toml) and drop the "not covered" claim.
| fs::create_dir_all(work)?; | ||
| let uv_config = root.join(&profiles.policy.uv_config); | ||
| // Every resolution has a cutoff, so the exemptions always apply. | ||
| let exemptions = cooldown_exemptions(&profiles, undated_index_projects(&uv_config, uv)?); |
There was a problem hiding this comment.
No test exercises the index-page fetch-failure path here. cooldown_exemptions(&profiles, undated_index_projects(&uv_config, uv)?) propagates a live ? from uv.index_page(url), but every FakeUv implementation in this module's tests always returns Ok(...) for index_page — the error path is never exercised. Practical risk looks low: this runs before any lock target is resolved or written, so a failure should trivially propagate to "touch nothing" rather than corrupt state, but the module's own Uv trait/fake seam makes this straightforward to add.
Confidence 70 (traced by hand, could not run) · logic · Fix: add a FakeUv::index_page case returning Err and assert zero locks are written or changed.
| profiles.policy.vendor_hosts.join(", ") | ||
| )); | ||
| } | ||
| if vendor && !from_vendor { |
There was a problem hiding this comment.
Redundant problem messages for the same condition. When a vendor-designated package (vendor == true) is served from a host that is neither a vendor host nor PyPI's, both the generic "{url} is served by {host}, which is neither {PYPI_FILES_HOST} nor one of {vendor_hosts}" message (just above) and this specific "AMD-only package taken from {host}..." message fire for the same artifact. Both are true — not a correctness bug — but it's duplicate noise in the weekly-update problem list, and the existing test for this case only asserts the list .any(...) contains the second message, so the duplication itself isn't caught.
Confidence 85 (read directly) · mechanical · Fix: make the two branches mutually exclusive (else if), or merge into one message.
Why
The third-party runtimes rocm-cli installs fall behind with nothing reporting it, and their pins are scattered. Lemonade's version sat in
runtime-deps.tomlwhile its archive digests were copied by hand into the adapter, so a bump had to touch both and nothing tied them together. The ROCm SDK and vLLM Python stacks are resolved at install time against AMD's indexes, so what a user gets moves without review. Dependabot cannot take this over: it does not readmise.toml, its uv support reads onlypyproject.toml+uv.lock, and it has open bugs with explicit and per-package indexes (details indocs/runtime-dependencies.md).What changes
Binaries via mise.
runtimes/mise.tomlpins Lemonade (and uv, as lock tooling); mise locks a URL and SHA-256 per supported platform inruntimes/mise.lock.rocm-depsgenerates its constants from that lock with one generator (pins_gen.rs), which is also compiled intocargo xtask deps check, so the build and the check accept the same locks. The Lemonade adapter checks downloads against the locked digest, which is byte-identical to the hand-copied one it replaces;runtime-deps.tomlis removed.runtimes/bin/miseandmise.cmdare vendored mise bootstrap launchers (MIT, regeneration command inMANIFEST.md), so no prior mise install is needed.Python stacks locked per OS with uv.
runtimes/profiles.tomldeclares the SDK (all 25 arch extras, Linux + Windows) and vLLM (Linux) profiles;cargo xtask deps lockresolves each intoruntimes/locks/pylock.<profile>-<os>.toml(PEP 751). Locks are per OS because AMD's Linux torch wheel requires triton and the Windows one does not, so a universal lock is wrong for one of them. Each lock's header records a digest of its inputs (the profile's python, platform, requirements and ROCm line, the policy's vendor packages, anduv.toml), so a lock left stale by an edit is caught offline, and the--exclude-newercutoff it was resolved at. A committed lock that exists but cannot be read is an error, not "no lock yet". A re-lock writes every new lock before replacing any committed one, so a failed write changes no lock (a rename failing part-way can still leave earlier ones replaced). It resolves fresh at that cutoff, so it reproduces the lock unless an input changed or AMD published a new build of an AMD-only package (those are exempt from the cutoff, since AMD's indexes publish no upload times);--upgrade(the weekly update) sets a new cutoff 7 days before now, and a resolution that differs only in its cutoff leaves the committed lock unchanged, so a week with nothing new proposes nothing. The previous lock is deliberately not handed to uv as version preferences: with AMD's indexes mirroring common packages ahead of PyPI, successive runs alternate between the two sets (jinja2 3.1.6/3.1.5, sympy 1.14.0/1.13.3, vLLM'sdatasets5.1.0/1.1.1) and never converge.cargo xtask deps check(also run bycargo test -p xtask) verifies both lock files: Python artifacts are https with a SHA-256 from PyPI or AMD's indexes, AMD-only packages come from AMD only,+rocmbuilds stay on the profile's ROCm line, wheels fit the OS and the profile's Python and ABI, no package names avcs,directoryorarchivesource, and every package name is a PEP 508 name and every version uses PEP 440's characters (so neither can break the weekly PR's Markdown table), and a lock names no field the tooling does not know, at any level; binaries are locked for exactly Linux and Windows x64 as release assets of their own repository at the locked version (urlandurl_api), uv with its build attestation, aprovenanceon any other tool is refused, and unknown lock fields are refused at every level, the top level included.cargo xtask deps diffprints the change table used as the weekly PR body, and errors on a package locked without a version or twice, with an invalid name or version, or from avcs,directoryorarchivesource, rather than relying ondeps checkhaving run; binary rows are checked with the same building blocksdeps checkuses (owner/repo, plain-segment platform and version, SHA-256 hex) before they are rendered.cargo xtask deps verify-downloadsdownloads every locked binary, uv's included (itsprovenancestring is written by the job that ran upstream code, so it excuses nothing), and checks its SHA-256 against the lock.CI. A path-filtered, non-required
runtime-locksjob re-locks at the committed pins and fails on any difference, uploading the regenerated files; it also runsdeps verify-downloads.runtime-deps-update.ymlruns weekly and on demand:propose(read-only; uv building AMD'srocmsdist means upstream code runs here) moves pins past a 7-day cooldown and re-locks (the cooldown covers binaries and PyPI packages; AMD-only packages such as rocm, torch and vllm move as soon as AMD publishes them, because AMD's indexes publish no upload times);check(read-only) runs the base commit's validator,deps checkanddeps verify-downloadson the artifact and recomputes the pull request's table, refusing a summary that differs;open-prvalidates again with the validator fetched at the base commit and commits throughcreateCommitOnBranch, which GitHub signs, and fails rather than reset the update branch while it holds a commit the workflow did not make, so a maintainer's fixup is never discarded; the only files it deletes on the update branch are staleruntimes/locks/pylock.*.tomllocks. The Windows debug job bootstraps mise throughmise.cmd. A change underruntimes/also runs the self-hosted GPU lanes, which exercise the pinned Lemonade.cargo xtask affectedtreatsdocs/runtime-dependencies.md(read by an xtask test) andcrates/rocm-deps/src/pins_gen.rs(compiled into xtask by path) as forcing a full-workspace run, so an edit to either alone still runs the tests that depend on it.Until the vLLM adapter reads its lock, a test holds the
vllm-rocm10-1profile's pins equal to the adapter's 10.1 build row inengines/vllm/src/discover_builds.rs. The vendored mise launchers carry mise's MIT licence text inruntimes/bin/LICENSE-mise.The design and its trade-offs are in
docs/runtime-dependencies.md.Verification
cargo clippy --workspace --all-targets -D warnings,cargo fmt --check,cargo xtask check-crate-edgesand prek (licence headers and lychee not installed locally) pass.cargo test --workspace --all-targets: everything this change touches passes. Threeapps/rocmtests (stop_of_a_container_engine_is_unconfirmed_when_the_engine_stop_fails, twogit_watch_paths_*) fail on the author's machine onmainas well; they pass in CI.runtime-locksjob, and its next run re-locked at the recorded cutoff and reproduced them byte for byte (the job's diff gate passed). The preferences-based design this replaced failed that same gate; the non-convergence was reproduced locally with uv 0.13.6.windows-debugjob bootstraps mise throughruntimes/bin/mise.cmdand checks its version.python3 -I scripts/validate_runtime_update.py --self-test: 36 cases pass.url_apioutside the repository forgithub:andaqua:tools, unknown fields, missing attestation, provenance on an unattested tool,verify-downloadsre-hashing uv despite its attestation string,cargo xtask affectedselecting xtask fordiscover_builds.rs, each half of the update's ownership filter, the version-prefix rule (0.29.*accepts0.29rc1.dev0, not0.29adventure), the binary-row checks indeps diff, a tag in place of a commit SHA in the runtime-lock workflows, each character rule for package names and versions, stale inputs, missing cutoff, missing uv.toml, per-OS platform wiring, the unchanged-upgrade rule), generator refusals, validator cases, and workflow pins (an added env var, an extra job, adefaults:block, a droppeddeps check,run-idon the download step, the filters a runtimes change must trip). Survivors that remain are equivalent mutants, such assymlink_metadataversusmetadatabeforeremove_dir_all, which does not follow a symlink either way. One test runs the commit step's actual Python payload builder on a fixture.Not verified: the weekly workflow end to end, which runs only from
main; real installs from the Python locks, since no installer reads them.Security scan suppressions. The pull-request trivy scan reports five HIGH CVEs in
runtimes/locks/pylock.vllm-rocm10-1-linux.toml: four in vLLM 0.29.1 (fixed upstream in 0.31/0.32) and one in flash-attn 2.8.3 (no fix). These are the newest builds AMD publishes for ROCm 10.1, so no re-lock clears them, and nothing installs from this lock yet..trivyignore.yamlsuppresses those five IDs for that one file only, expiring 2027-02-10; MEDIUM and LOW findings still appear in the report. ROCMAI-922 (installing vLLM from this lock) must not ship these builds unpatched.Why one PR. The pieces depend on each other: the weekly workflow runs the lock tooling, the tooling runs through the vendored mise launchers, and the Lemonade constants are generated from the same
mise.lock, so landing any one alone would leave it untested or unused. The next steps are separate: keeping the mise launchers current (#1028), installing vLLM from its lock (ROCMAI-922), and makingruntime-lockscheap enough to require once AMD's indexes publish hash links (ROCm/TheRock#9733).No Gherkin scenario: nothing a CLI user can observe changes (the Lemonade version and digest keep their values); the Python locks have no consumer.
Scope
No installer reads the Python locks yet; moving the SDK, vLLM, ComfyUI and llama.cpp installs onto them are follow-ups listed in the design doc. vLLM's version table in
engines/vllmis unchanged. The weekly workflow needs "Allow GitHub Actions to create and approve pull requests" enabled, and its PRs, opened withGITHUB_TOKEN, need a close/reopen to run CI. The workflow recognises its own commits on the update branch by the bot's login, or by the bot's commit email when GitHub leaves the author unresolved; a maintainer's fixup stops the run instead of being reset away. The reset is GraphQLupdateRefswith the checked head asbeforeOid, so a fixup pushed between the check and the reset fails the run instead of being lost; this, like the rest of the workflow, is unverified until the post-merge dispatch.engines/vllm/src/discover_builds.rs, which the vLLM drift test reads as text, now forces a full workspace run incargo xtask affected.tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. (Not a bug fix.)docs/architecture.md. (xtask/src/deps/.)deps check's remediation:relocking_clears_a_stale_lock_but_not_a_policy_problem.)