Skip to content

fix(ci): prevent poisoned Rust caches - #6618

Open
wesbillman wants to merge 1 commit into
mainfrom
carl/fix-sherpa-unit-cache
Open

fix(ci): prevent poisoned Rust caches#6618
wesbillman wants to merge 1 commit into
mainfrom
carl/fix-sherpa-unit-cache

Conversation

@wesbillman

Copy link
Copy Markdown
Collaborator

Summary

  • roll every Swatinem/rust-cache use back from v2.9.2 to the last known-good v2.9.1 digest
  • give Unit Tests a new sherpa-cache-v1 key so it cannot restore the existing poisoned artifact
  • pin Renovate to v2.9.1 and add a CI contract that rejects the bad digest, a missing generation key, or an automatic re-upgrade

Why

After #5441 upgraded rust-cache to v2.9.2, all five relevant main Unit Tests runs that restored the same 1.4 KB cache failed linking sherpa-onnx-c-api; all eight cold-cache runs passed. v2.9.2 changed target cleanup, while sherpa-onnx-sys treats its prebuilt lib/ directory as proof that the native archive exists. Rolling back the action and invalidating the affected key removes both sides of that failure state without disabling target caching.

Validation

  • scripts/test-rust-cache-contract.sh
  • negative contract fixtures reject the v2.9.2 digest and Renovate <=2.9.2
  • YAML parse for all changed workflows
  • scripts/test-release-ref-contract.sh
  • just file-size-check
  • pre-commit and pre-push hooks passed at cb4ca303dad8e63515884803bda8792e84782b5f

The PR run is the cold-cache proof because pull requests restore but do not save Rust caches. The first successful main run after merge will save the new Unit Tests key; the following main run will exercise the warm restore.

Related issue

None found.

Roll back rust-cache to the last known-good release, invalidate the Unit
Tests cache generation, and keep Renovate from restoring the broken
cleanup behavior until the upstream cache contract is safe for sherpa.

Co-authored-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Carl <32a2e2c9d428ee08902cab75d956da2c1d235a22d4766b0dd4138bf6e2e5db1d@buzz.block.builderlab.xyz>
Signed-off-by: Wes <wesbillman@users.noreply.github.com>
@wesbillman
wesbillman requested a review from a team as a code owner August 23, 2026 14:38

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

Blocking: make the contract discover every workflow. scripts/test-rust-cache-contract.sh hardcodes only ci.yml, mesh-lifecycle.yml, and release.yml. A future workflow can therefore add the known-bad v2.9.2 digest (or any other Swatinem/rust-cache pin) while this required contract still passes. I reproduced that at this head by adding .github/workflows/_rust-cache-contract-bypass.yml with Swatinem/rust-cache@6323deb...; the script printed rust cache contract passed and exited 0.

Please discover all .github/workflows/*.yml/*.yaml files (or otherwise use one authoritative inventory) before enforcing the digest set. The current rollback, Unit Tests generation bump, Renovate cap, and live cold-cache Unit Tests result otherwise look correct.

GitHub does not allow this account to request changes on its own PR, so this is posted as a comment review but remains blocking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant