Skip to content

fix(sdk): retry the zk verifier when it answers 503 busy - #490

Open
Bruce039 wants to merge 2 commits into
FhenixProtocol:masterfrom
Bruce039:fix/verify-batch-retry-503
Open

Bruce039 wants to merge 2 commits into
FhenixProtocol:masterfrom
Bruce039:fix/verify-batch-retry-503

Conversation

@Bruce039

@Bruce039 Bruce039 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

When the zk verifier is at its in-flight limit it answers /verifyBatch with 503, Retry-After: 1 and "Server busy, retry shortly" (the load-shed path in zee-k-verifier's api/mod.rs). That response is sent before the request is read, so nothing was verified or stored and the same proof can simply be sent again. zkVerifyBatch treats it like any other HTTP error, though, so one busy response fails the whole encryptInputs with ZK_VERIFY_FAILED and the proof that was just generated, which is the slow part of an encrypt, is thrown away. The app then has to start over from packing.

zkVerifyBatch now sends the same body again on a 503, up to three times, waiting for the Retry-After seconds between attempts. If the header is missing it waits one second, and a long value is capped at five seconds so an encrypt can't hang on it. If the verifier is still busy after the last retry the error is the same ZK_VERIFY_FAILED as before. Other statuses are not retried, and no option or export was added.

zkVerifyBatchBusy.test.ts covers a busy-then-success response (same body sent twice), the missing and capped Retry-After delays, giving up after three retries, and a 400 that still fails on the first call. The first three fail on master. The sdk node project passes (611 tests); the four files that need @cofhe/test-setup built were skipped, same as on master. Prettier is clean and tsc --noEmit shows only the existing @cofhe/test-setup import errors.


Note

Low Risk
Scoped HTTP retry logic for 503 only on the verifier call; bounded delays and unchanged error paths for real verification failures.

Overview
encryptInputs no longer fails the whole encrypt on a single verifier 503 (busy). zkVerifyBatch now re-POSTs the same proof to /verifyBatch up to three times, sleeping per Retry-After (defaults to 1s, capped at 5s) before each retry.

If the verifier is still busy after those attempts, behavior is unchanged: ZK_VERIFY_FAILED. Non-503 errors (e.g. 400) are not retried. A patch changeset documents the fix for @cofhe/sdk.

zkVerifyBatchBusy.test.ts adds coverage for busy-then-success, delay defaults/caps, exhaustion after three retries, and no retry on other statuses.

Reviewed by Cursor Bugbot for commit bba9d03. Bugbot is set up for automated code reviews on this repo. Configure here.

The verifier sheds load with 503 and Retry-After: 1 before it reads the
request, so the same proof can be sent again. Retry up to three times,
waiting for Retry-After (1s when missing, capped at 5s).
@changeset-bot

changeset-bot Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 8900e58

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 10 packages
Name Type
@cofhe/sdk Patch
@cofhe/abi Patch
@cofhe/hardhat-3-plugin Patch
@cofhe/hardhat-plugin Patch
@cofhe/mock-contracts Patch
@cofhe/react Patch
@cofhe/site Patch
@cofhe/example-react Patch
@cofhe/test-anvil Patch
@cofhe/foundry-plugin Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@alexshchur alexshchur added the run-matrix Run the matrix and the live testnet suite on this PR with the keys. Add after reading the diff. label Nov 18, 2026
@github-actions github-actions Bot removed the run-matrix Run the matrix and the live testnet suite on this PR with the keys. Add after reading the diff. label Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants