Let the embedder keep V8 code caches for worker code

[floitsch]

An embedder can now give an isolate a jsg::CodeCacheStore, a process-wide store of V8 code caches for worker bundle modules and service-worker main scripts. Without one, nothing changes.

With a store, both module registries and NonModuleScript::compileWorkerScript() compile through one per-isolate helper, jsg::IsolateCodeCache. It looks the source up in the store, pre-checks a hit with CachedData::CompatibilityCheck(), compiles with kConsumeCodeCache, and reports refused data to the store and to the CompilationObserver hooks. On a miss the store may elect the isolate to produce the entry, so concurrent isolates do not all serialize the same code.

V8 checks a cache only against its version, flags, read-only snapshot and the source length, not the content, so entries are keyed by a SHA-256 of the exact UTF-8 source and the compilation unit (CodeCacheKey).

Worker::Script opens a production window and Worker closes it after the top level has run, so caches hold the functions V8 compiled lazily while running it, not only the eagerly compiled ones (on a Hono bundle: 336 functions instead of 12, first response 8.5 ms instead of 11.1 ms in d8). Code first compiled during a request may consume an existing entry but never produces or replaces one. Import modules during startup to populate their caches. Modules whose evaluation failed and scripts whose startup failed are not cached. A Script no Worker runs closes the window on destruction. Pending entries hold v8::Globals, released in IsolateBase::dropWrappers().

Source hashes and initial store lookups are prepared before startup CPU accounting. Identical modules share a prepared lookup and transfer the producer election only once. V8 deserialization remains CPU-counted. New-registry builders opt into CODE_CACHE_KEYS; builtins and isolates without a store keep their per-module cache slot.

Tests cover both registries through Worker construction (including lookup timing and the accepted-cache hook), module and classic Worker lifecycle, prepared lookups (including duplicate and unused sources), hits versus rejection, startup-only production, request-time rejection without replacement, and store/window invariants.

Verification and landing

Rebased onto workerd main 587bab1c2; current head 75990bad5. Conflict resolution retains main's replacement I/O task test and JSG helper table. The companion !15320 pins this exact head. Formatting and whitespace checks pass. Fresh full upstream and downstream CI are running.

Earlier full CI passed all upstream platforms, coverage and clang-tidy. Strict focused CI passed 160 first-attempt result events, with explicit completion for every Worker cache case: combined verification, targeted Windows cases. Diagnostic workflow and benchmark commits are excluded from this PR.

Erik's addressed comments have replies and are resolved. Hashing and initial lookups run before startup CPU accounting; V8 deserialization and rejection housekeeping remain counted. The process-local store's cross-account sharing and accepted whole-module timing existence oracle are documented in the companion embedder header. Formal human approval remains a landing requirement.

Merge this PR first, then update the companion runtime pin if the merge changes the SHA.

Direct eager-compilation comparison

CI benchmark, five interleaved rounds of the same 815,423-byte Hono/Zod bundle, fresh isolates, V8 15.6.75. Default CI build with V8 compiled at -O3; these are wall times. Values are medians. This uses kEagerCompile directly and is not directly comparable to the older d8/edgeworker V8 15.5 measurements on a different host.

Production	Blob bytes	Blob/source	Serialize	Deserialize	Consumer load + top
After compile	267,240	0.33	2.183 ms	0.816 ms	7.311 ms
After top-level	315,432	0.39	2.508 ms	0.912 ms	4.542 ms
Explicit eager	997,288	1.22	6.266 ms	2.396 ms	6.349 ms

The eager blob is 3.16 times the after-top-level blob, and its deserialization cost is 2.63 times as large. The startup-only after-top-level policy remains unchanged. The benchmark commit is isolated from this PR/MR.

[ask-bonk]

Since last review: 0 resolved, 0 still open, 0 new.
LGTM!

Not re-run: docs, design-simplicity (no author changes in their files since the last review)

---

_{Reviewed commit: 04aeb430 · github run}

[erikcorry]

Review of the store/consume/produce design, with the edgeworker MR (!15320) read alongside. Nothing blocking; inline comments cover the specifics. Summary:

• Startup limit: the SHA-256 and store lookup run inside enterStartupJs on both hit and miss, which the validator (no store) never pays. Deterministic and small, but it is the "passes upload, exceeds the limit in prod" direction. Move the key computation/lookup outside the limit scope.
• Request-time compiles (module first imported during a request) are serialized synchronously in the request on the elected isolate, and that cache is the weak eager-functions-only kind.
• onCompileCacheFound and onCompileCacheRejected both fire for one compile when V8 refuses the data during deserialization.
• The production window is per isolate but opened/closed per Script/Worker; pin the one-Script-per-isolate assumption with an assert.
• Small items: bool parameter, nullptr as ArrayPtr, stale cachedData comment, jsg/AGENTS.md KEY FILES row for code-cache.h, docs/reference/detail/new-module-registry.md compile-cache sections still describe the per-module slot as the only path.

Measurements that would help judge the limits above and maxEntryBytes: blob size and deserialize time for the Hono bundle (and blob/source ratio); CreateCodeCache wall time on the producing isolate; and a kEagerCompile production row as a data point (expected to deserialize more than the validator measured, so not a recommendation, but it would show the bound).

Verified and fine: key completeness (origin fields are reapplied from ScriptDetails, not serialized; flags all set pre-init in V8System; decoders split by Unit); cross-isolate-group validity (RO refs are page index + offset, checksummed; no raw pointers); no-store behaviour unchanged; v8::Global release before Dispose on every failure path; lookup.hit Arc outlives compileSource.

Possible follow-up, not for this PR: a two-level key, rapidhash(source) with a per-process seed as the table key and the SHA-256 stored in the entry and verified only on a hit. Misses then cost ~50 µs/MB instead of ~0.5 ms/MB, hits are unchanged, and the SHA-256 remains the integrity control.

These comments were drafted with an AI assistant and may contain mistakes.

[github-actions]

The generated output of @cloudflare/workers-types matches the snapshot in types/generated-snapshot 🎉

[floitsch]

The recovered follow-up is pushed; current head is 1f70292f8.

• Worker construction prepares source hashes and store lookups before the startup CPU limit, for classic scripts and both registries. Identical sources reuse their prepared lookup.
• Production is startup-only. Request-time imports can consume an existing entry, but never elect a producer or serialize a replacement.
• Found is emitted only after V8 accepts the cache. Added the production-window and store-installation assertions, removed the boolean/null-view ambiguities, and updated the registry documentation and source-integrity/process-sharing comments.
• Added a Worker-level new-registry regression that verifies preparation before the startup limit and consumption by the second Worker. Before the fixture-budget follow-up, focused ASan and coverage passed all 40 first-attempt runs, with five runs per variant and no retries: https://github.com/cloudflare/workerd/actions/runs/38255911384

The direct-API comparison uses an 815,423-byte Hono/Zod bundle, five interleaved rounds, V8 15.6.75, and optimized V8 in the default CI build. After-top-level production gives a 315,432-byte blob (0.39x source), median serialization 2.508 ms, and median deserialization 0.912 ms. Explicit eager production gives 997,288 bytes (1.22x), 6.266 ms and 2.396 ms respectively. The full table and CI link are in the description.

V8 deserialization and rejection housekeeping remain inside startup CPU accounting. The implementation does not claim exact validator/production CPU parity. Startup-only production and preparing hashes/lookups before the limit are the current scope; extra observability is deferred.

Windows CI reproduced nine unsafe-module fixture timeouts near the end of the 16-handler suite. Both affected fixtures use real-time waits and a 30-second eviction deadline, exceeding the default 15-second test budget. 1f70292f8 classifies them as large (60 seconds), preserving all assertions. Fresh strict Windows/ASan/coverage CI runs all variants of both fixtures and the cache tests five times without retries (150 runs total). Full CI is also running; original failure logs are retained.

@erikcorry, please re-review the current head, including the remaining deserialization accounting and sharing documentation.

[jasnell]

Nice to see the more robust approach.

[clanky-the-clanker]

💬 Approved with comments — No blocking or production-impact problems found. Two documentation WARNINGs (store use on the new registry depends on CODE_CACHE_KEYS, and the legacy-registry reference is out of date) and a few INFO notes on edge cases in error and rejection handling.

Findings: 2 warnings · 5 info

_{The decision is advisory: this comment never approves, blocks or requests changes on the pull request.}

🔧 Fix with your agent

Paste the following into your coding agent to investigate and fix these issues.

Fix the following code review issues found in pull request #7664.

Before making changes, review each issue and present a brief summary table:
- For each finding, state whether you agree, disagree, or need clarification
- If you disagree (e.g., the fix would require disproportionate changes
  for minimal ROI, or the finding is factually incorrect), explain why
- If you need to clarify something before deciding, ask those questions
- Then share your plan for which issues to tackle and in what order

## WARNING (2)

### [Documentation] The new-registry doc says bundle modules use the store once a `CodeCacheStore` is set, but the new registry needs `CODE_CACHE_KEYS` for that; add the qualifier.
- **File:** docs/reference/detail/new-module-registry.md:47

### [Documentation] `legacy-module-registry.md:380-381` still describes bundle ESM compilation as passing `compileCache` to `ModuleInfo`; describe the code-cache overload that the bundle path now uses.

## INFO (5)

### [Code Quality] If the store's `Producer::insert()` throws during `produceDeferred()`, the error leaves the `Worker` constructor and worker startup fails; catch per entry or document that `insert()` must not throw.
- **File:** src/workerd/jsg/code-cache.c++:162

### [Code Quality] After V8 rejects a prepared hit, the hit is still served to later modules with identical source, so `store->reject()` and `onCompileCacheRejected` fire more than once.
- **File:** src/workerd/jsg/code-cache.c++:67

### [Code Quality] `KJ_ON_SCOPE_FAILURE` is registered before `deferProduction()`, so if its assertion fires, the handler discards another Script's open window. No current caller can trigger this.
- **File:** src/workerd/io/worker.c++:1584

### [Code Quality] The unrelated `size = "large"` test BUILD changes are bundled into this pull request; split them out or explain them.
- **File:** src/workerd/api/tests/BUILD.bazel:247

### [Documentation] The `prepareCodeCaches()` comment says it hashes worker code; on the new-registry path it uses keys the registry already computed.
- **File:** src/workerd/io/worker.c++:1510

---

📋 Code Quality: 💬 4 info

The cache-window lifecycle, the cache keys and both registry compile paths look correct and are well tested. Four low-impact edge cases.

Recommendation: 💬 Optional Nits

• [INFO] If the store's Producer::insert() throws during produceDeferred(), the error leaves the Worker constructor and worker startup fails; catch per entry or document that insert() must not throw. (src/workerd/jsg/code-cache.c++:162)
• [INFO] After V8 rejects a prepared hit, the hit is still served to later modules with identical source, so store->reject() and onCompileCacheRejected fire more than once. (src/workerd/jsg/code-cache.c++:67)
• [INFO] KJ_ON_SCOPE_FAILURE is registered before deferProduction(), so if its assertion fires, the handler discards another Script's open window. No current caller can trigger this. (src/workerd/io/worker.c++:1584)
• [INFO] The unrelated size = "large" test BUILD changes are bundled into this pull request; split them out or explain them. (src/workerd/api/tests/BUILD.bazel:247)

View full review

The production-window lifecycle, the SHA-256 source keys, the address-keyed prepared lookups, and the legacy and new registry compile paths all look correct, and the tests cover them well. Four low-impact notes:

A store failure can fail worker startup (src/workerd/io/worker.c++:2532-2538, src/workerd/jsg/code-cache.c++:162-173, 207)

• produceDeferred() runs after the top level has succeeded, outside the startup try/catch, and calls the embedder's Producer::insert() directly.
• If insert() throws, the exception leaves the Worker constructor, and KJ_ON_SCOPE_FAILURE tears the Worker down. A cache-store problem then becomes a worker startup failure.
• Fix: catch exceptions per entry in produceDeferred() (call generationFailed() and log), or state in the CodeCacheStore contract that insert() must not throw.

A rejected prepared hit is served again (src/workerd/jsg/code-cache.c++:67-77, 120-126)

• takeLookup() keeps a prepared hit and hands an addRef() of it to every module with the same source.
• When V8 rejects the bytes, the hit stays in place. The next identical module repeats the rejection, calls store->reject() again and fires onCompileCacheRejected again.
• This needs both duplicate module sources and bad cache data, and it only inflates the metric and the number of reject() calls. Clearing the hit after a rejection avoids it.

deferProduction() assertion versus the scope-failure handler (src/workerd/io/worker.c++:1584-1599)

• KJ_ON_SCOPE_FAILURE is registered before deferProduction(). If the "already open" assertion fires because another Script's window is open, the handler discards that other Script's window.
• Current callers build a Script and its Worker back to back, so this cannot happen today. Opening the window before registering the handler would keep a failure local if that ever changes.

Unrelated BUILD changes (src/workerd/api/tests/BUILD.bazel:247-248, src/workerd/server/tests/unsafe-module/BUILD.bazel:3-4)

• The size = "large" changes for actor-alarms-name-test and unsafe-module-test have nothing to do with code caching. Consider a separate commit, or explain in the description why this change needs them.

---

📝 Documentation: ⚠️ 2 warnings · 1 info

The new-registry docs overstate when the store is used, and the legacy-registry reference does not describe the new compile path.

Recommendation: 📝 Minor Comments

• [WARNING] The new-registry doc says bundle modules use the store once a CodeCacheStore is set, but the new registry needs CODE_CACHE_KEYS for that; add the qualifier. (docs/reference/detail/new-module-registry.md:47)
• [WARNING] legacy-module-registry.md:380-381 still describes bundle ESM compilation as passing compileCache to ModuleInfo; describe the code-cache overload that the bundle path now uses.
• [INFO] The prepareCodeCaches() comment says it hashes worker code; on the new-registry path it uses keys the registry already computed. (src/workerd/io/worker.c++:1510)

View full review

• docs/reference/detail/new-module-registry.md:47-50 says bundle modules use the store whenever the embedder gives isolates a CodeCacheStore. On the new registry this only holds for modules that have a codeCacheKey, which is set only when the registry is built with CODE_CACHE_KEYS. Say so, so embedders do not expect setting a store alone to turn on caching.
• docs/reference/detail/legacy-module-registry.md:380-381 (not changed by this pull request) still says worker bundle ESM is compiled through ModuleInfo(js, name, content, compileCache, BUNDLE, observer). The bundle path now uses the overload that checks the isolate code cache when there is one (src/workerd/io/worker-modules.h:371-374, src/workerd/jsg/modules.c++:467-479). Update it.
• src/workerd/io/worker.c++:1510-1512: the comment says prepareCodeCaches() hashes the worker code. The new-registry path uses keys the registry has already computed, so only the classic-script and legacy paths hash here.

---

✅ All other checks passed

🔒 Security: ✅ LGTM

No exploitable issues found in code-cache.{h,c++} or the worker.c++ integration. Cache entries are keyed by a SHA-256 of the exact source plus the compilation unit, and the companion embedder header documents that the process-wide store is shared across accounts. The security check did not cover the module-registry and script.c++ integration in detail.

---

📖 AGENTS.md: ✅ LGTM

The src/workerd/jsg/AGENTS.md changes match the new code-cache helper and the updated helper table. No issues.

---

⚡ Performance: ✅ LGTM

No performance regressions found.

• Isolates without a store skip the new paths and keep their per-module cache slot.
• Each source is hashed with SHA-256 once, identical sources share one lookup, and both happen before startup CPU accounting.
• Pending v8::Globals, prepared lookups and producer elections are released when the production window closes, so none of them can grow without bound.
• Note: prepareCodeCaches looks up every bundle module, including modules never imported at startup, and holds those producer elections until the window closes. This can briefly delay other isolates from producing those entries; the effect lasts only for startup. The pull request description tells users to import modules during startup so their caches are populated.

---

_{Reviewed commit: 04aeb430}