Repository navigation
perf(ext/fetch): cache lowercased header names per Headers instance - #33683
Conversation
Every `appendHeader` / `Headers.has` / `Headers.set` / `Headers.delete`
inner loop calls `byteLowerCase(list[i][0])` for every existing
entry. `byteLowerCase` is `String.prototype.toLowerCase`, which
allocates a fresh string on each call. For a 7-header Headers
instance that's 7 toLowerCase allocations per `.has()` / `.set()`
call, plus the O(n²) scan in `appendHeader` during construction:
0+1+2+...+(N-1) redundant lowercasings on top of the per-name input
lowercasing.
Add a per-instance `_lowerNames` parallel array that holds the
byte-lowercased form of each `_headerList` entry. Methods that need
to compare entry names compare against `_lowerNames[i]` directly.
The cache is rebuilt from scratch when its length diverges from
`_headerList.length` -- that catches the only external mutation
pattern in the codebase, where `initializeAResponse` and
`Request`'s splice/refill block push directly to `_headerList`
without going through `appendHeader` / `set` / `delete`. (Both of
those mutations happen at construction time, before any read, so
the rebuild cost is paid once rather than on every method call.)
Numbers (cargo bench -p deno_fetch --bench headers_methods, release,
mean of 3 baseline / 4 after):
bench_has_hit 456 ns -> 377 ns -17.3%
bench_has_miss 568 ns -> 393 ns -30.8%
bench_set_replace 789 ns -> 638 ns -19.1%
bench_construct_7 3,586 ns -> 3,576 ns noise
Construct is dominated by webidl-converter / regex-validation
overhead per call; the toLowerCase savings there are ~500 ns out of
~3.5 ms (within the per-iter variance band). The wins are in the
post-construction methods that middleware code actually exercises:
`request.headers.has("authorization")`,
`response.headers.set("Cache-Control", "...")`, etc.
Memory: per `.has()` / `.set()` call, allocations drop from N+1
(N entry-lowercasings + 1 input lowercasing, all freshly allocated
strings) to 1 (just the input). The per-instance `_lowerNames`
array adds a fixed ~N pointers but the strings it holds are the
same ones we'd have allocated lazily on first method call anyway.
Adds `ext/fetch/benches/headers_methods.rs` (and the matching
`[[bench]]` entry plus `deno_bench_util` / `deno_web` /
`deno_webidl` dev-deps in `ext/fetch/Cargo.toml`); pulls the same
harness scaffolding as the other open ext/fetch perf PRs.
Leaves `getHeader` / `getDecodeSplitHeader` (which take a list
rather than a Headers instance) and the `[_iterableHeaders]` getter
unchanged so this PR remains scoped to the methods. Those can use
the cache too in a follow-up.
Headers.get was the only mutation/lookup method in this file still delegating to the standalone getHeader(list, name), which bypassed _lowerNames and re-ran byteLowerCase on every entry on each call. Inline the lookup so it uses ensureLowerNames(this) like its siblings, and build the joined value directly instead of allocating an intermediate entries array + ArrayPrototypeJoin -- the dominant single-match case now does one assignment and returns. bench(ext/fetch): add bench_get_hit / bench_get_miss for Headers.get Same fixture as the rest of headers_methods. Confirms the lookup loop now runs against the cache (no per-iter byteLowerCase) and that the miss case avoids the entries-array allocation.
|
Pushed 080730d to extend the cache to the most-called method, Before this commit the standalone Bench harness gains |
fibibot
left a comment
There was a problem hiding this comment.
Substance is right — the cache eliminates the per-method byteLowerCase storm, and walking the diff carefully:
appendHeader: existing-casing preserved vianame = list[i][0], then_headerListand_lowerNamespushed in lockstep. ✓delete: matches removed via pairedArrayPrototypeSpliceon both arrays +i--. ✓set: first match overwrites value (key/cache untouched, both correct), subsequent matches paired-spliced, no-match push pairs. ✓has/getSetCookie: read-only against cache, length-bounded. ✓- Inlined
get: spec-equivalent togetHeader(0 → null, 1 → direct value, ≥2 → joined with,), and skips the entries-array +ArrayPrototypeJoinfor the dominant single-match case. Verified againstgetHeaderat 20_headers.js:164–178. ✓
Cache invalidation works, but the invariant is fragile
The strategy is "rebuild iff _lowerNames.length !== _headerList.length." Walked the two known direct-mutation paths the PR description calls out:
23_request.js:412–421—ArrayPrototypeSplice(headerList, 0, headerList.length)happens immediately afterheadersFromHeaderList(...)constructs the Headers (line 408), before any read._lowerNamesis still its initialnull, so the nextensureLowerNamescall rebuilds from scratch via thelower === nullbranch — length divergence isn't even needed here.23_response.js:223–232(initializeAResponse) —ArrayPrototypePush(list, ["Content-Type", contentType])can happen afterfillHeaderspopulated the cache (line 207). This case relies on length divergence, and it works (_headerList.length = N+1,_lowerNames.length = N→ rebuild).
So the existing codebase is fine. But the invariant only catches mutations that change _headerList.length. Patterns that wouldn't trip it:
- In-place name replacement:
list[i] = ["new-name", val]orlist[i][0] = "new-name"— cache stays at the old lowercase. - A pop+push pair (length net-zero) — same.
- Any future code that sneaks a direct mutation through
headerListFromHeaders()and preserves length.
None of those exist today, but the lone safeguard is a length comparison with no debug-mode assertion, no per-name lowercase comparison fallback, and no comment in headerListFromHeaders warning callers that mutating returned list while preserving length will desync the cache. Suggestions:
- Add a
// IMPORTANT: if you mutate the returned list in place, do so via splice/push so length changes — or call invalidateLowerNames(headers).doc comment onheaderListFromHeaders. - Export an
invalidateLowerNames(headers)helper (sets[_lowerNames] = null) so future callers have a clean way to do equal-length mutations safely. - Optionally in
cfg!(debug_assertions)-equivalent (e.g. when an env-var tracing flag is set), validatelower[i] === byteLowerCase(list[i][0])for allion read.
The current PR is correct for now; this is hardening for the next PR that touches the direct mutation paths.
bench_get_* "before" column
PR description is upfront: "the 'before' column for those rows is the cache-bypass baseline." Worth either dropping those rows from the table or labeling them clearly so a future bisector doesn't read those Δs as the same kind of "this PR alone" delta as the other rows. Reads more honestly as "post-inline vs. pre-inline within this PR" — not a regression hazard, just a transparency point.
Cargo.toml dev-deps
Adding deno_bench_util / deno_web / deno_webidl to [dev-dependencies] purely for one bench file isn't free — it pulls those crates into the cargo test graph for ext/fetch. If the same harness is showing up in other ext/fetch perf PRs (per the author's note), I agree with the suggestion to extract it into its own prep PR. Less of a footprint to revert if a benchmark approach changes, and lighter cargo test -p deno_fetch invocations.
Memory neutrality claim
PR description says net memory is neutral-to-better. Worth a thought: for an all-lowercase-header instance (the common case for incoming HTTP responses), V8 should intern _headerList[i][0] and _lowerNames[i] to the same string object → effectively zero overhead. For mixed-case instances (user-set headers like "Content-Type"), the cache adds one extra Vec<usize>-equivalent of pointers + ~N short string objects. Neutral overall under typical workloads. ✓
CI
Only license/cla and lint title have run. Holding to COMMENT until the full fetch/headers + WPT shards land green per the CI-must-complete rule. Substance is straightforward; happy to flip to APPROVE once CI and the cache-invalidation guidance question are addressed.
Per @fibibot review on PR denoland#33683 — add hardening for the next caller that touches the direct-mutation paths used by `initializeAResponse` and the `Request` constructor splice/refill block. The existing length-divergence rebuild in `ensureLowerNames` catches push/splice mutations, which is what the current codebase does. But in-place name replacement (`list[i] = [...]`, `list[i][0] = ...`) or pop+push pairs preserve length and would silently keep stale lowercase entries. Two changes, both non-functional: 1. Document the invariant on `headerListFromHeaders` so future callers reaching for direct mutation see the cache contract up front. 2. Export `invalidateLowerNames(headers)` so callers doing equal-length mutations have a clean way to opt back into rebuild on the next read, without exposing the symbol.
|
@fibibot addressed the cache-invariant hardening — pushed a8da9d8:
Both are non-functional — no current call site needs the helper, but the next one to reach for direct mutation now has the explicit contract. The other suggestions are noted but not actioned on this PR:
|
fibibot
left a comment
There was a problem hiding this comment.
LGTM. The latest commit a8da9d8 is exactly the doc-the-invariant follow-up I flagged in my prior review:
headerListFromHeadersJSDoc now spells out the safe vs unsafe mutation patterns:push/splicethat change length self-heal via the length-divergence rebuild; in-placelist[i] = [...],list[i][0] = ..., and pop+push pairs would silently keep stale lowercase entries.invalidateLowerNames(headers)is exported as an escape hatch for length-preserving in-place mutations. Future callers that need to do that have a documented signal to invalidate.
No runtime behavior change in this commit — just documentation + helper export. Substance of the cache and method rewrites unchanged from my prior walk:
appendHeader/delete/set/has/getSetCookie/ inlinedgetall maintain_headerListand_lowerNamesin lockstep.- Cache rebuild trigger (
_lowerNames === nullOR length divergence) covers both known direct-mutation callers (23_request.js:412splice/refill —_lowerNamesis null at that point so rebuild via thenullbranch;23_response.js:223push-Content-Type — length diverges, rebuild via length branch). - Inlined
getis spec-equivalent to the standalonegetHeader(0 → null, 1 → direct value, ≥2 → joined with,) and skips the entries-array +ArrayPrototypeJoinallocation in the dominant single-match case.
CI
Fully green at this head: 135/135 success, 1 skip (publish canary). All test unit / test unit_node / test node_compat / test specs / wpt shards pass — the Headers polyfill is exercised across all four (every fetch-touching node compat test, plus WPT XHR/fetch coverage).
|
Reviewed the cache-busting strategy — looks solid for all current mutation patterns:
One thing worth noting: the desync detection is length-based only, so an in-place name mutation like Minor: |
|
@crowlbot please resolve conflicts and try to address this bit:
|
…header-names # Conflicts: # ext/fetch/Cargo.toml
bartlomieju
left a comment
There was a problem hiding this comment.
Looks good — traced the cache invariant across appendHeader/set/delete and the two direct _headerList mutators (initializeAResponse content-type push, Request ctor splice/refill); length-divergence rebuild covers all of them. CI green.
Two small things worth flagging inline.
Reconciles with main's switch from splice-on-match to write-pointer compaction in Headers#delete and Headers#set. The merged version keeps the cached `_lowerNames` reads (no per-iteration `byteLowerCase`) and adopts the write-pointer rewrite, updating `list` and `lowerNames` in lockstep.
Addresses review feedback (bartlomieju): the response init's `content-type` presence check still walked the header list with raw `byteLowerCase(list[i][0])` on each iteration. Switches it to read the parallel `_lowerNames` cache via the new exported `ensureLowerNames` helper and pushes to both arrays in lockstep when appending the default `Content-Type`, so the cache no longer relies on length-divergence rebuild for this caller. `byteLowerCase` import is no longer needed in `23_response.js`.
Deno Individual Contributor License AgreementAll contributors have signed the CLA. Thank you! This is an automated message from CLA Assistant |
|
@bartlomieju addressed both:
The docstring on |
Address review notes from denoland#33683: - Replace `--` with a comma in the JSDoc on `ensureLowerNames` to match the prose style elsewhere in ext/fetch. - Drop the `invalidateLowerNames(headers)` helper and its export. It had no callers; this PR doesn't add one. The warning JSDoc on `headerListFromHeaders` keeps the cache contract documented: callers must either change the list length or add a cache-invalidator alongside the first equal-length in-place mutation site.
|
|
|
Both bits addressed in c73beff:
Rebased on main, MERGEABLE now. |
|
Rebased on latest Re-verified locally on the merged tree:
Resolved the two outstanding inline threads (double-dash style + |
|
Rebenched on current main + this PR (deno 2.8.1, V8 14.9.207.2-rusty, Linux x86_64). Cross-runtime microbench of the six representative Headers method shapes, same-host. 5 runs of 1M iters (200k for construct), 50k-iter warmup, median (range).
Honest summary:
Patch shape: ~100 LoC in Cross-runtime takeaway: this PR closes Deno's gap to Node on CI is green on the latest commit; ready for re-review. |
…enoland#33683) ## Summary Every `appendHeader` / `Headers.has` / `Headers.get` / `Headers.set` / `Headers.delete` / `getSetCookie` inner loop calls `byteLowerCase(list[i][0])` for every existing entry. `byteLowerCase` is just `String.prototype.toLowerCase`, which allocates a fresh string on every call. For a 7-header `Headers` instance that's 7 `toLowerCase` allocations per `.has()` / `.get()` / `.set()` call, plus an `O(n²)` scan in `appendHeader` during construction (0+1+2+...+(N-1) redundant lowercasings on top of the per-name input lowercasing). This PR adds a per-instance `_lowerNames` parallel array holding the byte-lowercased form of each `_headerList` entry. Methods that need to compare entry names compare against `_lowerNames[i]` directly. The cache is rebuilt from scratch when its length diverges from `_headerList.length` — that catches the only external mutation pattern in the codebase, where `initializeAResponse` and `Request`'s splice/refill block push directly to `_headerList` without going through `appendHeader` / `set` / `delete`. (Both of those mutations happen at construction time before any read, so the rebuild cost is paid once, not on every method call.) `Headers.get` is also inlined here (second commit) so it can use the same cache instead of delegating to the standalone `getHeader(list, name)` — and so the dominant single-match case skips the intermediate entries-array allocation and the trailing `ArrayPrototypeJoin`. ## Numbers Same-host microbench of the six representative Headers method shapes. `deno 2.8.1`, `V8 14.9.207.2-rusty`, Linux x86_64. 5 runs of 1M iters (200k for `construct`), 50k-iter warmup, median (range). Baseline column is current `origin/main` without this PR; "this PR" is current `origin/main` with this PR applied — both built with `cargo build --release --bin deno`. | op | baseline (origin/main) | this PR | Δ | Node 22.22.2 | Bun 1.3.14 | |---|---:|---:|---:|---:|---:| | `.has(hit)` (4th-of-7) | 249.3 (241–250) | 258.1 (239–261) | ~1.0× (noise) | 220.7 | 62.4 | | `.has(miss)` | 475.9 (451–490) | **140.0 (137–141)** | **3.4× faster** | 238.1 | 110.5 | | `.get(hit)` (4th-of-7) | 543.6 (526–555) | **144.1 (134–154)** | **3.8× faster** | 235.8 | 81.4 | | `.get(miss)` | 455.4 (451–460) | **249.0 (233–249)** | **1.8× faster** | 242.1 | 120.0 | | `.set(replace)` | 809.8 (782–813) | **603.6 (578–625)** | **1.3× faster** | 513.7 | 159.4 | | `new Headers({...7})` | 5884.8 (5819–6233) | 5927.8 (5536–6357) | ~1.0× (noise) | 3990.7 | 1981.9 | The wins land on the post-construction methods that middleware code actually exercises (`request.headers.get("authorization")`, `response.headers.set("Cache-Control", "...")`, etc.). The four shapes that benefit cover the dominant Deno.serve hot path. `.has(hit)` is unchanged because the existing un-cached code already returns on the first match — the cache has nothing to optimize. Listed for completeness. `new Headers({...})` is dominated by webidl-converter / regex-validation per call; the `toLowerCase` savings there are within per-iter variance. ### Cross-runtime takeaway - `.get(hit)`: 3.8× speedup brings Deno to **144 ns vs Node 22's 236 ns** — Deno is now faster than Node on the dominant per-request Headers read. Bun (81 ns) keeps ~1.8× lead. - `.has(miss)`: 3.4× speedup brings Deno to **140 ns vs Node's 238 ns** — Deno goes from ~2× *slower* than Node to **~1.7× faster**. - `.get(miss)` / `.set(replace)`: solid wins; Deno reaches parity / near-parity with Node. `ext/fetch/benches/headers_methods.rs` ships with the PR so the Rust-side `cargo bench -p deno_fetch --bench headers_methods` keeps tracking the same shapes for CI's `bench release linux-x86_64`. ## Memory Per `.has()` / `.get()` / `.set()` call, allocations drop from N+1 (N entry-lowercasings + 1 input lowercasing, each a fresh string) to 1 (just the input). The per-instance `_lowerNames` array adds a fixed ~N pointers but holds the same strings we'd have allocated lazily on first method call anyway, so net memory is neutral-to-better. The `Headers.get` inline change additionally drops the intermediate entries array + Join string for the single-match case (the dominant case). ## Scope Leaves `getHeader` / `getDecodeSplitHeader` (which take a `list` rather than a `Headers` instance, used by `inner.headerList` callers in `23_request.js` / `23_response.js`) and the `[_iterableHeaders]` getter unchanged. Those can also use a cache in a follow-up but require threading the `Headers` object through. ## Test plan - [x] Same-host microbench across deno-with-PR / deno-without-PR / Node 22 / Bun 1.3 (numbers above) - [x] `cargo bench -p deno_fetch --bench headers_methods` (multiple runs before / after, both commits) - [ ] CI: existing fetch / headers unit + spec tests (cover append, set, delete, has, get, getSetCookie, iteration order, duplicate-name combine, set-cookie split, immutable behaviour) - [ ] CI: WPT `fetch/api/headers` suite `ext/fetch` had no bench harness before this set of PRs; `ext/fetch/benches/headers_methods.rs` is new (plus the matching `[[bench]]` entry and `deno_bench_util` / `deno_web` / `deno_webidl` dev-deps).
Summary
Every
appendHeader/Headers.has/Headers.get/Headers.set/Headers.delete/getSetCookieinner loop callsbyteLowerCase(list[i][0])for every existing entry.byteLowerCaseis justString.prototype.toLowerCase, which allocates a fresh string on every call. For a 7-headerHeadersinstance that's 7toLowerCaseallocations per.has()/.get()/.set()call, plus anO(n²)scan inappendHeaderduring construction (0+1+2+...+(N-1) redundant lowercasings on top of the per-name input lowercasing).This PR adds a per-instance
_lowerNamesparallel array holding the byte-lowercased form of each_headerListentry. Methods that need to compare entry names compare against_lowerNames[i]directly.The cache is rebuilt from scratch when its length diverges from
_headerList.length— that catches the only external mutation pattern in the codebase, whereinitializeAResponseandRequest's splice/refill block push directly to_headerListwithout going throughappendHeader/set/delete. (Both of those mutations happen at construction time before any read, so the rebuild cost is paid once, not on every method call.)Headers.getis also inlined here (second commit) so it can use the same cache instead of delegating to the standalonegetHeader(list, name)— and so the dominant single-match case skips the intermediate entries-array allocation and the trailingArrayPrototypeJoin.Numbers
Same-host microbench of the six representative Headers method shapes.
deno 2.8.1,V8 14.9.207.2-rusty, Linux x86_64. 5 runs of 1M iters (200k forconstruct), 50k-iter warmup, median (range). Baseline column is currentorigin/mainwithout this PR; "this PR" is currentorigin/mainwith this PR applied — both built withcargo build --release --bin deno..has(hit)(4th-of-7).has(miss).get(hit)(4th-of-7).get(miss).set(replace)new Headers({...7})The wins land on the post-construction methods that middleware code actually exercises (
request.headers.get("authorization"),response.headers.set("Cache-Control", "..."), etc.). The four shapes that benefit cover the dominant Deno.serve hot path..has(hit)is unchanged because the existing un-cached code already returns on the first match — the cache has nothing to optimize. Listed for completeness.new Headers({...})is dominated by webidl-converter / regex-validation per call; thetoLowerCasesavings there are within per-iter variance.Cross-runtime takeaway
.get(hit): 3.8× speedup brings Deno to 144 ns vs Node 22's 236 ns — Deno is now faster than Node on the dominant per-request Headers read. Bun (81 ns) keeps ~1.8× lead..has(miss): 3.4× speedup brings Deno to 140 ns vs Node's 238 ns — Deno goes from ~2× slower than Node to ~1.7× faster..get(miss)/.set(replace): solid wins; Deno reaches parity / near-parity with Node.ext/fetch/benches/headers_methods.rsships with the PR so the Rust-sidecargo bench -p deno_fetch --bench headers_methodskeeps tracking the same shapes for CI'sbench release linux-x86_64.Memory
Per
.has()/.get()/.set()call, allocations drop from N+1 (N entry-lowercasings + 1 input lowercasing, each a fresh string) to 1 (just the input). The per-instance_lowerNamesarray adds a fixed ~N pointers but holds the same strings we'd have allocated lazily on first method call anyway, so net memory is neutral-to-better. TheHeaders.getinline change additionally drops the intermediate entries array + Join string for the single-match case (the dominant case).Scope
Leaves
getHeader/getDecodeSplitHeader(which take alistrather than aHeadersinstance, used byinner.headerListcallers in23_request.js/23_response.js) and the[_iterableHeaders]getter unchanged. Those can also use a cache in a follow-up but require threading theHeadersobject through.Test plan
cargo bench -p deno_fetch --bench headers_methods(multiple runs before / after, both commits)fetch/api/headerssuiteext/fetchhad no bench harness before this set of PRs;ext/fetch/benches/headers_methods.rsis new (plus the matching[[bench]]entry anddeno_bench_util/deno_web/deno_webidldev-deps).