perf(web): cut the round trips behind slow list-page loads

[Makisuo]

Why

The services list sat on a loading skeleton for ~10s. The DevTools timing was the giveaway — 2.23s waiting for the server, 10.05s in Queueing — so most of the wall time was spent before the request was even sent.

The warehouse is innocent. Bucketing maple-api server spans for /api/query-engine/* by how many requests landed in the same 5s window (Aug 4–11) gives a flat p50: 272ms alone, 403ms at 4–6 concurrent, 212ms at 13+. Client-side RUM (session_events) agrees. Adding concurrency does not slow the server down.

It was three separate client-facing round-trip problems.

1. No Access-Control-Max-Age

Without it Chrome falls back to a 5-second preflight cache. CLOUDFLARE_WORKER_PLACEMENT is { region: "aws:us-east-1" } (deliberate — it pins the worker next to Postgres), so from Europe a round trip measures ~165ms (cf-placement: remote-IAD on a -TXL ray). Every expiry cost a full extra transatlantic round trip before the real request went out.

2. Access-Control-Allow-Headers: * does not cover Authorization

This is the one that made fix #1 worthless on its own. The Fetch spec deliberately excludes Authorization from the * wildcard. Browsers still allow the request — so nothing looks broken — but they refuse to cache its preflight. Since every browser call to this API is authenticated, maxAge was a complete no-op.

Measured in Chrome, 3 identical authenticated GETs to one URL:

allowedHeaders	preflights
["*"]	3 (one per request)
["*", "Authorization"]	1

Control in the same session: a plain x-test custom header cached fine under ["*"] — which is exactly what makes this easy to misdiagnose.

These OPTIONS never reach our traces (HttpMiddleware.withTracerDisabledWhen() skips OPTIONS), which is why the server-side data looked healthy while the browser burned its time on round trips we never recorded.

3. Nothing was batched

There was no batch endpoint — every query was its own POST /execute. And services-table.tsx fired one GET /vcs/commits/:sha per row, eagerly on mount, contradicting the arm-on-hover design documented in commit-sha-hover-card.tsx. Each SHA is a distinct URL, so each also needed its own preflight.

What changed

• api-cors.ts — set maxAge: 86_400 and list Authorization explicitly.
• POST /api/query-engine/execute-batch — warmRoute once for the whole batch (the documented ordering fix), concurrency 4 to match the bucket-cache fill concurrency already tuned for the Cloudflare six-connection limit, and a wall-clock deadline shared by every item so total time is bounded no matter how many waves run. Per-item failures ride in the success payload, so one bad widget can't take down the others sharing its request.
• Client batcher at the single choke point executeQueryEngine, so no call sites changed. A timer window rather than queueMicrotask — atoms mount across separate React commits and a microtask can't span one. Deliberately not Effect.request/RequestResolver: those batch within a single fiber's execution graph, and these callers are on unrelated fibers.
• GET /api/integrations/vcs/commits?shas=… plus bulk repo lookups, so a table resolves in two DB reads instead of one per row. Unresolvable SHAs are omitted rather than failing the batch; provider probes for unknown SHAs are capped.

Verification

Ran a real stack (API + web + electric-sync on spare ports) and drove Chrome:

• access-control-max-age: 86400 on a live worker
• Two queries coalesced into one /execute-batch, both outcome: "success" with live warehouse data
• One bulk commit request resolving the whole 31-service table
• A full /services load issuing zero OPTIONS — 6 API calls, every preflight served from cache

Tests: 151 api + 58 web passing. Includes a regression guard asserting both access-control-max-age and that access-control-allow-headers contains Authorization, plus tests for the shared batch deadline (bounded so it discriminates a shared deadline from a per-item timeout) and per-item failure isolation.

Reviewer notes

• The services-table.tsx diff looks like 342 lines but is 67 real insertions — the rest is re-indentation from wrapping the JSX in the new provider. git diff -w shows the truth.
• Out of scope, and still the biggest remaining lever: the server-side p95 tail. Every endpoint — including trivial ones like /api/billing/customer (p50 92ms / p95 11.2s) and /v2/ingest_keys (p50 16ms / p95 12.7s) — shows a bimodal 11–17s tail uncorrelated with what the endpoint does. That's the org-config Postgres read on the hot path plus the documented dial hang. This PR reduces exposure to that tail by cutting request count, but does not fix it.
• Developed against 0b2721314; rebased cleanly onto f68f3ad60. The two perf PRs that landed in between touch no file in this change set.

Follow-up: batching added head-of-line blocking (second commit)

Worth calling out, because this PR creates the problem and only bounds it.

Batch delivery is all-or-nothing — the caller gets every result together, so a batch resolves at the speed of its slowest member. Against a per-query p50 of ~250–600ms and a p95 of ~5s, a large batch will often be gated by one tail query. On /services (2 queries) that's noise; on a dashboard grid it means every widget waits for the worst one. Eleven fast widgets blocked on one slow query is worse than the requests we saved.

The second commit holds the client cap (QUERY_ENGINE_BATCH_MAX) equal to the server's fan-out concurrency (QE_BATCH_CONCURRENCY, 4), so a batch is exactly one wave and can't be gated by a tail query sitting in a later wave. Both constants carry a note to keep them in step.

That's a bound, not a fix. The fix is to stream the outcomes as an SSE event stream so each query settles the moment it completes — one round trip, zero extra preflights, and progressive paint, which removes the tradeoff instead of tuning around it. It also improves failure behaviour: today a 45s client abort discards the whole batch, whereas anything already streamed stays delivered.

The pieces exist on both ends — Effect's HttpApi has first-class streaming success (HttpApiSchema.StreamSse, including a reserved mid-stream failure event) and HttpApiClient decodes it into an Effect Stream. The client batcher is already shaped for it: it deliberately sits outside the atom system holding a map of pending promises, so it only needs to settle them one at a time instead of all at once. That lands as its own PR, after which the cap can rise again.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[github-actions]

🍁 Maple PR preview

Note

Preview resources were removed when this pull request closed.

Final commit b5f22ba · View workflow run