Repository navigation
perf(cache): query cache returns stale data after writes (read-your-writes) #73
Description
Update (2026-05-18): Resolved by PR #119 (
feat(cache): implement per-table tagging and raw SQL bypass), which combines Option B + Option C below per the architecture review. Closing once #119 merges. The original problem statement and option analysis are kept below for historical context.Any post-merge follow-up (e.g. tail-chopping for high-write tables; tracked in #86) belongs in its own issue, not as a re-open here.
Problem
internal/api/query.go:QueryHandler caches /v1/query responses keyed on sha256(sql + params) with a 5s default TTL. The cache has no invalidation signal from ingest writes — reads can return stale data for up to TTL after a write to the same table(s).
Surfaced in #71's new E2E Tests job as a cache-key collision between two tests:
admin.test.ts:182runswh.sql('SELECT count() as cnt FROM default.clicks')against an emptyclickstable at the top of the run → caches{cnt: 0}.query.test.ts:97runs the byte-identical query afterbeforeAllseeds 10 rows → hits the cached{cnt: 0}and failstoBeGreaterThan(0).
The test collision is trivially fixable, but it exposes the broader read-your-writes hole: in production, an admin that inserts a row and immediately runs sql(count()) sees the stale count for up to 5s. Structured queries (/v1/query-builder-style) have the same risk whenever they hit the same cache layer.
Related
- Surfaced by: ci: bundle CI hygiene fixes for #70 #71 (bundled CI hygiene PR)
- Code:
internal/api/query.go—queryCacheKey(),h.Cache.Get/h.Cache.Set,h.sf.Dosingleflight - Cache layer:
internal/cache(TieredCache — L1 local + optional L2 shared) - Ingest path (the WAL): Bento → NATS JetStream → ClickHouse, 5s batch window
Options to decide in next standup
Option A — Read-your-writes via WAL buffer (Eric's initial proposal)
On read, augment the CH-cached result with rows still in the NATS WAL buffer for the tables the query references.
Pros: zero stale-read window by construction; users always see their own writes instantly.
Pushback (and why this is harder than it looks):
- Trivial for simple aggregates (
count,sum,avg) — you layer the in-memory buffer count on top of the cached result. - For anything richer —
GROUP BY, projections, joins,WHEREfilters on fields not in the cache envelope — you need to actually execute the query against the in-memory buffer. That's a mini query engine. - Call this out explicitly in the issue's design: don't drift toward 'let's write a query engine'. Scope the v1.
Option B — Per-table last-write-timestamp invalidation
Ingest writes to table X bump a last_write_ts[X] monotonic counter. Cache entries capture the list of tables they query (parse FROM at cache-set time). On cache Get, revalidate: miss if max(last_write_ts[tables-in-query]) > entry.written_at.
Pros: simple, works for every query shape (aggregates, GROUP BY, joins), no query engine needed.
Cons / tradeoffs:
- Cache thrash in hot-ingest scenarios — every insert to
clicksinvalidates everyclicks-touching cache entry. Tunable with coarser granularity (e.g. second-bucketed timestamps, per-tenant+table keys). - Requires a
FROM-clause parser. Can be naive (regex-based) for v1; lift to a real SQL AST parser later. - Still has a visibility window between the NATS publish and the cache invalidation — reads in that window still see stale data. Acceptable if the ingest → invalidate step is fast.
Option C — Bypass cache entirely for raw SQL
One-line change: in QueryHandler.Handle, skip the cache lookup when the query hits the /v1/query path (raw SQL, admin-gated).
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
EricAndrechek commentedon Apr 23, 2026
@taitelee thoughts?
taitelee commentedon Apr 29, 2026
Update: Post-Standup Decision & Architecture Path
Following our discussion, we've decided on an architecture that accepts eventual consistency for structured reads while providing explicit escape hatches for real-time and administrative workflows.
We are dropping option A (the NATS buffer merge / mini query engine) and option B (timestamp-based cache invalidation) in favor of a time-bucketed caching strategy combined with option C.
Here are some things we've agreed on:
1. The "acceptable delay" paradigm for standard reads$X$ seconds (the Bento buffer flush interval). We will rely entirely on ClickHouse as the source of truth for these queries, rather than trying to stich in-flight NATS data into the response. To ensure users can read writes once they land, we will trigger a full cache invalidation for a given table whenever Bento flushes a write batch to it.
We are accepting the fact that standard API reads will have an inherent delay of up to
2. The real-time escape hatch (streams)
If a client requires strict, sub-second real-time data, they will have to consume it using a direct stream rather than the API.
WHERE,LIMIT, aggregations) to the in-flight stream. Filtering must be handled client-side.3. Time-Bucketed caching and "tail chopping"
Instead of naive cache invalidation, we will leverage ClickHouse to bucket data based on timestamps, and we will cache those discrete buckets.
now() - 30s), the cache layer will serve the pre-computed, fully closed time buckets that fit within that window.now() - Xwill rarely align perfectly with our bucket edges, the system will take the cached full buckets and "chop off the tail" of the interval. This will dynamically query or slice the fraction of the most recent/boundary bucket to return the exact requested time window. This ensures that for overlapping relative queries between write flushes, we only need to dynamically calculate the narrow slice of recent data rather than the entire aggregate.4. Raw SQL cache bypass (implementing option C)
We are moving forward with option C for admin workflows. Any raw SQL executed against the
/v1/queryendpoint will completely bypass the API cache layer. This ensures that admins executing ad-hoc queries (like the ones in our E2E tests) will always hit ClickHouse directly, avoiding the confusion of stale cached aggregates during manual debugging.EricAndrechek commentedon Apr 29, 2026
Worth noting: we do eventually want realtime (push-based) data even for arbitrary queries/pipes, but this is proving more complicated than original though, even for "simple" cases like SUMs, as evaluating the WHERE and LIMIT cases requires us to build a lite version of the CH SQL engine which is beyond scope, so we are deferring anything beyond raw event streaming to use CH as the only source, and not attempt any aggregations with NATS buffer / pseudo-WAL data. This means a follow-up issue
will be opened SOON™️has been opened (#87) to address this and REMOVE code/functionality that is currently half baked to do this, instead only allowing streams for an entire table. More on that in #87.We also want the time-bucketed approach to be as CH-default/CH-managed as possible, so that we don't have to think about and implement different bucketting/binning tactics for different queries (ie SUM vs COUNT).
Additionally, we are acknowledging the limitations that our Bento-triggered full-table invalidation approach causes, including:
feat: caching for local table/scope invalidation (#177)