Repository navigation
bug(pipes): mutation pipe results are cached and coalesced — the write silently drops on repeat calls #386
Description
Summary
/v1/pipes/{name} runs its SQL through the pipe cache + singleflight with no mutation guard, even though the shared executor deliberately supports mutation-verb SQL. A pipe that writes returns a cached [] on repeat calls and never re-executes.
Detail
internal/api/clickhouse_exec.go:26-31dispatches mutation-verb SQL throughconn.Exec, so an INSERT/ALTER pipe does execute the first time.internal/api/pipes.go:176-221caches the result underqueryCacheKey(sql, params)withcache.QueryTimeToTTL(floor 10s, up to 1h) and wraps execution in singleflight — with noisMutation(sql)check on either thePut(store) or execute path.
Scenario: an admin authors INSERT INTO audit_log VALUES ({{msg}}, now()) with allowed_roles: ["writer"]. First call inserts and caches []. Every identical call within the TTL returns 200 + X-Cache: HIT and never reaches ClickHouse; concurrent identical calls are collapsed by singleflight to a single insert.
Impact
Silent data loss on mutation pipes. Docs frame pipes as the cached read path but never forbid mutation SQL, and the exec layer explicitly supports it.
Fix direction
Either reject mutation SQL at pipes.Store.Put, or bypass cache + singleflight when isMutation(sql).
Found in a repo-wide audit; verified by code trace. Distinct from #321/#365.
Activity
coderabbitai commentedon Jul 8, 2026
🔗 Related PRs
#164 - refactor(ingest)!: insert-only pipeline; mutations via /v1/query [closed]
#177 - feat: caching for local table/scope invalidation [closed]
#330 - fix(pipes): escape non-scalar param values [closed]
🧪 Issue enrichment is currently in open beta.
You can configure auto-planning by selecting labels in the issue_enrichment configuration.
To disable automatic issue enrichment, add the following to your
.coderabbit.yaml:💬 Have feedback or questions? Drop into our discord!
EricAndrechek commentedon Sep 25, 2026
#630 (E4, draft) widens the blast radius. With
cache.backend: redisevery instance shares the pipe result cache. So a repeated identical call to a pipe that writes is answered from the cache, and the write is skipped, on every instance until the entry's TTL, not just on the instance that cached it.Pipe entries have no table dependencies, so ingest never invalidates them. Only the TTL (10 s to 1 h) or
InvalidateTenantclears them. This should be fixed before any real deployment setsbackend: redis. #634 (draft) is the fix. The risk is documented in the "Multiple instances and the shared cache" section ofdeployment.md(#630).