Skip to content

feat(sdk): let liveQuery consumers distinguish NATS replay from genuinely-live events #98

Description

@jfwoods

Problem

In @wavehouse/sdk LiveQuery.next, messages still buffered in NATS retention are replayed through the same next: callback as truly-live events. The next: payload is just event.data — no event.replayed, no server-side delivered_at, no separate replay: stream — so a consumer can't gate replayed rows out of latency math without a heuristic.

Any consumer that subscribes with a non-trivial since window (the recommended way to get a smooth backfill) and experiences a brief disconnect→reconnect, or has retention containing messages emitted before the client connected, will see replayed rows arrive through next: indistinguishable from live arrivals. For latency monitoring (event-to-browser timing) this fabricates large recv_ts - emit_ts deltas that aren't real wire latency — observed downstream as AVG-LATENCY cards showing hundreds of seconds with a line decaying linearly toward zero as the page stays open.

Proposed Solution

Add a server-known signal to the next: path so consumers can separate replay from live. Any one of:

  1. event.replayed: boolean on next: payloads — minimal surface, the server already knows the distinction.
  2. Server-side delivered_at timestamp per event — lets a consumer compute delivered_at - emit_ts (server-side propagation) and now() - delivered_at (wire latency) separately. Most expressive.
  3. Separate replay: callback distinct from next: — clearest separation, biggest API change.

Recommend (2) for monitoring use cases, or (1) as the smallest change that unblocks all latency-monitoring consumers.

Alternatives Considered

  • Consumer-side threshold heuristic (drop rows where recv_ts - emit_ts > ~5s) — current downstream workaround; self-contained but blunt, and wrong whenever real latency or real backfill age crosses the threshold.

Additional Context

From nas-observability WHissues.md, 2026-05-08. Note #87 (MVP cuts) keeps "stream raw events as received" as the surviving live path — i.e. exactly the liveQuery surface this issue hardens — so this is complementary to that scoping, not in tension with it. Unrelated to #33/PR #83 (Pipes KV watcher) despite the shared "real-time" framing — different subsystem.

Activity

  1. added theissue type on May 11, 2026
  2. moved this from Backlog to Ready in WaveHouse Task Boardon May 18, 2026
  3. EricAndrechek commented on May 19, 2026

    @EricAndrechek
    Member

    Planning to tag data replayed or in a since query (from NATS) with metadata at the outer layer/wrapping to make it clear it is not live, and leave live data as un-batched and untagged.

  4. moved this from Ready to Backlog in WaveHouse Task Boardon May 20, 2026
  5. moved this from Backlog to Ready in WaveHouse Task Boardon May 20, 2026
  6. moved this from Ready to In progress in WaveHouse Task Boardon May 27, 2026
  7. moved this from In progress to Ready in WaveHouse Task Boardon Jun 2, 2026
  8. EricAndrechek commented on Jun 4, 2026

    @EricAndrechek
    Member

    Reprioritization (launch-gated pass): re-anchored P1→P2 — distinguishing NATS replay from genuinely-live events is SDK DX (and breaking): wanted, but not alpha-blocking. Rationale on #149.

  9. moved this from Ready to Backlog in WaveHouse Task Boardon Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area/observabilityMetrics, logs, traces, health, profilingarea/sdkTypeScript SDK (clients/ts/)area/streamingSSE / live-query delivery path (/v1/stream)breaking-changeBreaking change to public API, CLI, or configenhancementNew feature or request

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions