Skip to content

chore(api): simplify hub fan-out — drop NATS-style wildcard subscriptions #100

Closed

Description

@EricAndrechek

Part of #87. Scoped narrowly: this issue covers only the hub wildcard simplification bullet from #87 — it does not touch raw SQL, pipes, or structured query (those stay as-is for now).

Context

The hub (internal/api/hub.go) supports NATS-style * (one-token) and > (multi-token) wildcards on the subscriber side. After the MVP cuts in #89, every producer publishes a concrete ingest.<table> subject and the SDK always subscribes to one concrete subject. The wildcard fan-out is unused machinery — keep it and we keep paying for it in code, tests, and docs that imply the API is more general than it is.

What to change

internal/api/hub.go

  • Delete matchTopic (lines 121-142).
  • In Broadcast (lines 62-119): drop the wildcard pattern loop (lines 100-118) and the sent dedup map; keep only the exact-topic delivery loop.
  • Remove the now-unused strings import.
  • Keep RLock/RUnlock — the lock contract is unchanged.

internal/api/hub_test.go

Delete the eight wildcard tests (lines 212-411):

  • TestHub_WildcardGreaterThan
  • TestHub_WildcardStar
  • TestHub_WildcardStarNoMultiToken
  • TestHub_WildcardGreaterThanMultiToken
  • TestHub_WildcardDoesNotMatchExact
  • TestHub_BareGreaterThanMatchesAll
  • TestHub_WildcardNoDuplicateDelivery
  • TestMatchTopic

internal/api/stream_sse.go (lines 37-40)

Kill the topic = "ingest.>" default. Decide one of:

  • (A) Return 400 when ?topic= is missing.
  • (B, recommended) Replace ?topic=ingest.<table> with ?table=<name> and build the subject server-side. This stops leaking the NATS subject convention into the public HTTP API. SDK is a one-line change either way; direct curl users have to switch from ?topic=ingest.clicks to ?table=clicks.

internal/api/stream_ws.go

  • ?topic= query-param fallback (lines 125-140): same A/B decision as SSE.
  • In-band {\"action\":\"subscribe\",\"topic\":\"…\"} command (lines 149-158): reject any topic containing * or > with an in-band error frame. (Or rename the command field to table if going with B.)

internal/api/stream_test.go

Re-check: any test relying on the wildcard default needs to set an explicit table/topic.

clients/ts/src/stream/ws-manager.ts

  • Drop the wildcard dispatch loop (lines 172-180) — already unreachable since client.ts:90 only ever builds ingest.<table>.
  • Delete the matchTopicPattern helper (~line 243).
  • Update the doc comment at lines 16-24 to remove the "wildcard" hint.
  • If option B is chosen: change client.ts:90 to send table instead of topic to match the new server contract.

Docs

  • docs/api.md lines 305-306 (table row claiming wildcards) and line 352 — change topic default to "(required)" or rename to table, and remove the "Supports NATS wildcards: * matches one token, > matches one or more remaining tokens" sentence.
  • docs/api.md:335 — curl example already uses a concrete topic; just update if renaming.
  • docs/architecture.md:93 — "Default topic is ingest.> (all tables)" → describe the new explicit-table contract.
  • docs/development.md:96 — ?topic=ingest.clicks example still works; update if renaming.
  • CHANGELOG.md under [Unreleased] → Removed (and Changed if renaming): hub wildcard subscriptions; SSE/WS now require an explicit table.

Out of scope (do not touch in this PR)

Per the #87 audit, these are deliberately left alone here:

  • /v1/query (raw SQL), wh.sql() in the SDK, policy.RawSQL.
  • /v1/pipes/*, /v1/admin/pipes/*, internal/pipes/, WH_PIPES_DIR, wh.pipe() / wh.pipes.*.
  • /v1/tables/{table}/query (structured query AST) and internal/query/.

Decisions to lock in before coding

  1. Option A or B above? (Recommend B — clean break, no leak of NATS subject names into the HTTP API.)
  2. WS in-band command field name — stay on topic (with pattern rejection) or rename to table? Should match the SSE choice for consistency.

Bonus cleanup (optional, dead code already)

clients/ts/src/pipes.ts:40-42 — PipeRef.stream() calls _createStream(pipeName), which subscribes to ingest.<pipeName> for a NATS subject no producer publishes. Dead path regardless of this work; worth removing if you're already in the SDK streaming files.

Estimated impact

  • ~150 LOC removed (mostly tests).
  • Zero new abstractions, no new tests.
  • Breaking change to SSE/WS public API: callers must supply an explicit table/topic (and may need to rename the query param if going with B).
  • make ci should remain green — the deleted tests are the only ones exercising wildcard behavior.

Activity

  1. added
    area/apiHTTP handlers, routing, middleware
    area/sdkTypeScript SDK (clients/ts/)
    breaking-changeBreaking change to public API, CLI, or config
    on May 11, 2026
  2. moved this from Backlog to Ready in WaveHouse Task Boardon May 11, 2026
  3. moved this from Ready to In progress in WaveHouse Task Boardon May 12, 2026
  4. moved this from In progress to In review in WaveHouse Task Boardon May 15, 2026
  5. added a commit that references this issue on May 18, 2026
    4fac0c3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/apiHTTP handlers, routing, middlewarearea/sdkTypeScript SDK (clients/ts/)breaking-changeBreaking change to public API, CLI, or configchore

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions