Repository navigation
chore(api): simplify hub fan-out — drop NATS-style wildcard subscriptions #100
Description
Part of #87. Scoped narrowly: this issue covers only the hub wildcard simplification bullet from #87 — it does not touch raw SQL, pipes, or structured query (those stay as-is for now).
Context
The hub (internal/api/hub.go) supports NATS-style * (one-token) and > (multi-token) wildcards on the subscriber side. After the MVP cuts in #89, every producer publishes a concrete ingest.<table> subject and the SDK always subscribes to one concrete subject. The wildcard fan-out is unused machinery — keep it and we keep paying for it in code, tests, and docs that imply the API is more general than it is.
What to change
internal/api/hub.go
- Delete
matchTopic(lines 121-142). - In
Broadcast(lines 62-119): drop the wildcard pattern loop (lines 100-118) and thesentdedup map; keep only the exact-topic delivery loop. - Remove the now-unused
stringsimport. - Keep
RLock/RUnlock— the lock contract is unchanged.
internal/api/hub_test.go
Delete the eight wildcard tests (lines 212-411):
TestHub_WildcardGreaterThanTestHub_WildcardStarTestHub_WildcardStarNoMultiTokenTestHub_WildcardGreaterThanMultiTokenTestHub_WildcardDoesNotMatchExactTestHub_BareGreaterThanMatchesAllTestHub_WildcardNoDuplicateDeliveryTestMatchTopic
internal/api/stream_sse.go (lines 37-40)
Kill the topic = "ingest.>" default. Decide one of:
- (A) Return 400 when
?topic=is missing. - (B, recommended) Replace
?topic=ingest.<table>with?table=<name>and build the subject server-side. This stops leaking the NATS subject convention into the public HTTP API. SDK is a one-line change either way; directcurlusers have to switch from?topic=ingest.clicksto?table=clicks.
internal/api/stream_ws.go
?topic=query-param fallback (lines 125-140): same A/B decision as SSE.- In-band
{\"action\":\"subscribe\",\"topic\":\"…\"}command (lines 149-158): reject any topic containing*or>with an in-band error frame. (Or rename the command field totableif going with B.)
internal/api/stream_test.go
Re-check: any test relying on the wildcard default needs to set an explicit table/topic.
clients/ts/src/stream/ws-manager.ts
- Drop the wildcard dispatch loop (lines 172-180) — already unreachable since
client.ts:90only ever buildsingest.<table>. - Delete the
matchTopicPatternhelper (~line 243). - Update the doc comment at lines 16-24 to remove the "wildcard" hint.
- If option B is chosen: change
client.ts:90to sendtableinstead oftopicto match the new server contract.
Docs
docs/api.mdlines 305-306 (table row claiming wildcards) and line 352 — changetopicdefault to "(required)" or rename totable, and remove the "Supports NATS wildcards:*matches one token,>matches one or more remaining tokens" sentence.docs/api.md:335— curl example already uses a concrete topic; just update if renaming.docs/architecture.md:93— "Default topic isingest.>(all tables)" → describe the new explicit-table contract.docs/development.md:96—?topic=ingest.clicksexample still works; update if renaming.CHANGELOG.mdunder[Unreleased]→Removed(andChangedif renaming): hub wildcard subscriptions; SSE/WS now require an explicit table.
Out of scope (do not touch in this PR)
Per the #87 audit, these are deliberately left alone here:
/v1/query(raw SQL),wh.sql()in the SDK,policy.RawSQL./v1/pipes/*,/v1/admin/pipes/*,internal/pipes/,WH_PIPES_DIR,wh.pipe()/wh.pipes.*./v1/tables/{table}/query(structured query AST) andinternal/query/.
Decisions to lock in before coding
- Option A or B above? (Recommend B — clean break, no leak of NATS subject names into the HTTP API.)
- WS in-band command field name — stay on
topic(with pattern rejection) or rename totable? Should match the SSE choice for consistency.
Bonus cleanup (optional, dead code already)
clients/ts/src/pipes.ts:40-42 — PipeRef.stream() calls _createStream(pipeName), which subscribes to ingest.<pipeName> for a NATS subject no producer publishes. Dead path regardless of this work; worth removing if you're already in the SDK streaming files.
Estimated impact
- ~150 LOC removed (mostly tests).
- Zero new abstractions, no new tests.
- Breaking change to SSE/WS public API: callers must supply an explicit table/topic (and may need to rename the query param if going with B).
make cishould remain green — the deleted tests are the only ones exercising wildcard behavior.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
fix(sdk): WS envelope parsing; doc + test cleanup from CodeRabbit
chore(api)!: drop hub wildcard fan-out; SSE/WS use ?table= (closes #100…