Skip to content

feat(api): align health endpoints with K8s convention (/healthz, /readyz + per-dependency sub-endpoints) #144

Closed

Description

@EricAndrechek

Problem

WaveHouse exposes /health and /ready (registered in internal/api/router.go:87-88), which work but don't follow the K8s convention (/healthz, /readyz). Operators wiring kube livenessProbe / readinessProbe paths against a WaveHouse deployment have to remember the non-standard names.

Additionally, the current endpoints return a single overall status. When /ready flips to NOT_READY, there's no fast way to see which dependency is the cause (NATS? ClickHouse? Schema discovery? DLQ?). Per-dependency sub-endpoints would let on-call see the failing leg without grepping logs.

Decide before alpha

Compat window for /health and /ready is the gating decision. Options:

  • Hard-cut: rename only. /health and /ready return 404 in v0.1.0-alpha.1. Cleanest, but anyone who scripted against the old names in the pre-alpha period breaks.
  • Aliases: register both. Half the disk for the cost of one comment in CHANGELOG noting /health//ready are deprecated and will be removed in v0.2.0. (Recommended.) The alpha is the cheapest moment to introduce the K8s names; deferring locks /health//ready in as v1.0 stable surface forever.

Aggregate /readyz behavior is the other open question. Recommend evaluating all subsystems and returning a unified failure picture ({"ready": false, "checks": {"nats": "ok", "clickhouse": "fail: timeout", ...}}) rather than short-circuiting — same latency in the happy path and far more useful in the failure path.

Proposed Solution

  1. Register /healthz + /readyz as the canonical paths going forward. Keep /health + /ready as aliases for v0.1.x; drop in v0.2.0. Document in CHANGELOG.
  2. Add per-dependency status endpoints, all under /healthz/:
    • /healthz/nats — embedded JetStream + connection status
    • /healthz/clickhouse — DB ping + driver state
    • /healthz/schema — last successful refresh timestamp
    • /healthz/dlq — stream depth / consumer health
  3. Keep all of these lightweight (no fan-out to internal stats that would themselves block).

Sequencing

Recommend landing before v0.1.0-alpha.1 (#149) — the URL contract is harder to change post-launch than pre-launch.

Additional Context

Distinct from #95, which addresses the schema-discovery retry/503 semantics — that issue focuses on boot-time non-fatal handling. This issue is about endpoint naming + per-dependency drill-down.

Metadata

Metadata

Assignees

Labels

area/apiHTTP handlers, routing, middlewarearea/observabilityMetrics, logs, traces, health, profilingbreaking-changeBreaking change to public API, CLI, or configenhancementNew feature or request

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions