Skip to content

feat(observability): add WH_OTEL_ENABLED to short-circuit telemetry init #96

Description

@jfwoods

Problem

In cmd/wavehouse-{api,worker}/main.go (~lines 74 / 54) and
internal/observability/provider.go, when WH_OTEL_ADDR resolves but no collector
is listening, the OTLP exporter logs traces export: exporter export timeout: rpc error: code = Unavailable … connect: connection refused on every flush interval
(plus quieter noise on the metrics side). During a phased rollout — WaveHouse up
before the collector/receiver is deployed — this spams logs continuously. There's no
off switch; the only workaround is pointing WH_OTEL_ADDR at an unreachable
address, which just trades the timeout warning for a retry warning.

Proposed Solution

Add WH_OTEL_ENABLED (default true). When false, observability.InitProvider
returns a no-op shutdown and skips both exporter setup and runtime.Start.
Operators running WaveHouse without a collector shouldn't have to run a sink to
silence boot.

Alternatives Considered

  • Auto-detect collector reachability and disable on failure — racy, and hides
    genuine misconfiguration once a collector is expected.
  • Just lower the exporter log level — loses visibility when telemetry should be
    working; doesn't avoid the wasted export attempts.

Additional Context

From nas-observability WHissues.md, 2026-05-08. Related to #47 (env-var config)
for the override-precedence convention. This is the missing off-switch for the OTLP
pipeline stood up by #14 (Centralize Telemetry via Single OTLP Pipeline) and #28
(OTLP Log Exporting) — both closed, neither added a way to disable export.

Activity

  1. added theissue type on May 11, 2026
  2. moved this from Backlog to In progress in WaveHouse Task Boardon May 12, 2026
  3. moved this from In progress to In review in WaveHouse Task Boardon May 12, 2026
  4. added a commit that references this issue on May 13, 2026
    d1007c1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/infraCI, build, deploy, Docker, releasearea/observabilityMetrics, logs, traces, health, profilingenhancementNew feature or request

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions