Skip to content

chore: remove executeCHQuery and move to CH HTTP API #171

Open

Description

@EricAndrechek

Proposal: Unify ClickHouse Execution over HTTP API (Remove executeCHQuery)

Background & Problem

Currently, the API maintains two disparate paths for interacting with ClickHouse:

  1. HTTP Proxy: Used by internal/api/query.go to forward raw SQL directly to ClickHouse.
  2. Native Driver (clickhouse-go): Used by programmatic endpoints (internal/api/structured_query.go and internal/api/pipes.go), which rely on the executeCHQuery wrapper found in internal/api/clickhouse_exec.go.

The native driver path introduces significant complexity. To determine whether to use .Query() or .Exec(), clickhouse_exec.go relies on an isMutation heuristic, CTE parsing, and comment-stripping logic. As noted in internal/api/query.go, this logic is fragile, error-prone, and requires ongoing maintenance as ClickHouse introduces new keywords and syntax.

Proposed Solution

We should eliminate executeCHQuery entirely and standardize all ClickHouse interactions on the HTTP interface used in internal/api/query.go. This simplifies the architecture, reduces our bug surface, and potentially improves performance by reducing serialization steps.

Affected Files

  • internal/api/clickhouse_exec.go (Targeted for removal)
  • internal/api/query.go (Reference for HTTP proxy implementation)
  • internal/api/structured_query.go (Needs refactoring)
  • internal/api/pipes.go (Needs refactoring)

Implementation Options & Side Effects

1. Parameter Binding

  • Concern: Switching to the HTTP API means we lose clickhouse-go's standard ? parameter binding used in structured_query.go and pipes.go.
  • Option/Action: We must rewrite the parameter generation logic in query.Build and pipes.BindParams. Instead of standard placeholders, we need to implement ClickHouse's HTTP parameterization scheme (e.g., using {name: DataType} syntax in the query and appending param_my_var=... to the HTTP request query string).

2. Caching Mechanism Updates

  • Side Effect (Positive): Currently, the cached paths unmarshal the driver results into []map[string]any and then call json.Marshal to store the data in the TieredCache.
  • Option/Action: Moving to HTTP allows us to skip JSON marshaling/unmarshaling entirely. We can extract the raw data byte array straight from the ClickHouse HTTP JSON response and write those bytes directly into the cache and the http.ResponseWriter.

3. Standardizing Type Formatting

  • Concern: transformRow in clickhouse_exec.go currently intercepts and manually formats uuid.UUID and time.Time (to RFC3339Nano) for JSON safety.
  • Option/Action: When proxying via HTTP, we rely on ClickHouse to output the JSON directly. To ensure we don't break our API contract for Dates and UUIDs, we need to enforce formatting via URL query parameters, such as passing date_time_output_format=iso to the proxy client (similar to how it is currently handled in query.go).

Activity

  1. moved this from Backlog to Ready in WaveHouse Task Boardon Jun 2, 2026
  2. removed their assignment
    on Jun 2, 2026
  3. EricAndrechek commented on Jun 2, 2026

    @EricAndrechek
    MemberAuthor

    Update: We may still actually want to use both – there are cases where the native go version is ideal, like its lower latency and resources and higher efficiency and compression, etc, especially for cases where we know the exact query/format and know that there is no user-input we need to safely escape.

    HTTP remains the ideal method for structured queries and direct raw admin SQL, but schema discovery etc may be best to remain on the go driver. TBD on pipes and where all the other things fall.

    This issue should then become more focused on:

    1. migrating our structured queries and raw admin SQL to be HTTP requests so we can eliminate clickhouse_exec.go
    2. Building connections (regardless of native go or http api) to be made from a dedicated package, so passing CH access to other packages is a singular reference to the clickhouse connection package, which then handles auth + retries, clickhouse error handling, network issues, http transport options, pooling, etc.
  4. moved this from Ready to Backlog in WaveHouse Task Boardon Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/apiHTTP handlers, routing, middlewarearea/cacheLocal / shared / tiered cachingarea/pipesNamed query pipesarea/queryStructured query AST, SQL builderbreaking-changeBreaking change to public API, CLI, or configenhancementNew feature or request

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions