Repository navigation
chore: remove executeCHQuery and move to CH HTTP API #171
Copy link
Copy link
Open
Labels
area/apiHTTP handlers, routing, middlewareHTTP handlers, routing, middlewarearea/cacheLocal / shared / tiered cachingLocal / shared / tiered cachingarea/pipesNamed query pipesNamed query pipesarea/queryStructured query AST, SQL builderStructured query AST, SQL builderbreaking-changeBreaking change to public API, CLI, or configBreaking change to public API, CLI, or configenhancementNew feature or requestNew feature or request
Description
Proposal: Unify ClickHouse Execution over HTTP API (Remove executeCHQuery)
Background & Problem
Currently, the API maintains two disparate paths for interacting with ClickHouse:
- HTTP Proxy: Used by
internal/api/query.goto forward raw SQL directly to ClickHouse. - Native Driver (
clickhouse-go): Used by programmatic endpoints (internal/api/structured_query.goandinternal/api/pipes.go), which rely on theexecuteCHQuerywrapper found ininternal/api/clickhouse_exec.go.
The native driver path introduces significant complexity. To determine whether to use .Query() or .Exec(), clickhouse_exec.go relies on an isMutation heuristic, CTE parsing, and comment-stripping logic. As noted in internal/api/query.go, this logic is fragile, error-prone, and requires ongoing maintenance as ClickHouse introduces new keywords and syntax.
Proposed Solution
We should eliminate executeCHQuery entirely and standardize all ClickHouse interactions on the HTTP interface used in internal/api/query.go. This simplifies the architecture, reduces our bug surface, and potentially improves performance by reducing serialization steps.
Affected Files
internal/api/clickhouse_exec.go(Targeted for removal)internal/api/query.go(Reference for HTTP proxy implementation)internal/api/structured_query.go(Needs refactoring)internal/api/pipes.go(Needs refactoring)
Implementation Options & Side Effects
1. Parameter Binding
- Concern: Switching to the HTTP API means we lose
clickhouse-go's standard?parameter binding used instructured_query.goandpipes.go. - Option/Action: We must rewrite the parameter generation logic in
query.Buildandpipes.BindParams. Instead of standard placeholders, we need to implement ClickHouse's HTTP parameterization scheme (e.g., using{name: DataType}syntax in the query and appendingparam_my_var=...to the HTTP request query string).
2. Caching Mechanism Updates
- Side Effect (Positive): Currently, the cached paths unmarshal the driver results into
[]map[string]anyand then calljson.Marshalto store the data in theTieredCache. - Option/Action: Moving to HTTP allows us to skip JSON marshaling/unmarshaling entirely. We can extract the raw
databyte array straight from the ClickHouse HTTP JSON response and write those bytes directly into the cache and thehttp.ResponseWriter.
3. Standardizing Type Formatting
- Concern:
transformRowinclickhouse_exec.gocurrently intercepts and manually formatsuuid.UUIDandtime.Time(to RFC3339Nano) for JSON safety. - Option/Action: When proxying via HTTP, we rely on ClickHouse to output the JSON directly. To ensure we don't break our API contract for Dates and UUIDs, we need to enforce formatting via URL query parameters, such as passing
date_time_output_format=isoto the proxy client (similar to how it is currently handled inquery.go).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area/apiHTTP handlers, routing, middlewareHTTP handlers, routing, middlewarearea/cacheLocal / shared / tiered cachingLocal / shared / tiered cachingarea/pipesNamed query pipesNamed query pipesarea/queryStructured query AST, SQL builderStructured query AST, SQL builderbreaking-changeBreaking change to public API, CLI, or configBreaking change to public API, CLI, or configenhancementNew feature or requestNew feature or request
Type
Projects
- StatusShow more project fieldsBacklog
Activity
refactor(table names): handle unsafe table names (#174)
EricAndrechek commentedon Jun 2, 2026
Update: We may still actually want to use both – there are cases where the native go version is ideal, like its lower latency and resources and higher efficiency and compression, etc, especially for cases where we know the exact query/format and know that there is no user-input we need to safely escape.
HTTP remains the ideal method for structured queries and direct raw admin SQL, but schema discovery etc may be best to remain on the go driver. TBD on pipes and where all the other things fall.
This issue should then become more focused on:
clickhouse_exec.go