typelayer: tenant server settings aren't described to chtypes (after #712)

Not actionable yet. This is follow-up work for after #712 lands on chtypes v2. It depends on code #712 introduces and on a chtypes release that doesn't exist yet (see Blocked on below). Please don't pick it up before both are true.

Problem. chtypes judges ingest records and evaluates stream row filters in-process. It is told the tenant's ClickHouse server time zone, but not the settings that server applies to every query: the settings profile of the ClickHouse user WaveHouse connects as, or the server's own defaults.

The real paths run under those settings:

the ingest worker's INSERT, with the type layer's parsing settings pinned over them;
/v1/query reads, with WaveHouse's read settings pinned over them.

chtypes uses ClickHouse's stock defaults plus the same pins. So when a tenant's profile changes a setting that affects parsing, DEFAULT evaluation or comparison, the in-process answer can differ from the server's:

Ingest: a record accepted in-process can be refused by the real INSERT, and is then parked on the DLQ.
Streams: a role's row filter on the SSE stream can answer differently from the same role's /v1/query. Because only a definite true admits a row on the stream, this matters for row-level security: a profile that changes how WHERE compares values could let the stream admit a row that /v1/query hides.

Known instance (documented in #712). docs/src/content/docs/access-control.mdx (claim comparison on a timestamp column): a 26.8 server whose profile sets cast_string_to_date_time_mode=basic refuses an RFC 3339 claim on /v1/query (code 53), while the stream still matches it.

What WaveHouse reads today. Schema discovery reads, per refresh over the tenant's own connection:

SELECT timezone();
SELECT version();
system.columns;
system.tables.create_table_query.

It reads nothing about settings. The settings that matter are the ones the tenant's user sees in its own session; SELECT name, value FROM system.settings WHERE changed over that connection lists them.

What chtypes offers. With #712's move to chtypes v2, each tenant's tables compile on a chtypes server handle (chtypes.ServerProfile) that carries the tenant's time zone (Wave-RF/chtypes#419). The profile also has a Settings member, for a server profile's settings layered under a schema's and a call's own. Upstream has not yet proven that member, and today it reaches parsing but not filter evaluation. chtypes tracks the filter-enforcement requirements in Wave-RF/chtypes#567.

Blocked on:

feat(typelayer)!: validate, coerce and filter through chtypes #712 landing on chtypes v2. The server handle doesn't exist on main.
chtypes v2 applying server-profile settings to filter evaluation, honouring each setting or declining it by name.

Then (see the latest comment): pass the tenant's changed settings, minus WaveHouse's own per-query pins, in its server profile. Refuse streaming for that tenant if chtypes declines any of them, or if a setting that selects rows by content (additional_table_filters, additional_result_filter, apply_deleted_mask, final, …) is changed.

Status update from chtypes (still not actionable; this remains after #712):

Input settings: proven upstream. chtypes reports that profile input settings now round-trip on its v2 dev builds, with per-call > schema > server precedence. Covered: date_time_input_format, input_format_null_as_default, input_format_defaults_for_omitted_fields, input_format_skip_unknown_fields and input_format_allow_errors_num. A refused profile fails server creation with the server's own error code. So the ingest half (parse, acceptance, export) can take a tenant's input settings once this is picked up.
Filters: still open. Profile settings don't reach filter evaluation yet, so row filters and insert checks still use stock semantics. The stream//v1/query divergence stays until chtypes closes that, or until the fail-closed rule (refuse streaming when a WHERE-affecting setting is changed) can use chtypes' published list.

Two notes for whoever picks this up (still not actionable; still after #712):

Key the refuse-to-stream rule on chtypes' tier of settings that change a row's verdict, not on a raw list of every setting that touches query evaluation. chtypes is publishing its list of WHERE-affecting settings in tiers. The broad tier deliberately over-approximates: it includes max_threads, max_execution_time, log_queries and output-format settings. Keyed on that tier, the rule would refuse streaming for nearly every tenant.
When reading system.settings WHERE changed, leave out the settings WaveHouse pins per query itself. Those include readonly, max_execution_time, wait_end_of_query, the JSON and date output settings, and the ingest pins. They show what WaveHouse sets, not what the operator's profile does.

Correction to my previous comment. Keying the refuse-to-stream rule only on settings that change a row's verdict is too narrow. Refuse streaming when a tenant has changed any setting in these chtypes tiers:

settings that change a predicate's verdict, including ones measured not to flip the verdicts tested so far;
settings that select rows by content: additional_table_filters, additional_result_filter, apply_deleted_mask, final, the newer filter/select-construction settings, and anything chtypes hasn't classified. An operator can use these as access control, so streaming past them would leak rows by the operator's own intent.

Settings that only truncate or page results (limit, offset, max_*_rows with break) don't refuse, since they affect position or count, not which rows a role may see. chtypes' v2 spec will state this rule with the tiered list. Excluding the settings WaveHouse itself pins per query still applies.

This supersedes my two previous comments. The plan is simpler. Output-format settings turn out to affect filter evaluation too (through functions like formatRow, toJSONString and casts to String), so a refuse rule keyed on a list of setting tiers would fire for most profiles. Instead, once chtypes' v2 applies server-profile settings to filter evaluation, which WaveHouse waits for anyway:

Read the tenant's SELECT name, value FROM system.settings WHERE changed, excluding the settings WaveHouse pins per query itself, and pass them in the tenant's chtypes server profile.
chtypes honours each setting in filter evaluation, or declines it by name.
Refuse streaming for that tenant if chtypes declines any setting, or if any setting that selects rows by content is changed: additional_table_filters, additional_result_filter, apply_deleted_mask, final, and the newer filter/select-construction settings.

No interim refuse-by-list rule is needed. The issue body is updated to match.

chtypes has published the lists this issue refers to (still not actionable; still after #712):

Location: spec/abi-v2/where-settings/ on the chtypes v2 branch. There is one list per ClickHouse line (26.3.json, 26.7.json, 26.8.json, 26.9.json) plus union.json.
Entries: each is {name, kind, tier, reason, scopes, sources}, with tiers predicate, predicate-unflipped, result-content, result-truncate, execution and output.
Pinning: the README holds the checksums, and scripts/abi-v2/check-where-settings.py --check verifies a copy.

The v2.0 rule, as the README states it, matches this issue's plan. Pass the tenant's changed settings in the server profile, and refuse streaming when chtypes declines a setting by name, or when any result-content setting has a non-default value. result-truncate never refuses. The lists document which settings matter; they are not the refuse key.

While WaveHouse is on a v2 dev build that doesn't yet decline settings by name, refuse on any changed setting whose tier is predicate, predicate-unflipped or result-content.

Navigation Menu

Repository navigation

typelayer: tenant server settings aren't described to chtypes (after #712) #762

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

typelayer: tenant server settings aren't described to chtypes (after #712) #762

Description

Activity

EricAndrechek commented on Oct 7, 2026

EricAndrechek commented on Oct 7, 2026

EricAndrechek commented on Oct 7, 2026

EricAndrechek commented on Oct 7, 2026

EricAndrechek commented on Oct 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions