Skip to content

fix(types): parse JSON type hints with parenthesized types - #478

Merged
abonander merged 1 commit into
ClickHouse:mainfrom
hsdfat:json-type-hints-nested-parens
Oct 6, 2026
Merged

abonander merged 1 commit into
ClickHouse:mainfrom
hsdfat:json-type-hints-nested-parens

Conversation

@hsdfat

@hsdfat hsdfat commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #442.

JSON columns with a typed path whose type has its own parentheses could not be parsed from the RowBinaryWithNamesAndTypes header, so reading from or inserting into such a table failed:

CREATE TABLE t (bar JSON(test Nullable(String))) ENGINE = MergeTree
-> type parsing error: Unknown data type: Strin

Two independent problems in parse_json():

  • remove_json_header() used trim_end_matches(')'), stripping every trailing ) instead of just the one closing JSON(, so JSON(a Nullable(String)) became a Nullable(String and the inner type parsed as Strin.
  • The hint list was split on commas and settings were detected by = without tracking parentheses or string literals (only back-quoted paths were handled, since fix(types): parse back-quoted JSON path names #458). So JSON(a Map(String, Int32)) was split into two broken hints, JSON(a Enum8('x' = 1)) silently lost its hint because it looked like a name=value setting, and a SKIP REGEXP pattern containing a comma was split as well.

The fix

remove_json_header() now strips exactly JSON( and one closing ). The splitting helper introduced in #458 now also skips separators inside parentheses and single-quoted literals (renamed from unquoted_positions to top_level_positions), and is still used for the hint separator, the setting = check and the path/type boundary. Settings (max_dynamic_paths=N), SKIP, SKIP REGEXP and back-quoted paths behave as before; the existing tests are unchanged.

Verification

New unit test test_data_type_new_json_with_nested_hint_types, on main:

$ cargo test -p clickhouse-types --lib test_data_type_new_json
test data_types::tests::test_data_type_new_json_with_nested_hint_types ... FAILED
called `Result::unwrap()` on an `Err` value: TypeParsingError("Unknown data type: Strin")

With the fix, cargo test -p clickhouse-types passes (44 + 1 doctest). cargo fmt -- --check and cargo clippy -p clickhouse-types --all-targets are clean with RUSTFLAGS=-Dwarnings.

Checklist

  • Unit tests covering the common scenarios were added
  • A human-readable description of the changes was provided so that we can include it in CHANGELOG later

Suggested CHANGELOG entry (Fixed): JSON columns with typed paths whose type has parentheses, e.g. JSON(a Nullable(String)) or JSON(a Map(String, Int32)), no longer fail to parse. ([#442])

@CLAassistant

CLAassistant commented Sep 26, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

remove_json_header() trimmed every trailing `)`, so `JSON(a Nullable(String))`
lost the closing paren of the hint type and failed with
`Unknown data type: Strin`. The hints were also split on commas and `=`
without regard for parentheses or string literals, which broke hint types
such as `Map(String, Int32)` or `Enum8('x' = 1)` and `SKIP REGEXP` patterns
containing a comma.

Strip exactly one closing `)` and split only on separators outside
parentheses, back-quoted names and string literals.

Fixes ClickHouse#442
@abonander
abonander force-pushed the json-type-hints-nested-parens branch from df1cebd to e8581d5 Compare October 6, 2026 04:59
@abonander
abonander merged commit 687eb37 into ClickHouse:main Oct 6, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JSON type hint with Nullable fields misparsed

3 participants