Repository navigation
fix(types): parse back-quoted JSON path names - #458
Merged
abonander merged 1 commit intoAug 25, 2026
Merged
Conversation
teddytennant
requested review from
abonander,
joe-clickhouse,
kavirajk and
peter-leonov-ch
as code owners
August 8, 2026 20:01
ClickHouse back-quotes a JSON typed path whose name contains a character that requires quoting, and reports the column type accordingly, e.g. JSON(`a b` Int64) or JSON(`a,b` Int64). parse_json() split the hint list on every comma and every space, without looking at those quotes, so such a type was torn apart: the comma case produced a fragment with no space in it and panicked on map[1] with an index out of bounds, and the space case produced the key `a and the type b`, failing with "Unknown data type: b`". Both happen while parsing the columns header, so the whole query fails. Split the hint list, and each hint into a path and a type, only on separators that are outside a back-quoted span, and return a parsing error instead of indexing blindly when a hint has no type. Path names are kept exactly as the server spelled them, including the quotes, so that the type is serialized back unchanged in an INSERT header.
Contributor
Author
|
CLA is signed and this is ready for review whenever someone has a moment. |
abonander
approved these changes
Aug 25, 2026
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #453.
ClickHouse back-quotes a
JSONtyped path when its name contains a character that requires quoting, and reports the column type that way in theRowBinaryWithNamesAndTypesheader:parse_json()split the hint list on every comma and each hint on every space, with no awareness of those quotes:JSON(`a,b` Int64)was split into`aandb` Int64. The first fragment contains no space, somaphad a single element andmap[1]panicked withindex out of bounds: the len is 1 but the index is 1— a panic in library code rather than a returnedErr.JSON(`a b` Int64)yielded the key`aand the typeb`, failing with `Unknown data type: b``.Both happen while parsing the columns header, so the whole query fails — reading or inserting into any table with such a column is impossible, and there is no way to opt out.
The fix
Splitting is now done on separators that are outside a back-quoted span, both for the hint list and for the path/type boundary inside a hint (a back-quote escaped with a backslash does not end the span, matching how the server escapes it). The same applies to the
=check that tells a JSON setting such asmax_dynamic_types=8apart from a path hint, so a path named`b=c`is no longer silently dropped. Where the old code indexedmap[1]blindly, a hint with no type now returns aTypeParsingError.This is the right layer for it: the quoting is part of the type as the server spells it, and
parse_json()is the only place that takes it apart.Path names are deliberately kept exactly as the server wrote them, quotes included, rather than being unquoted.
DataTypeNode'sDisplayimpl is used byput_rbwnat_columns_header()to send the type back on INSERT, so keeping the name verbatim makes that round-trip byte-identical; unquoting the name would silently produce an invalidJSON(a b Int64)header unlessDisplaylearned to re-quote and re-escape. That is a separate change, so I left it out here — the unit test pins the round-trip so the two sides cannot drift apart.Not addressed, and independent of this: #442, where
remove_json_header()trims all trailing)and so mangles a nested type argument likeNullable(String). A fix for that would not have helped the back-quoted cases and vice versa.Verification
Unit test, on
mainbefore the fix:The integration test, against a real server (26.7.3.19), reproduces the panic itself:
Both pass with the fix, and the rest of the suite is unaffected:
cargo fmt -- --checkandcargo clippy --all-targets --no-default-featuresare clean withRUSTFLAGS=-Dwarnings.Checklist
Suggested CHANGELOG entry (Fixed):
JSONcolumns whose typed paths are back-quoted, e.g.JSON(`a b` Int64)orJSON(`a,b` Int64), no longer fail to parse; the comma case previously panicked with an index out of bounds. ([#453])