Skip to content

Accept internationalized labels in the idn-hostname format - #242

Open
bardrop wants to merge 2 commits into
horejsek:masterfrom
bardrop-oss:fix-idn-hostname-unicode
Open

bardrop wants to merge 2 commits into
horejsek:masterfrom
bardrop-oss:fix-idn-hostname-unicode

Conversation

@bardrop

@bardrop bardrop commented Nov 20, 2026 •

Copy link
Copy Markdown

The idn-hostname format (draft 07 and 2019-09) rejects most internationalized host names, which is what the format is for:

v = fastjsonschema.compile({'$schema': 'http://json-schema.org/draft-07/schema#', 'format': 'idn-hostname'})
v('münchen.de')         # JsonSchemaValueException: data must be idn-hostname
v('실례.테스트')          # rejected
v('localhost')          # rejected (one label)
v('a.b.c.example.com')  # rejected (more than two labels)
v('ex_ample.com')       # accepted
v('example.com\n')      # accepted (`$` instead of `\Z`)

The regex only allowed ASCII letters and digits, needed exactly two or three labels, allowed _, and ended with $.

The fix uses the same shape as the hostname regex (labels of 1-63 chars, no - at the start or end, joined by dots, \Z at the end), but a letter or digit can be any Unicode letter or digit ([^\W_]), and combining marks of common scripts (Latin diacritics, Hebrew, Arabic, Indic, Thai, ...) are allowed after the first character, so हिन्दी.भारत works. Punctuation, symbols and spaces stay out, also the fullwidth ones: evil.com, example.com/path and user@example.com are rejected, since they turn into ., / and @ after NFKC. It stays a "good enough" regex like the other formats: IDNA2008 context rules (ZWJ, middle dot, bidi, ...) are not checked, so that suite file stays ignored.

Against the latest JSON-Schema-Test-Suite format-idn-hostname.json (draft 7), valid names that are rejected go from 36 to 18 (the rest need context-rule characters such as U+00B7 or U+30FB). Invalid names that are accepted go from 1 to 14; those are context-rule cases, which passed before only because every non-ASCII name was rejected.

Tests: test_idn_hostname in tests/test_format.py (fails on master), plus a long-input timing case. make test: 5588 passed. pylint score unchanged. Added a CHANGELOG line.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant