Keep NaturalStringComparer transitive across digit scripts - #57
Merged
matt-edmondson merged 2 commits intoSep 27, 2026
Merged
Conversation
A digit chunk compared with a text chunk used its raw code points, so a non-ASCII digit such as the Arabic-Indic five sorted after letters while the ASCII five it equals sorted before them. Sort output then depended on input order. Digit chunks are now held as ASCII digits, so a number against text orders the same way whatever script spells it, and ASCII input orders exactly as before. Chunking also walks code points instead of UTF-16 units, so digits outside the BMP, such as the mathematical digits, form numbers as the remarks say. Fixes #55 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SZUfnvhMFzj96bfFNzbfpn
Brings its cognitive complexity under Sonar's S3776 limit without changing the chunks it produces. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SZUfnvhMFzj96bfFNzbfpn
|
This was referenced Sep 27, 2026
matt-edmondson
deleted the
claude/sorting-55-transitive-unicode-digits
branch
September 27, 2026 15:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #55
What changed
Mixed digit/text chunks. When a digit chunk met a text chunk, the comparer compared raw code points. That let
٥(U+0665) sort after letters while5, which it equals, sorts before them. The result was a cycle (٥ < 10 < z < ٥), and sort output depended on input order. Digit chunks are now stored as ASCII digits of the same value. A text chunk never starts with a digit, so a number compared with text is decided by its first character, the same way for every script.I chose this over the issue's "numeric always before text" suggestion because it leaves ASCII ordering exactly as it is today. For example,
"-5"-style punctuation still sorts before numbers and letters still sort after them. The only change is where non-ASCII digits land.Chunking by code point. The
\d+|\D+regex is gone. It worked on UTF-16 code units, and those treat surrogate halves as non-digits. In its place is a small splitter that walks code points usingCharUnicodeInfo.GetUnicodeCategory(string, int)andGetDecimalDigitValue(string, int), which works back to netstandard2.0. Digits outside the BMP, such as the mathematical digits, now form numbers:"file𝟗" < "file𝟏𝟎".Tests
New tests:
Compare_NonAsciiDigitAgainstText_OrdersLikeTheAsciiDigitCompare_SortOfMixedScriptDigitsAndText_DoesNotDependOnInputOrderCompare_IsTransitiveOverMixedScriptDigitsAndText, a check of antisymmetry and transitivity over every triple from a mixed set of ASCII, Arabic-Indic, Devanagari and mathematical digits, letters, punctuation and emojiCompare_DigitsOutsideTheBasicMultilingualPlane_ComparedByNumericValueAll four fail with the fix stashed and pass with it. Full suite: 19/19 locally. The library builds clean for net10.0, net9.0, net8.0, netstandard2.1 and netstandard2.0.
🤖 Generated with Claude Code
https://claude.ai/code/session_01SZUfnvhMFzj96bfFNzbfpn
Generated by Claude Code