json parser: \u escape sequences are not supported #671

Description

The parser rejects any string containing a \uXXXX escape sequence and fails the whole document with a parse error. These escapes are part of the JSON string grammar and are emitted by most JSON producers.

The library's own writer emits them too: control characters are escaped as \u00XX, as required. As a result, output produced by JsonWriter cannot be read back by JsonParser.

Analysis results

Unescaping handles the two-character escapes (\n, \t, \\, …) and returns an error for u. There is no decoding of the four hexadecimal digits and no handling of surrogate pairs.

Two consequences:

• Documents from any other producer fail to parse whenever a string uses \u — common for non-ASCII text and for ASCII-safe emitters that escape everything above U+007F.
• The write-then-read round trip through this library is broken for any string containing a control character.

Solution

Decode \uXXXX during string parsing: convert the code point to UTF-8, and combine surrogate pairs into a single code point. Reject lone surrogates and malformed sequences as parse errors.

Error Occurrence Rate

Reproducible

How to reproduce

Parse a document containing a \u escape:

score::json::JsonParser{}.FromBuffer(R"({"value":"\u00e4"})");

Actual: parse error.
Expected: a string value of ä.

Round trip through the library:

score::json::Object o;
o["value"] = score::json::Any{std::string{"a\x01z"}};
const auto buffer = score::json::JsonWriter{}.ToBuffer(o).value();
score::json::JsonParser{}.FromBuffer(buffer);

Actual: the writer produces "a\u0001z", which the parser then rejects.
Expected: the document parses back to the original value.

Supporting Information

• The writer side is already correct per RFC 8259 §7; only the parser side is missing.
• The now deprecated nlohmann parser is escaping control characters correctly, so it can be used as a reference.

Classification

Minor

First Affected Release

pre-0.5

Last Affected Release

not released (main)

Expected Fixed Release

before release (main)

Category

• Safety Relevant
• Security Relevant