[Engine] Opt-in LaTeX-safe JSON recovery for json_object responses (LightRAG VLM) #2893

Summary

When a vision-language model generates JSON containing an unescaped LaTeX
backslash, Strata's response_format: {"type":"json_object"} validation
rejects the completion with HTTP 502 (structured_output_failed).

This breaks LightRAG's multimodal PDF ingestion because LightRAG already
implements tolerant JSON parsing and LaTeX escape recovery, but Strata
rejects the response before LightRAG can access the generated text.

The issue has been reproduced using the same image and prompt.

Environment

• Strata v0.1.44.1 (Strata-NVFP4 fork)
• Qwen3.8-Flash-Next NVFP4
• GPU vision enabled
• LightRAG v1.5.7
• MinerU PDF pipeline
• OpenAI-compatible Chat Completions API

Minimal Reproduction

Model-generated content:

{"name":"alpha coefficient","type":"diagram","description":"Coefficient \alpha_0"}

The sequence \a is not a valid JSON escape.

This can also be reproduced directly against Strata's validator:

from serve.structured import _ObjectOnly, validated_json

raw = r'{"name":"alpha coefficient","type":"diagram","description":"Coefficient \alpha_0"}'

print(validated_json(raw, _ObjectOnly(), "stop"))

Actual Behavior

• Strict JSON parsing fails.
• Strata returns HTTP 502 (structured_output_failed).
• The original generated response is unavailable to LightRAG.
• LightRAG retries the failed provider request.
• The document eventually enters the FAILED state.

Expected Behavior

An optional recovery mode could repair unambiguous LaTeX escaping errors
before strict validation.

For example:

{"name":"alpha coefficient","type":"diagram","description":"Coefficient \\alpha_0"}

The repaired output should still pass strict JSON validation before
being returned to the client.

Proposed Solution

Introduce an opt-in, conservative JSON recovery mechanism for
response_format=json_object.

Suggested behavior:

1. Preserve existing strict validation by default.
2. Attempt limited LaTeX-aware repair when explicitly enabled.
3. Validate the repaired JSON using the existing validator.
4. Return canonical JSON only after successful validation.
5. Continue returning HTTP 502 for unrecoverable output.
6. Preserve existing json_schema behavior.

The implementation must also account for LaTeX commands such as:

• \alpha
• \beta
• \frac
• \tau
• \nabla

Some commands conflict with valid JSON escape sequences, so simply
doubling every backslash is unsafe.

Regression tests should cover legitimate JSON escapes, already-correct
LaTeX escaping, duplicate keys, invalid constants, and truncated output.

Related Issues and PRs

• Strata [Feature Request] Tracking Issue for Grammar-Constrained Sampling #2368 — Grammar-Constrained Sampling
• Strata serve + engine: token masks for JSON formats that keep MTP drafts under the mask #1328 — Token Masks for JSON Formats with MTP
• LightRAG PR [Engine] Linux: "exit code -9" with no OOM when started from a PipeWire desktop without rtkit (RLIMIT_RTTIME hard limit 0) #3238 — Repair LaTeX Escape Damage in LLM JSON Responses
  fix: repair LaTeX escape damage in LLM JSON responses (multimodal analysis & entity extraction) HKUDS/LightRAG#3238
• LightRAG PR [Engine] build: pin CMP0169 OLD around FetchContent_Populate (CMake 4.4 warns on every configure) #3242 — Retry Malformed Multimodal Analysis JSON
  fix(pipeline): retry malformed multimodal analysis JSON HKUDS/LightRAG#3242

Constrained decoding may prevent syntactically invalid JSON, but it does
not necessarily preserve LaTeX semantics when a model emits JSON-valid
escape sequences.

This feature would improve compatibility with LightRAG and other
applications processing mathematical or technical documents.

Retest (2026-10-21)

Retested on Strata 0.2.0-beta (source, tag v0.2.0-beta) and on the current release (Strata 0.2.1.4
source build, engine strata 0.1.46.3). Still reproduces on both; serve/structured.py is unchanged
since v0.1.44.1.

• GPU: RTX 5090 32 GB, driver 615.71.10. Memory: 121 GiB. System: Ubuntu 26.04.1, kernel 7.0.0-38.
• What I see: with response_format: {"type":"json_object"}, a completion containing
  {"description":"$\alpha_0 > 1$"} gives HTTP 502 structured_output_failed (validator:
  Invalid \escape), and the text never reaches the client. Reproduced on both with the built-in mock
  engine (--engine mock --script), which is the same validation path. Not reproduced with a real
  model: my runs escaped the backslashes correctly, so the rejection comes from the validator, not the
  model.
• The 502 says nothing about what the model wrote, so the caller cannot recover the text itself.