[voltcrash]

Repeated unclosed TeX math openers make typing and Markdown rendering rescan the remaining text from each opener. This is a small, focused fix for an obvious performance bug in the existing tokenizer; it changes no math syntax or product behavior.

Cache only failed scans whose end proves there is no closer: text scans stop at their existing EOF/backtick/single-line boundary, and flow scans cache only EOF failures. Keep text caches separate by delimiter and single-line mode, sharing the owning event array across GFM's speculative contexts. Weak keys limit their lifetime to parsing. Content-dependent citation failures and lazy flow boundaries remain uncached.

Proof on upstream main d2c25950b4d9183ea342d5d5fc14c326b9e0172a and this fix:

• The focused operation-count regression fails without the fix and passes with it, without elapsed-time assertions. With GFM, 200 repeated inline openers (1,000 characters) consumed 201,001 characters before and 2,399 after; 400 openers consumed 802,001 before and 4,799 after. The same test covers inline bracket openers and bare display-opener lines.
• Coverage preserves escaping, code/paragraph/container/table boundaries, mixed delimiters, citations, and later valid closers. 93 scoped math/composer tests passed. Separately, 20,000 seeded mixed-delimiter documents produced identical ASTs and offsets; the reviewer independently checked another 30,000.
• vp run check:changed passed: unused patches, formatting, lint, both knip checks, affected-package typechecks, and focused tests (61).
• Integrated web client: pasted 5 KB of repeated inline openers and 700 bare display-opener lines, then edited both; source and edits were retained. Formula source after failed openers remained editable. Used the built-in Browser panel first, then the installed Chromium/Playwright after repeated preview recording timeouts. 11-second recording, inline capture, display capture, operation counts. The free replay turn completed, but its history stayed loading in this dev client, so rendered-message verification is not claimed. State was a worktree-local copy; no writes to the live database and no paid turns.

Duplicate check: inspected current upstream source and open PRs #30658 (single-dollar composer precheck), #32110 (forge math dialect), and #27866 (web rendering parser replacement). None fixes the shared composer tokenizer's repeated failed TeX scans; #27866 leaves that shared parser in place. Keyword searches for math, TeX, micromark, unclosed delimiters, and quadratic parsing found no exact duplicate. These were targeted searches, not an exhaustive review of every open PR.

Scope: the patched dependency serves web/desktop rendering and shared composer parsing, including mobile's composer callers. No provider, protocol, permission, or connection-mode changes. Native mobile, Electron shell, relay/tunnel, and production-build performance were not exercised. Overall Markdown parsing still has other costs; this bounds the failed TeX scan work. The scenario guide explicitly excludes performance proof, so scenarios:prove was not used to substitute a timing threshold for the operation-count regression.

Reviewed with review-loop: 1 round with gpt-6.1-sol on xhigh. Reviewer verdict: approved, no findings.

Created with OpenAI gpt-6.1-sol (xhigh) in the Codex harness through T3 Code.

[macroscopeapp]

Approvability

Verdict: Approved at b656698

Macroscope's review found this PR approvable — The patch avoids repeated scans of unclosed TeX math in Markdown parsing. Its caches are tokenizer-local, with no changes to product defaults, stored content, or external systems.

^{You can add or adjust custom eligibility rules. Learn more.}

[coderabbitai]

No actionable comments were generated in the recent review.

Walkthrough

The tokenizer patch changes math delimiter recognition, failed-scan handling, and the package default export target. New tests measure consumption for repeated unclosed delimiters and check that later valid formulas still parse.

Changes: Math tokenizer scan handling

Layer / File(s)	Summary
TeX flow closure patches/micromark-extension-llm-math@3.1.0.patch	The TeX flow tokenizer checks opening metadata and requires a closing fence. It records an EOF failure offset for an unclosed flow.
Text scan recovery and validation patches/micromark-extension-llm-math@3.1.0.patch, packages/shared/src/markdownMath.test.ts	Text scans cache failed opener positions and apply delimiter boundary rules. The patch also changes inline delimiter handling and the default export target. Tests bound consumption for repeated unclosed delimiters and check that later valid formulas still parse.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Possibly related issues

• [Bug]: Typing after a selected skill lags, worse on long prompts #30657 — The issue reports latency from repeated Markdown and math parsing after an unclosed opener. This PR adds failed-scan caching for the tokenizer path.

Possibly related PRs

• pingdotgg/t3code#29739 — That PR uses the shared math parser for mobile TeX delimiter conversion, and this PR changes scan behavior in that parser.

Suggested labels: bugfix, size:M

Suggested reviewers: yash-singh1, macroscopeapp, nikitamgrimm

Merge Risk: Minimal · up to b6566

The change stops repeated rescans of unclosed TeX math without altering math syntax. No actionable merge-blocking risk was found in the reviewed changes.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title is concise, uses the required conventional format, and clearly identifies the primary change: stopping repeated scans of unclosed TeX math.
Description check		The description covers the problem, implementation approach, scope justification, focused verification, observed results, limitations, and agent attribution. It is complete and relevant to the templat…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description covers the problem, implementation approach, scope justification, focused verification, observed results, limitations, and agent attribution. It is complete and relevant to the template.

Finishing Touches

Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Recent review info

Run configuration

• Source mode: Auto
• Repository: pingdotgg/t3code
• Branch: main
• File: .coderabbit.config.ts
• Commit: d2c2595
• Layers: Repository TS config
• Review profile: CHILL
• Plan: Advanced
• Run ID: 0cbd12f7-edc6-4b30-ad7c-0c061cd97c13

Commits

Reviewing files that changed from the base of the PR and between d2c2595 and b656698.

Files ignored due to path filters: 1

• pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

Files selected for processing: 2

• packages/shared/src/markdownMath.test.ts
• patches/micromark-extension-llm-math@3.1.0.patch

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

[maria-rcks]

Note

Written by claude-opus-5-5 on behalf of Maria

Verified in a real dev environment

Ran vp run dev on github/main (0629735d) and then on this PR head (b6566980), one server at a time, on a Linux VM. The data was the same in both runs. I added a local git repo as a project through the Add project dialog, then ran a real Claude claude-haiku-5-5 turn that streamed a message with thousands of unclosed \( a openers. The main worktree's .t3 state was copied into the PR worktree, so both runs saw identical threads. I drove the web client with Playwright and Chrome.

Composer (the path this PR speeds up). I typed check plus a @README.md mention, then inserted N copies of \( a , then typed one key at a time. The number below is the time from each keypress to the next painted frame.

	main	PR
1000 openers, per key	5206 to 5354 ms	224 to 288 ms
1500 openers, per key	9490 to 17161 ms	482 to 557 ms
1500 openers, inserting the text	8993 ms	483 ms
1500 openers, longest main-thread task	10562 ms	531 ms

Typed text was kept in both versions (composer ends with \( a typing).

Recordings, trimmed to start once the page is loaded:

Shared parser, same code the app imports (markdownMathRanges, run with node from packages/shared):

input	main	PR
10,000 chars of \( a	1497 ms	141 ms
20,000 chars of \( a	5683 ms	494 ms
6,000 chars of \[+newline	1738 ms	61 ms
12,000 chars of \[+newline	7953 ms	131 ms

Neighboring behavior on the PR:

• I turned on Settings > Appearance > math rendering, then ran a second Haiku turn in the same thread. Inline \(E = mc^2\), $a+b$ and a \[ \int_0^1 x\,dx = \tfrac12 \] display block all rendered with KaTeX, in both the user and assistant messages.
• With math on, opening the thread that holds the ~255 KB assistant message took 2054 ms on the PR. With math off it took 1.7 to 2.0 s on both versions.
• In \( a \( b then closed \(x^2\), the first \( pairs with the first \). That output is identical on main and on the PR, so it is existing behavior.
• collectComposerInlineTokens returns identical mention and formula masking on main and the PR for mentions inside, beside and after unclosed openers.

CI on b6566980 is green.

Not verified

• I did not measure chat rendering with math turned on against main. It is off by default, and the main server was already stopped when I found the setting.
• Native mobile composer (iOS and Android). It uses the same shared parser, but there is no simulator on this Linux VM.
• Electron desktop shell and production builds. Only the Vite dev web client was exercised.
• The first Haiku turn (asked to repeat \( a 1500 times) streamed its text but then kept "Thinking" for more than 14 minutes, so I stopped it. This looks like model behavior, not this PR. It happened on main, before the PR was running.