Skip to content

Align Lua parser diagnostics and validate missing closers - #351

Merged
akeit0 merged 3 commits into
mainfrom
codex/issue113-token-diagnostics
Oct 10, 2026
Merged

akeit0 merged 3 commits into
mainfrom
codex/issue113-token-diagnostics

Conversation

@akeit0

@akeit0 akeit0 commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Malformed Lua diagnostics exposed enum names in the public AST parser, and the execution compiler narrowed reserved token IDs to char, producing messages such as ą expected. Both now use Lua keyword/punctuation spelling and token categories. For example, if true print() end reports 'then' expected near 'print', and local = 1 reports <name> expected near '='.

The public LuaSyntaxTree.Parse parser now reports the opening construct and its line when a closing token is missing on a later line: 'end' expected (to close 'do' at line 1) near <eof>. It rejects missing block/table closers that it previously accepted and handles valid multiline grouped expressions and table keys. EOF diagnostics use the lexer end position, including trailing newlines and comments, rather than the retained previous token.

The execution compiler already tracked opening constructs. Preserve its full int token IDs, quote expected punctuation, and use categories for expected names/numbers/strings. This fixes its diagnostics without changing valid execution behavior.

Follow the official Lua 5.2 lexer token2str/txtToken conventions and parser check_match behavior. The fixed syntax/category distinction also applies in 5.3 and 5.4. Public signatures remain compatible. AST string context still uses existing token display text, so this does not promise byte-for-byte diagnostic parity for original quote styles or long-bracket delimiter levels.

Fixes #113.

Validation:

  • 71 new regression cases; ParserTests: 72 passed. Missing-token and missing-closer cases check both the execution compiler and public AST parser against explicit expected diagnostics. Also covers nested blocks, EOF/comment positions, valid multiline constructs, Turkish-culture category names, and preserved literal display.
  • Runtime tests: 397 passed. Source generator tests: 24 passed.
  • Release builds for netstandard2.1/net6.0/net8.0/net10.0: zero warnings/errors.
  • CSharpier checks all 247 files; git diff check passes. Temporary verification files remain outside git.

@akeit0 akeit0 changed the title Use Lua token spelling in parse error messages Align parse error token names and context with Lua Oct 10, 2026
@akeit0 akeit0 changed the title Align parse error token names and context with Lua Align Lua parser diagnostics and validate missing closers Oct 10, 2026
@akeit0
akeit0 merged commit c1e8a69 into main Oct 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LuaParseException prints tokens in wrong case (ie 'Then' rather than 'then')

1 participant