Skip to content

A line comment may end the input (#1039) - #1049

Merged
Rafael-SOWNet merged 2 commits into
masterfrom
fix/line-comment-at-end-of-input
Aug 24, 2026
Merged

Rafael-SOWNet merged 2 commits into
masterfrom
fix/line-comment-at-end-of-input

Conversation

@Rafael-SOWNet

Copy link
Copy Markdown
Member

Fixes #1039.

COMMENT in AngouriMath.g was

COMMENT: ('//' ~[\r\n]* '\r'? '\n' | '/*' .*? '*/') -> skip ;

so the // alternative only matched when a newline followed. A comment on the last line never
matched at all and its first / reached the parser as an operator. The end of the input is where a
comment is most likely to be written — the last line of a file, or a single-line string handed to
FromString — and the block form carries no such condition, so this reads as a missing optional
rather than a decision.

COMMENT: ('//' ~[\r\n]* ('\r'? '\n')? | '/*' .*? '*/') -> skip ;

Measured on a build of this branch:

input before now
x + 1 // done UnhandledParseException: extraneous input '/' x + 1
x + 1 // same x + 1
x + 1 // a // b same x + 1
x + 1 // done\n x + 1 x + 1
x + 1 // done\r\n x + 1 x + 1
x + 1 /* done */ x + 1 x + 1
x + // c\n 1 x + 1 x + 1
x / 1 x / 1 x / 1

// nothing on its own is still a parse error — the comment is skipped and an empty input is not an
expression, which is the same reason "" is an error.

Regeneration

Per Docs/Contributing/ImproveParser.md: the vendored antlr-4.13.1-complete.jar, then the
AntlrPostProcessorReplacePublicWithInternal step that makes the generated classes internal. Worth
noting for review — only the lexer changes. Regeneration first rewrites AngouriMathListener.cs,
AngouriMathBaseListener.cs and AngouriMathParser.cs with public in place of internal, and the
post-processor puts them back byte-identical, so the diff is the grammar, AngouriMathLexer.cs and
AngouriMathLexer.interp and nothing else.

One behaviour change beyond the fix

"//" on its own is still an error, and its reported position moves from line 1:0 to line 1:2.
That is the convention the neighbouring FromStringTest rows already follow — " " is line 1:2,
"/**/" is line 1:4, both pointing at the end of the skipped text — so // stops being the one
row that pointed at a character the lexer was meant to have consumed. The row is updated with a
comment saying why.

Tests

SyntaxDocumentedTest.ALineCommentNeedsItsNewline pinned the old behaviour, with #1039 linked
against it. It becomes ACommentMayEndTheInput over six spellings, plus
ACommentIsNotAnExpressionOnItsOwn. Docs/Usage/Syntax.md said the newline was required and now
says it is not, keeping the issue reference for anyone reading an older version.

Suite: 8107 passed, 0 failed, 14 skipped.

Part of #746 item 8 — this was found while deriving what the grammar accepts.

Rafael-SOWNet and others added 2 commits August 24, 2026 18:49
`COMMENT`'s `//` alternative required a newline after it, so a comment on the
last line never matched and its first `/` reached the parser as an operator:
`x + 1 // done` was `extraneous input '/'` while `x + 1 /* done */` was `x + 1`.
The end of the input is where a comment is most likely to be written -- the last
line of a file, or a single-line string handed to `FromString` -- and the block
form carried no such condition, so this was a missing optional rather than a
decision. The newline is now optional.

Regenerated with the vendored ANTLR 4.13.1 and the post-processor that makes the
generated classes internal, per `Docs/Contributing/ImproveParser.md`. Only the
lexer changes: the listener and parser diffs were the accessibility churn alone
and come back byte-identical.

`"//"` on its own is still an error, and its reported position moves from
`line 1:0` to `line 1:2`. That is the convention the neighbouring rows already
follow -- `"  "` is `line 1:2` and `"/**/"` is `line 1:4`, both pointing at the
end of the skipped text -- so it stops being the one row that pointed at a
character the lexer was supposed to have consumed.

`SyntaxDocumentedTest` pinned the old behaviour with the issue against it; it now
pins the fix across five spellings, plus a case for a comment being the whole
input, which is a parse error because nothing is left rather than because the
comment failed to match.

Part of #746 item 8.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bjumi5K7fg8yx6UK1mZTQd
…-end-of-input

# Conflicts:
#	BREAKING-CHANGES.md
@Rafael-SOWNet
Rafael-SOWNet merged commit 5cf7973 into master Aug 24, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A // comment that ends the input is a parse error, because the lexer rule requires the newline

1 participant