[Hrushikesh-ramilla]

Overview

Fixes #29495.

In src/llama-grammar.cpp, llama_grammar_parser::parse_alternates and parse_sequence were mutually recursive when parsing nested groupings ( with no recursion depth bound. A client-supplied grammar payload containing deeply nested parenthesized groups or nested alternations (e.g. root ::= (((... "x" ...)))) caused unbounded call-stack consumption, leading to a stack overflow and terminating llama-server via SIGSEGV (Denial of Service).

Changes:

1. Introduced MAX_GRAMMAR_RECURSION_DEPTH 256 in src/llama-grammar.cpp.
2. Added depth tracking across parse_alternates and parse_sequence in src/llama-grammar.h and src/llama-grammar.cpp (defaulting to 0 for complete backward compatibility).
3. Enforced a recursion depth check when encountering grouping (: throws std::runtime_error when depth exceeds MAX_GRAMMAR_RECURSION_DEPTH, enabling llama_grammar_parser::parse() to catch the exception, clear rules, and return false cleanly instead of crashing the process.
4. Added a regression test in tests/test-grammar-parser.cpp verifying that deeply-nested groupings (> 256) are safely rejected.

Additional information

• Tested against test-grammar-parser: all 20+ grammar parser test cases pass without regressions.
• The depth limit (256) is generous for valid real-world GBNF grammars (JSON schema conversions typically stay below 30-50 levels of nesting) while preventing stack exhaustion.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES
  • Used Antigravity AI agent to assist in diagnosing issue bug: the GBNF grammar-text parser recurses without a depth limit, can cause stack-overflows and crash server (SIGSEGV) #29495, drafting the recursion depth tracking diff, and executing the local test suite.

[ServeurpersoCom]

LGTM, tested on Linux against llama-server: a grammar with 50,000 nested groups segfaults on master and returns a 400 with this PR, 300 nested groups are rejected cleanly, and long chains of named rules still work. Bounding the recursion where it happens is the pattern ngxson set in #30242.