server / tokenizer: SIGSEGV (stack overflow) in unicode_regex_split_stl — the o200k / GPT4O pre-tokenizer regex recurses once per character, so a ~26,000-character unbroken run of one character class kills the process #27783

Name and Version

$ llama-server --version
version: 0.1.0-dev (build 1, commit 4df29be)
built with GNU 14.2.0 for Linux x86_64

Source at 4df29be4f4c3673f428170fda944a5b19f743bb8, CMAKE_BUILD_TYPE=Release
(CMAKE_CXX_FLAGS_RELEASE=-O3 -DNDEBUG), g++ 14.2.0.

The llama-server that produced the coredump is a Vulkan build; the llama-tokenize used for every
threshold below is a HIP/ROCm build of the same commit with the same flags. The pre-tokenizer path is
CPU-only and identical in both, and both report the same version string.

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

libllama (core), llama-server, llama-tokenize

Command line

llama-server \
  --model gpt-oss-120b-MXFP4.gguf \
  --n-gpu-layers 99 --ctx-size 131072 --parallel 2 --cont-batching \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --batch-size 2048 --ubatch-size 2048 --load-mode none \
  --host 127.0.0.1 --port 8081 --metrics --no-ui --no-slots

None of these flags is load-bearing. The crash is in pre-tokenization, before anything
backend-, batch- or KV-related is reached, and llama-tokenize reproduces it with no GPU at all.

Problem description

A single POST to /v1/chat/completions whose prompt contains a long unbroken run of one character
class terminates the server process with SIGSEGV, during tokenization, before any inference
happens. It takes ~26,000 characters at the default 8 MiB stack. There is no error, no 4xx and no
log line — the process is simply gone, taking every slot's KV cache and prompt cache with it.

It requires no tools, no grammar, no streaming and no concurrency.

What matters is the longest unbroken run that one alternative of the pre-tokenizer expression can
consume, not the prompt length. libstdc++'s executor recurses once per repetition, and every
quantified alternative in this expression is a driver — not only the letter ones. Measured against
the shipped llama-tokenize at an 8 MiB stack, exit 139 = SIGSEGV:

Reading the alternation, three kinds of run recurse:

• letters of a single case — ((?=[\p{L}])([^A-Z]))+ drives lowercase runs and
  ((?=[\p{L}])([^a-z]))+ drives uppercase runs. A case change breaks both, which is why
  'Xx' * 18,000 survives while 'x' * 26,500 and 'X' * 26,250 each die. Non-ASCII letters sit in
  both classes.
• non-letter, non-digit, non-space characters — ?[^\s\p{L}\p{N}]+[\r\n/]*, so a long run of a
  single punctuation or symbol character is its own driver.
• whitespace — \s*[\r\n]+, \s+(?!\S), \s+.

Digits are the exception: \p{N}{1,3} is bounded, so 66,000 digits tokenize fine.

So a space or a full stop ends a letter run but opens a run of its own. What keeps ordinary text
safe is that it alternates short runs of different classes — prose interleaves letter runs of a
dozen characters with one-character space runs, and neither approaches the threshold. Measured:
66,000 characters of English prose and 51,000 codepoints of Japanese with 。 and 、 both
tokenize normally.

Non-ASCII does not lower the codepoint threshold. unicode_regex_split collapses each non-ASCII
codepoint to a single category byte, so one CJK codepoint costs the same stack as one ASCII letter —
'日' brackets at 26,000 / 26,250, identical to 'X'. What differs for scripts without inter-word
spaces is that long unbroken letter runs arise there naturally, where in English they have to be
constructed.

It is reachable by an untrusted caller without anything exotic: a ~26 KB body of one repeated
character — a letter, a space, a full stop — kills the process, repeatedly. set_payload_max_length
is never called anywhere in the tree outside its own definition in the vendored cpp-httplib, so
there is no size cap in front of it, and tokenization happens before the context-length check
(tokenize_input_prompts at tools/server/server-context.cpp:4198, inside
handle_completions_impl; the "exceeds the available context size" check is at :3115, after the
task is queued), so --ctx-size does not bound it either.

We have not surveyed which real-world content reaches ~26,000 characters of one class. The cases we
can name are programmatically whitespace-stripped text, a pathologically long single-case identifier
or concatenated token stream, and long runs of padding, separator or indentation characters.

Root cause

gpt-oss resolves to LLAMA_VOCAB_PRE_TYPE_GPT4O (o200k_harmony, case label at
src/llama-vocab.cpp:428). Its pre-tokenizer expression is src/llama-vocab.cpp:433:

[^\r\n\p{L}\p{N}]?((?=[\p{L}])([^a-z]))*((?=[\p{L}])([^A-Z]))+(?:'[sS]|'[tT]|…)?|
[^\r\n\p{L}\p{N}]?((?=[\p{L}])([^a-z]))+((?=[\p{L}])([^A-Z]))*(?:'[sS]|'[tT]|…)?|
\p{N}{1,3}| ?[^\s\p{L}\p{N}]+[\r\n/]*|\s*[\r\n]+|\s+(?!\S)|\s+

That string is not in unicode_regex_split_custom()'s list of hand-written fast paths
(src/unicode.cpp:1050), so every tokenize call for this vocab falls through to
unicode_regex_split_stl() (src/unicode.cpp:739, reached from :1367) and therefore to
libstdc++'s std::regex, whose executor is a recursive backtracking DFS.

The worst driver is:

((?=[\p{L}])([^A-Z]))+

a lookahead inside a greedy repetition. libstdc++ recurses once per repetition
(_M_dfs → _M_rep_once_more → _M_dfs, with _M_lookahead allocating on each step), so stack
depth is proportional to the length of the run being consumed. After the \p{...} →
collapsed-class substitution the lookahead class is [\xD2A-Za-z]; \xD2 is the marker every
non-ASCII letter collapses to (k_ucat_cpt, src/unicode.cpp:1231), which is why a CJK codepoint
costs exactly the same stack as an ASCII letter.

The lookahead is not required for the recursion — it only makes each frame bigger. The plain
repetitions ?[^\s\p{L}\p{N}]+ and \s+ recurse as well, which is what the '.' and ' ' rows
above are showing.

Depth measured from a real core: the faulting thread was 78,124 frames deep on an 8 MiB stack,
i.e. ~107 bytes per frame. Combining that with the threshold table below (3,250 characters per
MiB, so ~323 bytes of stack per input character) gives ~3 frames per input character. The core came
from a request containing a 36,000-character run, but the stack was exhausted ~26,000 characters
into it — the process never reached the end of the run.

Backtrace

Faulting thread is an httplib::ThreadPool worker, in request intake. Outermost frames, from gdb
on the core (coredumpctl only stores 64 frames, all of which are interior to the regex executor —
the caller is invisible without unwinding the whole stack):

#0     ?? (libc.so.6+0xa14ba)
#1     malloc
#2     operator new(unsigned long)
#3     std::__detail::_Executor<…>::_M_lookahead(long)
#4     std::__detail::_Executor<…>::_M_dfs(_Match_mode, long)
#5     std::__detail::_Executor<…>::_M_rep_once_more(_Match_mode, long)
…      _M_dfs / _M_rep_once_more, 78,099 frames
#78102 std::__detail::__regex_algo_impl<…>
#78103 unicode_regex_split_stl<char>(…)                       libllama.so
#78104 unicode_regex_split(…)                                 libllama.so
#78105 llm_tokenizer_bpe_session::tokenize(…)                  libllama.so
#78106 llama_vocab::impl::tokenize(…)                          libllama.so
#78107 llama_vocab::tokenize(…)                                libllama.so
#78109 common_tokenize(…)                                     libllama-common.so
#78110 tokenize_mixed(…)                                      libllama-server-impl.so
#78111 tokenize_input_subprompt(…)                            libllama-server-impl.so
#78112 tokenize_input_prompts(…)                              libllama-server-impl.so
#78113 server_routes::handle_completions_impl(…)              libllama-server-impl.so
#78114 std::_Function_handler<unique_ptr<server_http_res>…>   (two handler frames)
#78116 server_http_context::post(…)                          libllama-server-impl.so
#78117 httplib::Server::dispatch_request(…)
#78118 httplib::Server::process_request(…)
#78119 httplib::Server::process_and_close_socket(int)
#78120 httplib::ThreadPool::worker(bool)
#78121 (libstdc++) → #78122 (libc) → #78123 clone

The same three outermost frames — tokenize_input_prompts → handle_completions_impl →
httplib::ThreadPool::worker — appear on 4 of 4 coredumps examined. Only the one dump above was
unwound in full.

Note that the crash is on a spawned thread, whose stack size is fixed at creation. It is not the
main thread and it cannot grow. httplib's ThreadPool does not set a stack size, so on glibc these
workers get RLIMIT_STACK's soft value; the only pthread_attr_setstacksize call anywhere in the
tree is in vendored miniaudio, which is not on this path.

Steps to Reproduce

All of the below were run against gpt-oss-120b-MXFP4.gguf, whose tokenizer.ggml.pre is gpt-4o.
Any GGUF whose tokenizer_pre is gpt-4o, llama4, kanana2 or talkie selects the same
expression (src/llama-vocab.cpp:2307), as does LLAMA_VOCAB_PRE_TYPE_MINIMAX_M2, which shares the
case. LLAMA_VOCAB_PRE_TYPE_GRANITE_EMB_MULTI (granite-embed-multi-97m,
src/llama-vocab.cpp:442) has the same two lookahead-inside-greedy-repetition drivers with \p{M}
added, and is likewise absent from unicode_regex_split_custom(), so it should be affected
identically — its own source comment notes the expression family is chosen to "avoid excessive
backtracking", which is the hazard rather than the safeguard here. We only tested the one GGUF, so
treat the rest as inference from the source.

1 · llama-tokenize — one command, no GPU, no inference (loads vocab_only, ~0.8 s; it still
needs the GGUF present):

python3 -c "open('/tmp/x.txt','w').write('x'*36000)"
llama-tokenize -m gpt-oss-120b-MXFP4.gguf -f /tmp/x.txt --ids
# → Segmentation fault (core dumped)

python3 -c "open('/tmp/x.txt','w').write('x'*20000)"
llama-tokenize -m gpt-oss-120b-MXFP4.gguf -f /tmp/x.txt --ids
# → fine

Set the limit explicitly if your shell's default differs from 8 MiB — every threshold quoted here is
from ( ulimit -s 8192; llama-tokenize … ).

2 · llama-server — kills the process:

python3 - <<'EOF' > /tmp/body.json
import json
print(json.dumps({"model":"gpt-oss-120b-MXFP4.gguf","max_tokens":512,"temperature":0,
                  "stream":False,"messages":[{"role":"user","content":"x"*36000}]}))
EOF
curl -sS -X POST http://127.0.0.1:8081/v1/chat/completions \
     -H 'Content-Type: application/json' --data @/tmp/body.json
# → curl: (52) Empty reply from server; the server process is gone with SIGSEGV

3 · The controls that establish what the trigger actually is. Each line is one
( ulimit -s 8192; llama-tokenize -m gpt-oss-120b-MXFP4.gguf -f FILE --ids ):

# DIES — a run of one class, whatever the class is
python3 -c "open('/tmp/a.txt','w').write('x'*36000)"       # lowercase letters
python3 -c "open('/tmp/b.txt','w').write('X'*36000)"       # UPPERCASE letters — also dies
python3 -c "open('/tmp/c.txt','w').write('.'*36000)"       # punctuation — also dies
python3 -c "open('/tmp/d.txt','w').write(' '*36000)"       # spaces — also dies
python3 -c "open('/tmp/e.txt','w').write('日'*36000)"       # CJK letters
python3 -c "open('/tmp/f.txt','w').write('日本語のテキストです'*3000)"   # 30,000 cp, unpunctuated

# SURVIVES — the class changes often enough that no single run gets long
python3 -c "open('/tmp/g.txt','w').write('Xx'*18000)"      # 36,000 letters, alternating case
python3 -c "open('/tmp/h.txt','w').write('1234567890'*6600)"  # 66,000 digits: \p{N}{1,3} is bounded
python3 -c "open('/tmp/i.txt','w').write('吾輩は猫である。名前はまだ無い。どこで生れたか、とんと見当がつかぬ。'*1500)"

The 'Xx' and 'X' pair is the one worth running first: it shows the run has to be case-homogeneous,
and it rules out "long input" as the explanation.

4 · Self-contained, no llama.cpp and no model — for confirming the mechanism in isolation. Takes
the same expression, applies the same \p{...} → collapsed-class substitution that
unicode_regex_split() performs, and iterates with the same optimize | nosubs flags. It has an
explicit-pthread_attr_setstacksize arm and a default-attribute std::thread arm, and bisects both.
Posted as a follow-up comment on this issue rather than linked, so nothing here depends on an
external host.

Measurement: the threshold is linear in stack size

Isolated probe, -O3 -DNDEBUG, subject 'x' * N, bisected to a 500-character bracket:

Cross-checked against the shipped binary: at the same 8 MiB, llama-tokenize survives 26,000
and dies at 26,500 for 'x', and at 26,250 for 'X' and '日' — the same bracket as the isolated
probe, so the constant is the real one and not an artefact of the reproducer. The sweep above was
run for 'x' only; the other classes were bracketed against the shipped binary, not swept.

Raising RLIMIT_STACK moves the threshold proportionally (glibc sizes default-attribute thread
stacks from it), which is a usable stopgap for an operator but not a fix: the request body is
unbounded and tokenization runs before the context-length check, so a prompt too long to serve
still crashes the process on its way to being rejected.

Suggested directions

Offered in case they are useful; the maintainers will know which is right for the codebase.

1. Add o200k / LLAMA_VOCAB_PRE_TYPE_GPT4O to unicode_regex_split_custom(). It is the same
   treatment GPT-2, LLaMA-3, Qwen2, Qwen3.5, Kimi-K2 and AFMoE already get, and it removes
   std::regex from this vocab's path entirely. This is probably the cheapest complete fix, and it
   will also be considerably faster. unicode,test: add Qwen3.5 non-backtracking tokenizer handler and regr… #22110 did exactly this for Qwen3.5 after Eval bug: Stack overflow / crash when tokenizing long text with Qwen3.5 models #21919, and unicode : add custom Qwen2 regex handler to fix segfault on long input #21257 for
   Qwen2 — so this is a path the codebase has already taken twice for the same defect class.
2. Bound the subject. unicode_regex_split_stl already iterates over offsets; pre-splitting a
   long run on a safe boundary before handing it to std::regex would cap depth. Care needed:
   splitting mid-token changes tokenization, so the boundary must be one the expression cannot match
   across.
3. Refuse rather than crash. A depth or input-length guard in unicode_regex_split_stl that
   throws (the caller already wraps this in a try at src/unicode.cpp:1301 with
   catch (std::regex_error & e) at :1384) turns a process kill into a 500 for one request. Worth
   having regardless of 1 or 2, because the same recursion exists for every other std::regex
   pre-tokenizer path.
4. Cap the request body in the server (set_payload_max_length is currently never called), so
   an unbounded subject cannot arrive from the network in the first place.

Happy to test any patch against the reproducers above and report back.

First Bad Commit

Not bisected, and probably not a regression — the mechanism is present wherever
LLAMA_VOCAB_PRE_TYPE_GPT4O lacks a unicode_regex_split_custom fast path, so it likely dates from
o200k support rather than from a change.

It is, however, a recurrence, and that seems worth saying plainly. #17636 reported this same
symptom on the same model family — gpt-oss-120b, 'A' * 10000, with real prose of far greater length
unaffected — and it was closed by #17786, which added optimize | nosubs to the std::regex
construction in unicode_regex_split_stl.

Those flags are present in this build and the crash still occurs. At src/unicode.cpp:745:

constexpr auto regex_flags = std::regex_constants::optimize | std::regex_constants::nosubs;

They reduce the size of each recursion frame, but the executor still recurses once per repetition, so
depth remains linear in the length of the run and a long enough run still exhausts the stack. That is
why what we measured is a threshold rather than a fix: nosubs moved the threshold outward, it did
not remove the recursion. The eleven measurements above are all on a tree that contains #17786.

#21919 is the same defect for Qwen3.5, and it was resolved by #22110 — a non-backtracking handler in
unicode_regex_split_custom() plus a regression test. That is suggestion 1 below, and it is the
treatment LLAMA_VOCAB_PRE_TYPE_GPT4O has never received.

Relevant log output

systemd: <unit>.service: Main process exited, code=killed, status=11/SEGV
systemd-coredump: Process <pid> (llama-server) of user <uid> dumped core.

Nothing is logged by the server itself. The last lines before the fault belong to the previous
request completing normally, which is why this initially looks like a crash after a successful
generation rather than during intake of the next one.