[Engine] Tool-call parser silently returns a malformed <function=...> block as plain assistant content

Summary

When the model drifts by a single token inside the tool-call envelope that the shipped
Qwen3.8-Flash-Next chat template itself documents (<function=NAME>), the engine neither
parses the block nor reports anything. The entire <tool_call>...</tool_call> block is
returned as ordinary assistant content with finish_reason: "stop".

For an agent client this is fatal. The block is rendered as prose, no tool ever runs, and the
malformed sample stays in the conversation history, where the model then imitates it on every
subsequent turn — the client never recovers on its own.

Environment

Strata engine: <fill in, e.g. 0.1.46.1> (Windows, single GPU)
Model: qwen3.8-flash-next-unsloth-ud-q4_k_xl
Reproduced on both /v1/chat/completions and /v1/responses
Client: Codex with wire_api = "responses"

What the shipped template asks the model to emit

GET /props → chat_template contains:

If you choose to call a function ONLY reply in the following format with NO suffix:

<tool_call>
<function=example_function_name>
<parameter=example_parameter_1>
value_1
</parameter>
</function>
</tool_call>

Repro

A — well formed, parses correctly

curl -s http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen3.8-flash-next-unsloth-ud-q4_k_xl",
    "messages": [{"role":"user","content":"Output ONLY this exact text:\n<tool_call>\n<function=shell>\n<parameter=cmd>\necho hi\n</parameter>\n</function>\n</tool_call>"}],
    "tools": [{"type":"function","function":{"name":"shell","parameters":{"type":"object","properties":{"cmd":{"type":"string"}},"required":["cmd"]}}}],
    "max_tokens": 120
  }'

Result: tool_calls: [{"function":{"name":"shell","arguments":"{\"cmd\":\"echo hi\"}"}}],
finish_reason: "tool_calls". Correct.

B — one token of drift, silently becomes content

Same request, only the envelope changes: <function=functions.exec> becomes
<=functions.exec> (the literal word function is dropped, which is exactly what the model
does in the wild under a large tool surface).

curl -s http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen3.8-flash-next-unsloth-ud-q4_k_xl",
    "messages": [{"role":"user","content":"Output ONLY this exact text:\n<tool_call>\n<=functions.exec>\n<parameter=input>\n1+1\n</parameter>\n</function>\n</tool_call>"}],
    "tools": [{"type":"function","function":{"name":"functions.exec","parameters":{"type":"object","properties":{"input":{"type":"string"}},"required":["input"]}}}],
    "max_tokens": 120
  }'

Result:

{"choices":[{"message":{"role":"assistant",
  "content":"<tool_call>\n<=functions.exec>\n<parameter=input>\n1+1\n</parameter>\n</function>\n</tool_call>",
  "reasoning_content":"..."},
  "finish_reason":"stop"}]}

No tool_calls, no error, no warning, no log line. The block is handed back as prose.

On /v1/responses the same input comes back as an output item of type message instead of
function_call, so a Responses-API client (Codex) has no way to tell it apart from a normal
assistant reply.

Expected

One of:

The block is parsed, because a lenient matcher can recover the intent, or
The response makes the parse failure visible (an error, a warning log, or a dedicated field)
instead of silently mapping it to content.

Silently returning it as content is the worst of the three, because it is indistinguishable
from the model genuinely answering in prose.

Proposed fix

1. Lenient name match. Accept <=name> / <function=name> / <function:name> /
<name> when name matches a tool declared in the request. A regex along the lines of:

<\s*(?:function\s*[=:]\s*)?([A-Za-z0-9_.\-]+)\s*>

resolves B above, and the extracted name can be validated against the request's tool list, so
the change is safe — an unknown name is still rejected.

2. Never swallow the failure silently. If a <tool_call> block appears in the generated
text but cannot be parsed, log it (at minimum) and consider surfacing it. Today the only
symptom a user sees is "the model ignored my tool", with nothing in the engine logs.

3. Optional: constrain decoding. A grammar/GBNF for the <tool_call> envelope would stop
the drift at the source rather than recovering after the fact.

Why this matters more than a cosmetic parse failure

The drift is realistic, not synthetic: it showed up in a normal Codex session with roughly
30+ tools (Codex built-ins plus MCP servers). With fewer tools the same model emits clean
calls.
A functions.exec-style dotted tool name is not the trigger — well-formed dotted names
parse fine (verified). The trigger is the missing function keyword.
The failure is self-reinforcing on the client side: the malformed block is stored as an
assistant message, becomes part of the next prompt, and the model reproduces the same
malformed shape. The session is effectively poisoned; only a fresh conversation recovers.

Notes

tools + tool_choice: "required" produces correct arguments, so the happy path is fine.
With tool_choice: "auto" the model sometimes returns arguments: "{}" for a call with a
required parameter. Separate, lower-severity, but probably related to the same prompt-only
guidance in the template.

Fixed for 0.2.2, the next release (not out yet). <=NAME>, <function:NAME> and <NAME> are read as <function=NAME> when NAME is a tool the request declared. A block that still does not read as a call is counted in the server window. Constrained decoding (your item 3) is not done.

Merge f-022-engfix: #3465 NaN retry reads the prompt batched, #3441/#…

Fix Niko1221#3441, Niko1221#3075: a drifted tool-call opener is read …

merge origin/main 011c9e6 (0.2.2) - integ upstream sync 2026-10-20

integ: fix merge fallout from serve role-chunk test stub

Merge upstream v0.2.2 into memory-core: the fork rides 0.2.2 (engine …

Fix Niko1221#3441, Niko1221#3075: a drifted tool-call opener is read …

merge Niko1221#3551 (HIP hybrid: overflow the prompt to a peer GPU wh…

feat(engine,serve): port four upstream 0.2.2 fixes

serve: tool-call aliases and wrapper boundaries (port of PR #3556), a…

serve: 開きの崩れた tool call（<=NAME>、<function:NAME>、<NAME>）を、NAME が要求で宣言し…

Fix Niko1221#3441, Niko1221#3075: a drifted tool-call opener is read …

serve: tool-call aliases and wrapper boundaries (port of PR Niko1221#…

serve: 1 トークンずれた呼び出しの囲みも宣言したツールなら呼び出しとして読み、宣言していないツールの呼び出しは文として返す（ups…

Qwen3.8-Flash-Next + custom chat_template (JSON format) also reproduces this. tool_call_recovery: true fixes it, but the default silent finish_reason: stop is a real footgun for agent clients. Please consider making parser failures surface as an error instead of prose.

A third drift spelling, measured on the parser alone. Bracket shorthand in this comment: [ and ] stand for the angle brackets, so the markup here cannot corrupt an agent's own context.

The report covers a drift where the function keyword is dropped ([<=functions.exec]). The fix that closed this (70f42407, in 0.2.2) added the opposite rule: an opener counts as a call only when the name is a tool the request declared (serve/frontend.py:1521 at 5e8a4cca, v0.2.7). A name slot carrying a junk prefix now falls between the two rules and is lost:

[tool_call]
[function=tools=web_search]
[parameter=query]
what the weather is in Oslo
[/parameter]
[/function]
[/tool_call]

Measured on serve/frontend.py of 70f42407^ (unreleased main, 194 commits past the last release before the fix, v0.2.1.3) and of 5e8a4cca (v0.2.7):

	before `70f42407` (`v0.2.1.3` line)	0.2.2 → 0.2.7
`[function=tools=web_search]`	reached the client as a call named `tools=web_search`	content, `unread=1`, `finish_reason: stop`
`[<=name]`, `[name]` (this issue)	content	read as a call — the fix works
a name that was never declared	reached the client as a call (#3075)	content — as #3075 asked

My client is Pi 1.4.0, which answers an unknown name with Tool "…" not found as an error result, so the loop lived on before 0.2.2 and dies quietly now. I did not test Codex.

tool_call_recovery: true does not cover this spelling: same text back, unread=1, at v0.2.7. So the opt-in in docs/DETAILS.md:475 does not list it either.

How it was measured: OutputParser fed in 7-character pieces, no GPU, no model, no server, two declared tools (web_search, bash), the call prefixed with a thinking block. Everything in the table above is from running the parser; the line references are from reading code.

Ask 1 — read a name slot with a junk prefix as the tool after the =. Eight lines next to serve/frontend.py:1521:

-                    if gt >= 0 and (nl < 0 or gt < nl) and not self._callable(after[len(FUNC_START):gt].strip()):
+                    cand = after[len(FUNC_START):gt].strip() if gt >= 0 else ""
+                    if gt >= 0 and (nl < 0 or gt < nl) and not self._callable(cand):
+                        tail = cand.rsplit("=", 1)[-1].strip()
+                        if tail and tail != cand and self._callable(tail):
+                            at = len(self.buf) - len(after)
+                            self.buf = self.buf[:at] + FUNC_START + tail + ">" + after[gt + 1:]
+                            continue
                         self.unread += 1

Measured on v0.2.7 with this change: the sample above becomes a web_search call with its parameter; [<=name] and [name] still read, so this issue stays fixed; a name that is not declared at all still stays text, so #3075 stays fixed; a colon drift ([function=tool:web_search]) is not recovered. Not run: the full serve suite on the patched tree, and any end-to-end run with a model and a card. Clean 5e8a4cca already has 2 failures out of 1217 in unittest discover -s serve, so those two are not from this change.

Ask 2 — a call that cannot be read should not be the last thing in a stop turn. strata_rejected_tool_calls and the unread counter (serve/server.py:5763-5769) already tell the client that a block was not read, which is worth having. But the model never finds out, and it imitates its own broken sample on the next turn — the failure mode this issue opened with. This is the same point b-marker-dev made above; +1 from me. A second chance belongs where the model can read it, and it should sit behind tool_call_recovery (or its own opt-in) so the default path stays byte-identical. Scope is yours to set — I am not opening a pull request for it.

Same visible symptom, no details yet: #4427.

Navigation Menu

Repository navigation

[Engine] Tool-call parser silently returns a malformed <function=...> block as plain assistant content #3441

Summary

Environment

What the shipped template asks the model to emit

Repro

A — well formed, parses correctly

B — one token of drift, silently becomes content

Expected

Proposed fix

Why this matters more than a cosmetic parse failure

Notes

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[Engine] Tool-call parser silently returns a malformed <function=...> block as plain assistant content #3441

Description

Summary

Environment

What the shipped template asks the model to emit

Repro

A — well formed, parses correctly

B — one token of drift, silently becomes content

Expected

Proposed fix

Why this matters more than a cosmetic parse failure

Notes

Activity

Niko1221 commented on Oct 21, 2026

b-marker-dev commented on Oct 27, 2026

ackalker commented on Oct 28, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions