Tool Call/Reasoning truncates when output contains </parameter> or </tool_call> #210

Firstly, many thanks for collating all the Qwen3.8 Flash Next optimizations together to pull a literal miracle. Opus 5.5 couldn't have been cheap. I ran into a pretty bad bug while using vscode copilot to refactor my agentic harness notes. I'm guessing any kinda vibecoding involving LLM templates will likely run into the same bug I did.

Anyways, with regards to the bug and reproduction (tested), here is what Deepseek v4.1 Flash has to say:

Component: frontend.py (OutputParser, parse_tool_call)
Severity: data loss + silent success (the client is told the call completed).

Summary

An XML-format tool call is delimited by literal strings, and the parser ends a string argument at the first occurrence of </parameter> anywhere in the value, and ends the call body at the first </tool_call> regardless of nesting. If a generated argument contains either sequence, everything after it is dropped, and the streamed tool-call JSON is then closed artificially so it parses. The client writes the truncated file and reports success.

Same class also swallows normal assistant text (for example when reasoning): a literal <tool_call> in prose puts the parser into call state (serve/frontend.py:436), so the following text is emitted as a bogus tool call or discarded at finish() (serve/frontend.py:473).

Root cause

• TOOL_CALL/PARAMETER termination is a bare str.find on the raw text:
  • serve/frontend.py:260 (parse_tool_call), :354 and :371 (_scan, str/raw states): rest.find("</parameter>")
  • serve/frontend.py:452 (feed): self.buf.find(CALL_END) — no nesting check
• serve/frontend.py:389 _close_scan then appends " + } to whatever it holds, fabricating a well-formed JSON object. An interrupted/ambiguous call can therefore never fail loudly, and any tail held for a partial tag by _hold (:404) is additionally lost.
• parse_tool_call (:245) accepts only the <function=…> form; the template's auto mode can emit the JSON form (serve/chat_template.jinja:293), which would raise "malformed tool call" — separate latent bug, not this one.

A second, independent trigger: a literal control token in the content (<|im_end|> / <|endoftext|>) stops generation before the text reaches the parser — serve/server.py:844, ids from :576. _close_scan then closes the call silently, producing the identical symptom.

Reproduction (deterministic, no model)

from serve.frontend import OutputParser

CONTENT = "line\n</parameter>\nmore"          # the literal to break the parse
body = (
    "<tool_call>\n<function=create_file>\n"
    "<parameter=path>\na.md\n</parameter>\n"
    f"<parameter=content>\n{CONTENT}\n</parameter>\n"
    "</function>\n</tool_call>"
)
p = OutputParser(thinking=False, stream_tools=True)
for ev in p.feed(body) + p.finish():
    print(ev.kind, "|", ev.text or (ev.call.arguments if ev.call else ""))

Expected: content == CONTENT ("line\n\nmore").
Actual: call.arguments == {'path': 'a.md', 'content': 'line'} — the tail is gone, and the streamed argument pieces (_scan + _close_scan) form a valid, complete JSON object, so the client sees success.

Swapping </parameter> for </tool_call> in CONTENT reproduces it through the :452 path instead.

End-to-end: ask the model to write a file whose content contains a literal </parameter> or </tool_call> inside a fenced block. Intermittent, because it depends on the sampling landing on that token, and note the tail is cut at the opening of the sequence, so the artifact looks like a mid-line truncation with no error.

Suggested fix — grammar-aware termination, with fail-loud as a safety net

Restrict terminators to positions the grammar can actually follow, using state _scan already tracks:

• </parameter> closes a value only when the next non-space token is <parameter= or </function>; otherwise skip it and keep scanning (:354, :371, and :260 in parse_tool_call).
• </tool_call> closes the call only when the scan has reached done (§</function> complete); replace the blind find at :452.
• Defer via _hold (:404) while the successor is not yet visible, so this stays correct across streaming boundaries.

Then make incompleteness non-silent: if a call ends without a real </function>/</tool_call>, return the raw bytes as content (or an explicit error) instead of _close_scan inventing a valid close (:389). Also gate the stop-id check (serve/server.py:844) on the parser not being mid-call, or let the same safety net catch it.