Tool call written inside <think> (no </think>) is returned as reasoning_content: no tool_calls, finish_reason=stop

Summary

Qwen3-family models (seen with Qwen3.8-Flash-Next, GSQ-RCO IQ3_S) sometimes emit a complete, well-formed <tool_call> inside the thinking block without closing </think> first. serve/frontend.py → OutputParser only leaves the "reasoning" state on </think>, so the whole call ends up in reasoning_content. The client (an agent harness) receives tool_calls: [], empty content and finish_reason: "stop", and the agent loop silently stops: the call is never executed.

Frequency observed: 4 of ~1050 responses in real agent use (Hermes Agent, OpenAI Chat Completions API, streaming).
Engine 0.1.38, serve/server.py, thinking mode on, official Qwen thinking sampling (temperature 1.0, top_p 0.95, top_k 20, presence 0). Not a sampling issue.

What the model generated (shape of all 4 cases)

...the reasoning text, then the model decides to act.

<tool_call>
<function=execute_code>
<parameter=code>
print("hello")
</parameter>
</function>
</tool_call><|im_end|>

No </think> anywhere. The call itself is complete and parseable.

Minimal reproduction (no GPU)

from serve.frontend import OutputParser

text = ("I will run it now.\n\n<tool_call>\n<function=execute_code>\n"
        "<parameter=code>\nprint(1)\n</parameter>\n</function>\n</tool_call>")
p = OutputParser(thinking=True, tools=None, stream_tools=True)
evs = p.feed(text) + p.finish()
print([e.kind for e in evs])
# current main: ['reasoning']: no 'tool_call' event

Root cause

In OutputParser.feed(), the "reasoning" branch searches only for THINK_END (</think>). CALL_START is only looked for in the "content" state. A call generated before </think> is never seen as a call.

Prior art: vLLM

vLLM PR #35687 ("[Bugfix] Treat <tool_call> as implicit reasoning end in Qwen3 parser", merged 2026-04-24) fixed the same bug. In current vLLM (vllm/parser/qwen3.py, the grammar in qwen3_config()), the transition (REASONING, "TOOL_START") -> TOOL_PREAMBLE emits REASONING_END + TOOL_CALL_START, so any <tool_call> ends the reasoning, unconditionally.
vLLM PR #59821 ("Preserve quoted Qwen3 tool markup in reasoning") is still open. It addresses the false-positive side: a model that only mentions <tool_call> in its reasoning (quoting the format, explaining it) should not trigger a call.
vLLM's rule that ignores paired <tool_call> tokens applies to prompt tokens, not generated output, so it is not relevant here.

Proposed fix (implemented and tested locally, ready as a PR)

Adopt vLLM's idea (an implicit end of reasoning), but with a stricter trigger so quoted markup stays reasoning. In the "reasoning" state, a generated <tool_call> ends the thinking only when all three hold:

it is at the start of a line (right after \n, or the very first text of the thinking);
it is outside a fenced code block (``` or ~~~) of the reasoning;
it is followed, after only whitespace, by <function= (the start of a real call body).

The text before it stays reasoning_content, and the call goes through the normal "call" state, so it is extracted in streaming (stream_tools) and non-streaming paths alike. While the text after <tool_call> is still only whitespace or a prefix of <function=, it is held back (the same approach as for partial tags), so streamed and whole outputs are identical.

Why stricter than vLLM: an unconditional rule turns things like "I must emit `<tool_call>` next", "The format is <tool_call>…" or an indented or >-quoted example into a bogus call. All real failures had the exact shape \n\n<tool_call>\n<function=NAME>, so the three conditions cost nothing in recall.

Edge case: if an implicit call never completes (output truncated), it is returned as reasoning, as before, not as content.

Observability: each implicit end is logged ([strata] implicit end of thinking: N tool call(s) written inside the thinking without </think> were read as calls) and counted in GET /metrics (totals.implicit_reasoning_ends plus a per-request field), so the rate can be tracked per model and quant.

Defense in depth: chat template

In serve/chat_template.jinja, when an earlier assistant message has a call left inside the thinking, </think> is now closed before the call when the history is rendered. This applies when:

content starts with an open <think> and contains a line-start <tool_call>\n<function=, or
there are no tool_calls and reasoning_content ends with such a call.

This way the model is never shown a call inside <think> as an example to imitate. All 10 cases in chat_golden.json render byte-identical. (Reference: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates.)

Note: the server loads the template from the pack (<pack>/tokenizer/chat_template.jinja), not from serve/. A template fix must also reach the pack, or strata_pack must copy it.

Tests

New serve/test_reasoning_toolcall.py (12 tests):

The 4 real failures (sanitized fixture): each yields exactly 1 tool_call, with the reasoning preserved. Checked when fed 1 char at a time, in random chunk sizes, and whole, with stream_tools on and off. All 4 yield 0 calls on current main.
10 synthetic mention-only controls, none of which yield a call:
- backticks mid-sentence;
- a whole call written mid-sentence;
- backticks at line start;
- a line-start tag with no <function=;
- another tag after it;
- indented;
- a markdown > quote;
- inside ``` and ~~~ code blocks;
- a bare tag before </think>.
They also stay correct when followed by </think> and a real call.
Server-level: finish_reason=tool_calls, the log line, and the /metrics counter.
Template: chat_golden.json matches 10/10, and both history repairs are covered.
Full serve/test_*.py suite: 209 pass, 9 skipped (environment: Windows-only, no jsonschema, no pack tokenizer).

Side finding: agent harnesses and `preserve_thinking`

Hermes Agent strips reasoning_content from the history for generic OpenAI-compatible endpoints. It keeps it only for DeepSeek, Kimi and MiMo, or with model.reasoning_echo: true in its config (source: agent/message_sanitization.py). /metrics agrees: after a 9,977-token reply, the next prompt grew by only 64 tokens. With the default preserve_thinking=true, past assistant turns therefore render as empty <think>\n\n</think> blocks, and prefix-cache reuse stops at the last assistant turn. This is not a bug in Strata, but worth documenting for users of agent harnesses.

Keywords (for search)

tool_call inside think, missing </think>, reasoning_content contains tool_call, empty tool_calls, finish_reason stop instead of tool_calls, agent stops after reasoning, Qwen3 implicit reasoning end, OutputParser, Qwen3.8-Flash-Next, IQ3_S, Hermes Agent, vLLM #35687.

I have a massive problem with that (and even more when trying to get Qwen to patch this issue). Could you kindly post your private PR findings?

Thanks!

serve: stranded-call's opener must begin a line, outside any code fence

@pawg

@pawg the fix is now a PR: #970 (2 commits on top of current main / v0.1.39).

It has been running in production here since 2026-10-04 with Hermes Agent. The server recovered 2 real cases (a tool call inside <think>, no </think>) that previously stopped the agent, with no regressions seen. Each recovery is logged ([strata] implicit end of thinking) and counted in GET /metrics → totals.implicit_reasoning_ends, so you can check whether it fires for you.

The trigger is deliberately strict: a line-start <tool_call>, outside code fences, followed by <function=. Quoted markup stays in the reasoning. Details are in the PR description.

To try it before it is merged, cherry-pick the PR's two commits onto your checkout and restart the server (the parser change needs no rebuild).

Another independent data point, from Hermes Agent on a separate machine.

Setup: Qwen3.8-Flash-Next GSQ-RCO, engine 0.1.39, thinking on, the usual Qwen sampling (temperature 1.0, top_p 0.95, top_k 20). I rotate between IQ3_XXS, IQ2_XS and the Coder IQ1_M build. Across about two weeks of real agent traffic (a Telegram bot doing sysadmin and coding work), roughly 1300 responses, Hermes logged 13 reasoning-only clean stops. That is around 1 in 100, a higher rate than the 4 in 1050 reported here, and it showed up on all three quants, so it does not look quant-specific.

The worst one is the case that actually hurts: a turn with 74 tool calls behind it ended this way. The model had been working through a task, announced the next step in its reasoning, and stopped. To the user it reads as "I'll do X now" followed by nothing, a job left half-done with no explanation anywhere in the response.

For completeness on the client side: Hermes has a recovery for this exact shape. Empty content plus finish_reason=stop makes it promote the reasoning into the visible answer (logged as "Reasoning-only clean stop"). The user then gets the model's internal monologue twice, once as the reasoning recap and once as the promoted answer, so even a recovered turn surfaces thinking instead of the answer the model never wrote. I did not keep raw transcripts, so I cannot confirm every one of the 13 had the complete stranded call of #804, but the stall-after-announcement symptom matches, and the per-quant spread (10 of 13 on IQ3_XXS, the rest on IQ2_XS and Coder IQ1_M) tracks how much traffic each build carried.

serve: a rescued call must name a tool the request declared

serve: a specimen corpus for the format fixes

@47Hunter47

Thanks @47Hunter47, that matches what we saw (the "I'll do X now", then nothing).

For anyone landing here: the fix now lives in #525 (format_fixes / stranded-call, opt-in, off by default). It uses the same strict trigger (a line-start <tool_call>, outside code fences, followed by <function=) plus a natural-stop gate. I closed my #970 in its favour, and our 4 recorded failures and the mention-only controls are offered there as tests.

Related: #843 found that an empty assistant turn left in the history makes the model imitate it and stop calling tools. A stranded call like this one is a likely source of such empty turns.

Cross-posting the measurement here, since this is where the symptom was reported.

12 real strandings recovered in full from the client's session DB (~1,300 replies, Hermes Agent on a second Windows machine, Qwen3.8-Flash-Next across IQ3_XXS / IQ2_XS / coder IQ1_M). All 12 end ON a complete call block — line-start, outside any fence, no prose after it — so the tightened contract costs nothing on this corpus either.

The live fix in #525 (opt-in format_fixes: stranded-call) recovers 9 of the 12, identically whole and streamed. The three misses: one turn ended non-naturally (correctly refused by the finish-reason gate), and two are the <function=tool_call> envelope shape, which is neither rescued nor even logged today — those still stall silently with the fix on.

Details, the recall table (main / #525 fix off / #525 fix on / #970) and minimal fixtures: #525 (comment)

serve: a stranded act nothing delivers is loud in both flag states

@Niko1221

A consolidated status for this issue, since the fix and its evidence are now spread over a PR thread. The short version: the bug is characterised, the fix is written and reviewed, and it is opt-in — what is missing is landing it.

Frequency, two independent harnesses. The counts below are real agent traffic, not synthetic tests.

harness	replies	strandings	models / quants
Linux server + 1× L40S vGPU, Hermes Agent + Unreal MCP (this repo's reporter, #970)	~1,050 + ~1,270	4 + 5	Qwen3.8-Flash-Next IQ3_S, UD-IQ4_XS
Linux server + 1× RTX 3090, Hermes Agent (Telegram sysadmin/coding agent)	~1,300	12	IQ3_XXS, IQ2_XS, coder IQ1_M

Roughly ~1% of replies on both harnesses — about twenty recorded events in total, on four different quants, two different GPUs and two unrelated workloads — so this is model-family behaviour, not a quant or a hardware artefact. The damage is specific: the turn ends after the model has planned the next step, so the client shows "I'll do X now" and stops. The worst one here had 74 tool calls behind it.

This is the third member of a class this repo has already fixed twice. #211 (a call cut off at end of turn) was fixed in 0.1.31, and #530 (a reply reaching max tokens while still thinking) got its server-window fix in 0.1.38 — where the budget stayed opt-in, for exactly the reason the flag below is off by default. This one is the same family: OutputParser only leaves the reasoning state on </think>, so a call the model writes inside an unclosed thinking span goes out as reasoning_content with tool_calls: [] and finish_reason: "stop" — the client's turn ends with nothing to execute. Related and still open: #843, where an empty assistant turn in the history makes later turns imitate it and stop calling tools.

The fix is ready and opt-in. #525 (head 4111c85) adds --format-fixes / format_fixes in strata-<model>.json, off by default, so no existing answer changes unless asked. Its rescue fires only when all of these hold: the turn ended naturally (stop, not a max_tokens cut), the opener is at the start of a line, outside a code fence, followed by <function=, the tail is only complete calls, and every called name is one the request declared. Measured against the 12 real Windows specimens above: 9 of 12 turns delivered as real calls, 11 of 12 loud — the remaining one is the non-natural end the gate is designed to refuse. Streaming is byte-identical between the two flag states, verified at whole / 1-char / random 1-9 / 7-char feeding with stream_tools both ways. Full serve/ suite on that head: 281 passed, 7 skipped.

Verdict-by-verdict, since vLLM made the same call (vllm-project/vllm#35687 treats any <tool_call> in reasoning as an implicit end): both approaches fix these 12, but #525's stricter trigger also refuses the mentions that the unconditional rule would fire on, and its declared-name gate matters — two of the twelve are a call wrapped in <function=tool_call>, a name no request declares, and delivering that hands the client a bogus tool.

Reproducing it takes minutes, no GPU. OutputParser is pure Python. The 12 specimens are published genericised, in this repo's own corpus schema, with a replay script: https://gist.github.com/47Hunter47/4abcd3a80701f5634f85b96e4c067cc3 — append them to serve/fixtures/format_fix_specimens.json and the runner exercises them at four chunk sizes with stream_tools both ways; that tree is green (25 → 37 specimens, 507 subtests). Which is also the point: a fix here comes with recorded live input, so a later change to any trigger re-runs the whole recorded world.

The ask. @Niko1221 — this is the third issue in this family and the one with the most recorded evidence behind it; the shape to land is the opt-in one, so nothing changes by default. Either #525 as it stands, or the parser part ported into your own branch — and if it is ported, the two gates are the parts worth keeping: the natural-stop gate (a budget cut is when the span is most often open, and a call quoted in that reasoning was considered, not made) and the declared-name gate (the <function=tool_call> envelope is about a third of live strandings across both harnesses, and it must stay reasoning rather than become a bogus tool).

Happy to re-run the measurement against any head or to extend the corpus if a different shape of the fix is preferred.

serve: a tool call inside the reasoning counts only for a declared to…

serve: a <tool_call> in the reasoning that never closes stops holding…

Fixed in 0.1.40: a tool call written inside the thinking is returned as a tool call when its name is one of the request's tools. Anything else stays reasoning, and without tools the output is unchanged.

@sergqwer

Verified 0.1.40 (tag = 1735d64, fix commit 9fb1b9f) against two sets of recorded input. Short version: it fixes the reported bug and it recovers more live cases than the opt-in PR did — but because the only condition is "the name is declared", it also acts on reasoning that was only describing a call, including the one shape where acting is destructive.

On live traffic it is the better fix. The 12 real strandings from this machine (Hermes agent, ~1,300 replies, three quants) all delivered as real calls: 11/12, against 9/12 for #525's opt-in rescue. The mixed tail is handled better too — where #525's tail contract is all-or-nothing and drops a declared call that sits beside the <function=tool_call> envelope, 0.1.40 delivers the declared one. No flag to set, and the two envelopes stay reasoning, as they should.

What changes for everyone, though. The rescue now fires wherever a <tool_call> appears in the thinking, when the called name is one the request declares. It does not need the opener to start a line, does not care about code fences, and does not care how the turn ended. Against #525's own edge-case corpus (serve/fixtures/format_fix_specimens.json at 4111c85, 25 entries, each with a stated expectation): 5 of 5 "act" entries correct, but 10 of the 20 negative controls now deliver a call.

The one I would look at first — a call quoted in reasoning whose turn was then cut by max_tokens:

I could run
<tool_call>
<function=execute_code>
<parameter=code>
rm -rf build
</parameter>
</function>
</tool_call>
but first let me check what build holds...

finish_reason: "length", execute_code declared. 0.1.39 → reasoning, nothing to run. 0.1.40 → tool_call execute_code {"code": "rm -rf build"}. On a client that auto-approves (Claude Code in auto-approve mode, or an agent harness with a permissive allow-list — both are this repo's users), the command runs. This is the exact case @sergqwer raised on #525's thread, and the reason that PR gates on a natural stop.

The rest of the ten, same run, all offline:

a fenced documentation example (```xml block containing a complete call) → delivered;
The format is <tool_call>…</tool_call> ok. mid-sentence → delivered;
<parameter=code> followed by prose, two blocks with prose between them → two calls delivered;
a call whose body is <function=execute_code></function>, no parameters at all → delivered as a call with no arguments.

Cheap ways to close it, if you want them. All 12 live strandings have the shape \n\n<tool_call>\n<function=NAME> — opener at the start of a line, after a blank line — and all of them ended on a natural stop. So requiring the opener to begin a line (or merely refusing one inside a fence), and refusing the rescue when the turn ended by length, costs nothing in recall on the live data and removes the destructive case. Chunking is consistent either way: whole, 1-char and random 1-9 feeding agree on every entry above.

Reproducing. Both corpora are public and the parser is pure Python — no GPU, no model:

import sys; sys.path.insert(0, "<tree>")
from serve.frontend import OutputParser
TOOLS = [{"name": "execute_code", "parameters": {"properties": {"code": {"type": "string"}}}}]
text = open("case.txt").read()          # the cut case above, or any corpus input
p = OutputParser(thinking=True, tools=TOOLS, stream_tools=True)
print([(e.kind, getattr(e.call, "name", None), getattr(e.call, "arguments", None))
       for e in p.feed(text) + p.finish("length")])

#525's corpus: serve/fixtures/format_fix_specimens.json at 4111c85 (25 entries, expectations included). Mine (12 live strandings, genericised, this repo's schema): https://gist.github.com/47Hunter47/4abcd3a80701f5634f85b96e4c067cc3 — happy to re-run either against any head, or to add the cases above to a corpus if you would rather track them there.

With this in, #525's rescue is largely redundant and its remaining value is those two conditions — worth a look before that PR is closed either way. @Niko1221

serve: a <tool_call> in the reasoning that never closes stops holding…

serve: a tool call inside the thinking is a call only when it was an …

serve: a tool call inside the reasoning counts only for a declared to…

serve: a <tool_call> in the reasoning that never closes stops holding…

serve: a tool call inside the thinking is a call only when it was an …

serve: a tool call written <function= NAME> is the call NAME

Merge upstream main (7695859, engine 0.1.43): the server, the tokeniz…

serve: a tool call written <function= NAME> is the call NAME

Calls inside the reasoning: a "> " blockquote line is a quote

serve: keep the tools-less pass-through for Codex's compaction, not f…

reasoning_close_retry: a turn that ends on a call in its thinking is …

Fix #3441, #3075: a drifted tool-call opener is read as the call; a c…

Fix Niko1221#3441, Niko1221#3075: a drifted tool-call opener is read …

serve: 開きの崩れた tool call（<=NAME>、<function:NAME>、<NAME>）を、NAME が要求で宣言し…

Fix Niko1221#3441, Niko1221#3075: a drifted tool-call opener is read …

Navigation Menu

Repository navigation

Tool call written inside `<think>` (no `</think>`) is returned as `reasoning_content`: no `tool_calls`, `finish_reason=stop` #804

Summary

What the model generated (shape of all 4 cases)

Minimal reproduction (no GPU)

Root cause

Prior art: vLLM

Proposed fix (implemented and tested locally, ready as a PR)

Defense in depth: chat template

Tests

Side finding: agent harnesses and `preserve_thinking`

Keywords (for search)

11 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Tool call written inside <think> (no </think>) is returned as reasoning_content: no tool_calls, finish_reason=stop #804

Description

Summary

What the model generated (shape of all 4 cases)

Minimal reproduction (no GPU)

Root cause

Prior art: vLLM

Proposed fix (implemented and tested locally, ready as a PR)

Defense in depth: chat template

Tests

Side finding: agent harnesses and preserve_thinking

Keywords (for search)

Activity

pawg commented on Oct 5, 2026

talisp commented on Oct 5, 2026

47Hunter47 commented on Oct 5, 2026

talisp commented on Oct 5, 2026

47Hunter47 commented on Oct 5, 2026

47Hunter47 commented on Oct 6, 2026

Niko1221 commented on Oct 6, 2026

47Hunter47 commented on Oct 6, 2026

11 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Tool call written inside `<think>` (no `</think>`) is returned as `reasoning_content`: no `tool_calls`, `finish_reason=stop` #804

Side finding: agent harnesses and `preserve_thinking`