[Engine] Agent recovery when undeclared tool calls are returned as text

Problem

This follows up on #3075's declared-tool validation; it does not request reverting that policy.

In an OpenCode v2 + OMO-slim session, I observed two premature task stops with complete tool-call blocks naming bash, while the available command tool was shell. The parser behavior below reproduces conversion of the undeclared call to ordinary text. The original sessions have not been replayed end to end.

Environment

Parser baseline: Strata v0.2.2 (011c9e60).

Affected setup: Ubuntu 26.04, Python 3.14, RTX 5090 32 GB, 128 GB RAM, Qwen3.8-Flash-Next NVFP4, Chat Completions. Exact affected OpenCode and OMO-slim revisions are not included in this report.

Minimal reproduction

Run from the baseline repository root using its Python environment. No model or GPU is needed; no commands are executed.

from serve.frontend import OutputParser

tools = [{
    "name": "shell",
    "parameters": {
        "type": "object",
        "properties": {"command": {"type": "string"}},
        "required": ["command"],
    },
}]

for name in ("shell", "bash"):
    raw = (
        f"<tool_call>\n<function={name}>\n"
        "<parameter=command>\npwd\n</parameter>\n"
        "</function>\n</tool_call>"
    )
    parser = OutputParser(
        thinking=False, tools=tools,
        stream_tools=True, declared_only=True,
    )
    events = parser.feed(raw) + parser.finish("stop")
    calls = [e.call.name for e in events if e.kind == "tool_call"]
    content = "".join(e.text for e in events if e.kind == "content")
    print(name, calls, content == raw)

Observed output:

shell ['shell'] False
bash [] True

The undeclared-name result also holds with character-by-character input and recover=True.

Requested guidance

What is the supported way for an agent client to distinguish a rejected tool-call attempt from an ordinary final answer and attempt bounded recovery?

Would an opt-in machine-readable diagnostic be appropriate, or should recovery be handled entirely client-side? Unknown tools must remain non-executable, and quoted examples must remain text.

Scope and limitations

The reproduction establishes parser behavior, not an end-to-end OpenCode failure. A separate experimental fork is currently deployed, but it has known edge-case defects and is neither the reproduction baseline nor a proposed upstream fix.

Same problem here, from another client: Pithagoras (a web UI on top of the pi agent), Strata 0.2.2 (011c9e60), Qwen3.8-Flash-Next IQ3_S, /v1/chat/completions.

What happened

The chat offers a tool named godot_execute_gdscript.
The model called execute_gdscript instead. It dropped the godot_ prefix, and the call was also missing its closing </parameter> and </function> tags.
Strata sent the whole call back as normal text, with finish_reason: stop.
The client saw an ordinary answer, so the turn just ended. The model never found out it had made a mistake. The person had to tell it "you did the tool call wrong" before it tried again.

We hit this 4 times in two days: each time the model called a name that wasn't in that request's tool list.

Nothing in the log

Strata prints nothing when it turns a call to an unknown name into text. The "<tool_call> block(s) in the answer did not read as a call" line only shows up for some malformed forms, not for an unknown name. So the only way to notice is to read the chat.

What would help (any one of these)

Return the call as a tool call anyway. The client can't run a tool it doesn't have, so it answers "unknown tool", and the model fixes the name on the next step. Unknown tools still never run.
Or say it in the response, for example a field like "rejected_tool_calls": [{"name": "execute_gdscript"}], so a client can retry.
At least print a log line when this happens, so it can be found.

We fixed the cause on our side (our tool instructions named the tools without their prefix), but a small slip like this still ends the turn with no error.

I'm not a professional developer, and I wrote this with help from an AI assistant (Claude), so please read it with that in mind. The output and log lines below are copied from my own machine.

Same problem here, from a third client: OpenClaw, with its Tool Search on (OpenClaw's default). The request lists only a few tools, and the model is supposed to reach the rest through a tool_call tool, much like Open WebUI's deferred loader in #3883.

Environment: Strata from source at 8183c6b6 (v0.2.4.1 plus one docs commit), engine 0.1.48. Qwen3.8-Flash-Next IQ3_XXS. Windows 11, RTX 4060 Ti 16 GB, 64 GB RAM. Client: OpenClaw 2026.10.7 over Chat Completions, Tool Search in its default tools mode.

What happened: on 2026-10-24 I asked my agent for something that needs one of its skills, in three fresh sessions in a row. OpenClaw's prompt tells the model to load skills with skills_read, which is one of the tools behind tool_call, so the model called skills_read by name. Six replies across those sessions were exactly this, finish stop, and nothing ran:

<tool_call>
<function=skills_read>
<parameter=name>
briefing
</parameter>
</function>
</tool_call>

On 0.2.4.1 Strata does log this case, and the line names the undeclared tool. It logged this six times that day for skills_read, and once each for memory_search and notes_list:

[strata] 1 <tool_call> block(s) in the answer did not read as a call of a declared tool and went back as text (a name the request did not declare, or a form the template does not use); the names the model wrote: skills_read

From the git history (I haven't tested this), that logging arrived in 0.2.3 with 8bf178f, after the 0.2.2 build in the Pithagoras comment above.

Before: I updated straight from 0.1.45. On 0.1.45 the model made the same mistake six times over 2026-10-22 and 23 (skills_read or memory_search by name). Each time the call came back as a tool call, OpenClaw answered "Tool ... not found", and the model carried on in the same turn, for example by reading the skill file directly. On 0.2.4.1 there's no error for the model to react to, so the turn ends on the XML.

I understand why #3075 made this the rule, and I'm not asking to undo it. Either of the first two suggestions in the Pithagoras comment above would bring back that recovery for setups like mine: OpenClaw already refuses tools it doesn't know, so getting the call back as a call, the way 0.1.45 did, would let it answer "not found" again, and a field naming the rejected call would give it something to act on. I'm reporting the OpenClaw side to OpenClaw as well. For now I've added a line to the agent's instructions to always go through tool_call, which I haven't tested yet.

Reproduction status: reproduced locally.

Deterministic case behind this path: a mechanical MCP name spelling (mcp__server__tool -> mcp_server_tool) is rejected as "undeclared"

Another client hit this path in a way that is fully reproducible rather than intermittent, and the name it lost is only a spelling of a declared tool. I think it is worth recording here because it narrows the "undeclared name" set to one case that a small alias change fixes.

Environment

Strata 0.2.6, engine 0.1.51, built from source on Linux
Qwen3.8-Flash-Next UD-Q6_K_XL (125B MoE), /v1/chat/completions
Client: DSH (a Windows coding agent) with the Playwright MCP server

What happened

The request declares the Playwright tools under the mcp__server__tool convention (mcp__playwright__browser_navigate, mcp__playwright__browser_snapshot, mcp__playwright__browser_tabs). In one session the model wrote the same names with every __ collapsed to _:

<tool_call>
<function=mcp_playwright_browser_navigate>
<parameter=url>...</parameter>
</function>
</tool_call>

Ten answers in that session were returned as text with finish_reason: stop (navigate 3, snapshot 4, tabs 3), each one logged as:

[strata] 1 <tool_call> block(s) in the answer did not read as a call of a declared tool and went back as text (...); the names the model wrote: mcp_playwright_browser_navigate

In the same answers, calls with declared names (todo_write) were delivered as normal tool_calls, so the request did declare tools; only the spelling of the MCP calls differed. The client saw an answer with no call, retried, and the turn ended.

Why I believe this is not an "unknown tool"

mcp_playwright_browser_navigate is exactly mcp__playwright__browser_navigate.replace("__", "_") - the same tool, spelled the way Qwen often flattens names. The parser already accepts one such spelling difference (ns.member <-> ns__member through self.aliases); this is the same class of difference.

Proof it is the #3075 rule, not the model

I rolled back to the pre-0.2.6 parser (engine 0.1.44.1) and fed the identical content to OutputParser with the same declared tools: it emits two tool_call events, including mcp_playwright_browser_navigate. On 0.2.6 the second one is text. The model output is the same across the two versions.

Small fix I would suggest

In OutputParser.__init__, when building self.aliases, also register the collapsed spelling for declared names that contain __:

aliases.setdefault(declared.replace("__", "_"), set()).add(declared)

with the same unambiguous-only rule the existing aliases use, and skip it when the collapsed spelling is itself a declared name. Because the parser already maps an alias back to the declared name on the way out, the client receives mcp__playwright__browser_navigate and needs no change.

This does not replace the recovery this issue asks for (a genuinely unknown name would still end the turn silently); it removes one deterministic source of it.

Related: #4071 (the same log line, unreproducible there), #3995 (namespace spellings on the Responses route), #3883 (intermittent plain text).

I can send a PR with the alias change plus a unit test using the three names above if that direction is acceptable.

(Prepared with AI assistance from my own logs and source checkout; the excerpt and the parser check above are from my machine.)

Another client on the same path: LibreChat (agents with MCP servers), Strata 0.2.6 (engine 0.1.51, built from source), Qwen3.8-Flash-Next IQ2_XS.
LibreChat names MCP tools <tool>_mcp_<server>.

In a ~146k-token conversation, on the first turn after the user enabled three servers, the model called server1_label_values, server2_get_objects and server3_search_channels instead of the declared label_values_mcp_server1, get_objects_mcp_server2 and search_channels_mcp_slack. Strata logged:

[strata] 3 <tool_call> block(s) in the answer did not read as a call of a declared tool and went back as text (a name the request did not declare, or a form the template does not use); the names the model wrote: server1_label_values, server2_get_objects, slack_search_channels

LibreChat displayed the raw XML and the turn ended.

The tools were declared: the next turn was served from the same cached prompt prefix, tool list included, and the model called get_objects_mcp_server2 correctly. The names are reordered rather than respelled, so an alias rule like the __ collapse above would not catch this. Until the day before we ran 0.1.43.1, where such a call still reached the client as a tool call.

+1 for returning an undeclared call as a tool call (clients reject unknown tools anyway), or for a machine-readable field such as rejected_tool_calls.

Another data point for this, from a different client: Codex CLI 0.157.1 talking to Strata over the Responses API (wire_api = "responses"), model Qwen3.8-Flash-Next IQ3_S (GSQ-RCO), Strata 0.2.6 / engine 0.1.51, single 24 GB GPU + 94 GB RAM.

What we see: the model sometimes ends a turn with a well-formed <tool_call><function=NAME>...</function></tool_call> block as plain message text, and Codex treats that as the final answer, so the task stops with the work half done. Over a first set of 34 agent sessions, 15 ended this way.

Cause as far as we can tell (from reading the docs/issues, not the engine code): the request declares Codex's tools, and the model emitted a call using a name that is not one of them (it called apply_patch, which Codex does not declare as a tool; patches go through its exec_command tool as a shell command). Per #804/#3075 an undeclared name is not counted as a call, so the block comes back as text. If I read the thread right, 0.1.45 returned these as calls, which let the client answer "unknown tool" and the model recover.

Tools Codex declares in the request (captured by pointing it at a stub endpoint): exec_command (function, argument cmd), write_stdin, request_user_input, view_image, get_goal/create_goal/update_goal, a multi_agent_v1 namespace, and web_search. There is no apply_patch tool.

Mitigation that worked for us (client side): naming the real tool in the task rules ("run every command, including applying a patch, through exec_command; there is no tool called apply_patch") plus, when a session ends on an unexecuted call block, resuming it with a message that names exec_command. On the benchmark where it hit hardest, 2/5 tasks completed before and 5/5 after. It still happens occasionally (resuming recovers it), so a server-side option to return an undeclared-but-well-formed call as a call, or as an error the client can see, would still help.

Other numbers, in case they are useful (small samples, our own tasks, not a standard benchmark):

With parallel: 2, per-lane decode was around 34 tok/s, and 5 of 12 sessions hit the text-call ending, so we went back to a single slot.
Effective throughput over whole agent sessions (including tool time) was roughly 85-130 tok/s on Strata versus 47-90 tok/s for a dense 27B on llama.cpp on the same box.

Happy to share more detail or test a branch if that would help.

serve: a call returned as text for an undeclared name is named in the…

Merge f-027-gh-eng-g (#3935 sm_89 in the DeepSeek CUDA package builds, …

Thank you. Fixed in 0.2.7: A call returned as text for an undeclared name is named in the reply; the collapsed mcp__a__b spelling is the declared tool.

Release: https://github.com/Niko1221/Strata/releases/tag/v0.2.7

fix(engine,serve): port four upstream v0.2.7 fixes

Merge upstream v0.2.7 into memory-core: the two-card and DeepSeek-pic…

fix(serve): 非流式回复补发 strata_rejected_tool_calls 并移除 _with_reasoning 的错位引用

New data point, and a scope question raised as #4622: Pi (earendil-works/pi) hit the same silent stop 6 times in our own session store, 4 spellings (tools__web_search, tools=web_search, bare tools, tools.web_search) plus one undeclared AskUserQuestion. Pi's Tool "..." not found tool-result error is exactly the "answer with the names it has and ask again" recovery your docs describe — and it never fires, because the call never reaches the client. Pi cannot read strata_rejected_tool_calls today either: the SSE partial merge copies content/tool_calls/finish_reason only.

We fixed our side of the trigger (Pi's tool descriptions seeded the tools. token — earendil-works/pi#11573), so this is not a request to revisit the declared-only rule or the no-guessing rule. The ask is a config opt-out for clients that have their own unknown-name policy, and a refusal that does not look like a completed answer (finish_reason stays stop today). Precedent: vLLM's validate_tool_names defaults to off. Noted here so the original thread is not bypassed; a maintainer can decide whether to track it there or in the new issue.

Merge upstream v0.2.8 into memory-core: the Arc-pair and four-card ro…

serve: 宣言していない名前のために文で返した呼び出しを返事で名指しし、__ をつぶした綴りを宣言したツールとして受ける（upstream

serve、setup、tools: upstream Strata 0.2.4〜0.2.7.1 の修正を取り込む（Issue #49、そ…

Navigation Menu

Repository navigation

[Engine] Agent recovery when undeclared tool calls are returned as text #3796

Problem

Environment

Minimal reproduction

Requested guidance

Scope and limitations

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[Engine] Agent recovery when undeclared tool calls are returned as text #3796

Description

Problem

Environment

Minimal reproduction

Requested guidance

Scope and limitations

Activity

thecodacus commented on Oct 24, 2026

lyrakat commented on Oct 25, 2026

wbz183 commented on Oct 26, 2026

npdgm commented on Oct 26, 2026

Lyxcrit commented on Oct 26, 2026

Niko1221 commented on Oct 27, 2026

ackalker commented on Oct 29, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions