Repository navigation
Conversation
…not a malformed call
A reply such as "I will use the `<tool_call>` format now." followed by the call raised
ValueError("malformed tool call") in OutputParser.feed and ended the request: everything after the
first <tool_call> was taken as the call body. A call is the tag followed (after whitespace) by
<function=, as the chat template writes it; the tag with anything else after it is now content, and
it is held while streaming until its follower arrives, so streamed and whole output agree.
Found while porting tolerant tool-call recovery from a Qwen engine (ninfer --tolerant-tool-calls):
of its three recovery cases, Strata already handled values that quote the markup (Niko1221#210) and prose
after the calls; this was the one that failed. Tests: ToolCallTagInProse (fails on main with 21
subtest errors, passes here); python -m unittest serve.test_server: 120 tests OK.
Owner
|
Merged for 0.1.40: a <tool_call> named in prose before the real call is content, not a malformed call. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
OutputParser.feedtreated every<tool_call>in the content as the start of a call. A reply that names the tag in its prose before making the real call, for exampletook everything after the first tag as the call body, and
parse_tool_callraisedValueError("malformed tool call: ..."), which ended the request.A call is the tag followed (after whitespace) by
<function=, which is howserve/chat_template.jinjawrites it. With this change, a<tool_call>followed by anything else is content. While streaming, the tag is held until its follower arrives (the same way a partial tag is held), so streamed and whole output give the same events. A tag at the very end of the output is flushed as content byfinish(), as before.How it was found
I was porting tolerant tool-call recovery from a Qwen engine (ninfer's
--tolerant-tool-calls) and checked its three recovery cases against Strata's parser:</parameter>/<tool_call><tool_call>named in prose before the real callValueError, request endsOnly the third needed a change.
Tests
ToolCallTagInProseinserve/test_server.py: three prose forms (backticks, bare, two tags),stream_toolson and off, steps 1 / 7 / whole; plus a tag alone at the end of the output.mainthe new class fails (3 failures, 18 errors across subtests); with the change it passes.python -m unittest serve.test_server: 120 tests OK (Linux, Python 3.14).