Repository navigation
fix(gemini): keep same-name parallel tool calls separate in streaming bridges - #8676
junhoeKim-kkobi wants to merge 1 commit into
Conversation
… bridges Gemini returns every functionCall part complete (name plus full args) unless argument streaming is requested, which these bridges never do. The /v1/messages and /v1/chat/completions streaming bridges kept one tool_use block open for consecutive parts with the same name, so parallel calls to one function were merged into a single call whose arguments concatenated several JSON objects, and an identical second call was dropped. A same-name part now continues the open block only while its arguments are still an unfinished JSON fragment or when a later chunk repeats the finished call verbatim (cumulative stream); otherwise a new tool_use block starts.
|
Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA). To sign, please reply with the following comment:
You only need to sign once — it will be valid for all your future contributions to this project. I have read the CLA Document and I hereby sign the CLA You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
|
哦我也遇到了这个问题,都四天了咋还没人来处理 |
|
感谢贡献!同名并行工具调用在流式桥接中被合并的问题,已由修复相同问题的 #8828 合入 main 解决,本 PR 作为重复关闭。如果你的实现覆盖了 #8828 没有处理的场景,欢迎基于最新 main 另开 PR 说明。 Thanks! This is a duplicate of #8828 (now merged), which fixes the same same-name parallel tool call issue in streaming bridges. If your version covers a case #8828 misses, please open a new PR against the latest main. |
Summary
Gemini returns every
functionCallpart complete (name plus fullargs); arguments only stream in pieces whenstreamFunctionCallArgumentsis requested, which these bridges never do.The streaming bridges for
/v1/messages(GeminiMessagesCompatService.handleStreamingResponse) and/v1/chat/completions(handleChatCompletionsStreamingResponseFromGemini) keep onetool_useblock open for consecutive parts with the same function name ("If we receive streamed tool args in pieces, keep a single tool block open"). Parallel calls to one function are therefore merged:tool_calls[0](one id) whoseargumentsis{"path":"a"}{"path":"b"}{"path":"c"};tool_useblock with the same concatenatedinput_json_delta;geminiTextDelta.nexttreats it as a duplicate.Calls to different functions are unaffected, so this only shows up when an agent fans out the same tool (e.g. several
readcalls in one turn). Agents that validate tool arguments then fail with errors such asUnexpected trailing characters. The same model streams correctly through Antigravity accounts, whose transformer already opens one block perfunctionCallpart (pkg/antigravity/stream_transformer.go,processFunctionCall), and the non-streaming conversion returns separate calls too.Fix
New helper
geminiFunctionCallContinuesOpenTool: a same-name part continues the open block only while the open block's arguments are still an unfinished JSON fragment (an upstream streaming args in pieces) or when a later chunk repeats the finished call verbatim (a cumulative stream). Otherwise the open block is closed and a newtool_useblock with its own id starts. Both behaviours the original comment was guarding are kept.Reproduction
Gemini API-key (AI Studio) account, v0.2.21. Send a streaming request with a single tool
read(path)and ask the model to call it three times in parallel:/v1/chat/completions(stream: true): onetool_callsentry, index 0, one id, three JSON objects concatenated inarguments;/v1/messages(stream: true): onetool_usecontent block with the concatenated input;stream: false: three separate tool calls.Tests
TestGeminiMessagesHandleStreamingResponse_SameNameToolCallsTestGeminiChatCompletionsStreamingSameNameToolCallsBoth run the five same-name streams from the table (shared table
geminiSameNameToolCallStreams). They assert one tool call per expected call, distinct ids, and arguments equal to each call's JSON. The messages test also asserts that content blocks never overlap.Local checks:
gofmtis clean, and a standalone harness of the new decision rule reproduces the table above. I did not run theinternal/servicetest suite locally, so this relies on CI.