Skip to content

fix(gemini): separate same-name parallel tool calls in streams - #8828

Merged
Wei-Shaw merged 4 commits into
Wei-Shaw:mainfrom
little-greenbean:fix/gemini-parallel-tool-calls
Nov 16, 2026
Merged

Wei-Shaw merged 4 commits into
Wei-Shaw:mainfrom
little-greenbean:fix/gemini-parallel-tool-calls

Conversation

@little-greenbean

@little-greenbean little-greenbean commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The Gemini Chat Completions and Messages streaming bridges identify an open tool call by function name. Two independent calls to the same function therefore share one downstream block: different argument objects concatenate into invalid JSON, while identical arguments can be discarded. Non-streaming conversion correctly preserves both calls.

Based on upstream main 530c5e3.

Resulting behavior

  • A changed explicit upstream functionCall.id starts a new block, including same-name, same-argument calls. Each block receives its own downstream index, ID and function name.
  • An ID first received on a continuation is retained for that call.
  • String-valued args select string stream mode immediately, including when the first string is already complete JSON. Matching duplicate/rewind strings and the terminal object snapshot stay in the current call, preserving existing argument-delta behavior and identity when an intermediate part omits its ID.
  • Consuming an object snapshot exits string mode. Independent complete native argument objects remain separate calls, including identical anonymous calls.
  • Chat and Messages use the same boundary state and argument serialization. Messages continues closing each tool block before starting the next.

Non-streaming conversion, native Gemini passthrough, routing, retry and billing behavior are unchanged. Arbitrarily interleaved unfinished calls are outside this change. Without IDs, an identical object immediately following string args cannot be distinguished from a terminal snapshot; it is treated as the snapshot. A following native object is a new call. This assumption is documented and covered by tests.

Before / after

For two same-name native calls with IDs a and b and arguments {"path":"a"} and {"path":"b"}:

Unmodified main This PR
Chat tool indexes 0, 0 0, 1
Downstream call IDs One Two distinct IDs
Aggregated arguments {"path":"a"}{"path":"b"} (invalid JSON) One valid object per call
Messages output One combined tool block Two separate, correctly closed tool blocks

The complete-string regression identified during review is also covered: args string with ID a, matching ID-less args string, then matching args object with ID a now produce one call, preserving main's behavior. The preceding ddb7eac revision incorrectly produced three.

Regression evidence

All fixtures are synthetic native SSE and invoke the actual streaming bridges. Tests aggregate public Chat deltas by index and Messages events by block lifecycle, checking first-delta metadata, distinct call IDs, valid argument JSON, tool finish reasons and cached token usage.

Test set (Chat + Messages) Reference Current result
Original 18 parallel/fragment checks main: 10 fail / 8 pass 18 pass
Four reported complete-string snapshot sequences main: 8 pass; ddb7eac: 8 fail 8 pass
Entire expanded suite, 26 scenarios ddb7eac: 40 pass / 12 fail 52 pass

The expanded suite retains all prior 36 checks and adds complete-string snapshots plus stable-ID, changed-ID, native-call-after-snapshot and late-ID-transition coverage. Main's converters were tested with the same fixtures under Go 1.27.4. The two additional transition scenarios account for four failures beyond the eight reported snapshot regressions.

Validation

For ce026a2 on Go 1.27.4, Linux amd64:

  • PASS: protocol suite, 52/52 checks.
  • PASS: all Gemini service tests under the race detector, including the legacy text-delta checks.
  • PASS: full golangci-lint v2.14.0, 0 issues.
  • PASS: GitHub backend-security, frontend-security, frontend, shell, release helpers, CLA verification and backend unit tests.
  • PASS: GitHub backend integration tests. CI run 43682924566 and security run 43682924635 both completed successfully. PR status: 8 successful checks, 1 workflow-skipped cla-lock, 0 failures.

No live model request, external tool execution, production database access or deployment was used for this correction.

Toolchain security prerequisite

The repository's pinned Go 1.27.3 caused backend-security to fail on the existing net/http vulnerability GO-2026-6859. A separate commit upgrades Go to 1.27.4 and synchronizes backend/go.mod, all four workflow version assertions, all three Docker build images, and DEV_GUIDE.md. The original security gate is preserved; no exclusions are added. The current backend-security run reports no affected vulnerabilities.

@r266-tech

Copy link
Copy Markdown
Contributor

The revised snapshot handling still splits one call when the first string-valued argument chunk is already complete JSON (ddb7eac6330af796c92e940f0112d34c36a24811).

For example, feed these three events, followed by the normal STOP/usage event, to either streaming bridge:

data: {"candidates":[{"content":{"parts":[{"functionCall":{"id":"a","name":"read_file","args":"{\"path\":\"a\"}"}}]}}]}

data: {"candidates":[{"content":{"parts":[{"functionCall":{"name":"read_file","args":"{\"path\":\"a\"}"}}]}}]}

data: {"candidates":[{"content":{"parts":[{"functionCall":{"id":"a","name":"read_file","args":{"path":"a"}}}]}}]}

main@530c5e3a4 emits one tool call. This head emits three, at indexes 0, 1 and 2, with three distinct downstream IDs and the same {"path":"a"} arguments, even though the explicit upstream identity is still a. A client can therefore execute the same call repeatedly.

The first argument string makes seenJSON valid immediately, so observe() leaves fragmented=false. The ID-less duplicate then reaches startsNewBlock()'s final return true, resets the remembered upstream ID, and the final snapshot opens another block.

I reproduced this through handleChatCompletionsStreamingResponseFromGemini and handleStreamingResponse, aggregating their actual public SSE using the PR's index/block-lifecycle checks:

Snapshot sequence main, each protocol This head, each protocol
Complete string, duplicate string; no IDs 1 call 2 calls
Same, ID only on the first string 1 call 2 calls
First ID, ID-less duplicate string, same ID on final object 1 call 3 calls
Complete string, matching final object; no IDs 1 call 2 calls

All eight snapshot checks pass on main and fail here. All four control scenarios pass here: stable IDs on every string, initially incomplete fragments plus final object, distinct explicit IDs, and independent anonymous native objects. The first two controls also pass on main; the latter two confirm the intended parallel-call improvement. The PR's existing 36 protocol checks and TestGeminiTextDeltaMatchesLegacy pass on this head.

Could the state distinguish the string-argument stream mode even when its initial chunk is already valid, while preserving the new boundary for independent native object calls and changed explicit IDs?

AI-only local investigation on macOS/Go 1.27.4. Inputs are synthetic compatibility-stream fixtures; no live Gemini request, tool execution, database, or deployment was tested.

@r266-tech

Copy link
Copy Markdown
Contributor

Retested at ce026a2cbf1e1d366ddd64fe098e8a0334949a7b with the unchanged probe from my earlier comment: the 8 reported snapshot checks and all 8 controls now pass across Chat Completions and Messages. The complete-string duplicate, reappearing ID, and terminal object sequences each emit one call; distinct explicit IDs and independent anonymous native objects still emit separate calls.

The PR’s current 52 protocol checks and TestGeminiTextDeltaMatchesLegacy also pass locally:

go test -tags=unit ./internal/service -run '^(TestReview8828CompleteStringSnapshots|TestGeminiParallelToolStreams|TestGeminiTextDeltaMatchesLegacy)$' -count=1 -v

This resolves the specific duplicate-call regression I reported. AI-only local verification on macOS/Go 1.27.4 using synthetic upstream SSE; no live Gemini request or tool execution was tested.

@Wei-Shaw
Wei-Shaw merged commit 46b8792 into Wei-Shaw:main Nov 16, 2026
9 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 16, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants