Skip to content

fix(antigravity): stream Gemini compat thoughts and anchor pre-content timeout after headers - #8673

Merged
Wei-Shaw merged 1 commit into
Wei-Shaw:mainfrom
baryon:contrib/antigravity-compat-gemini-stream-timeout
Nov 16, 2026
Merged

Wei-Shaw merged 1 commit into
Wei-Shaw:mainfrom
baryon:contrib/antigravity-compat-gemini-stream-timeout

Conversation

@baryon

@baryon baryon commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Large-context streaming requests to Gemini models through Antigravity's OpenAI-compatible endpoints (/v1/responses and /v1/chat/completions -> /v1internal:streamGenerateContent) hit two avoidable stream_timeout / premature-abort failure modes:

  1. Missing applyGeminiCompatThinkingConfig (includeThoughts: true) on streaming compatibility calls:
    buildAntigravityCompatGeminiBody converted Claude-shaped intermediate payloads via convertClaudeMessagesToGeminiGenerateContent without calling applyGeminiCompatThinkingConfig (unlike GeminiMessagesCompatService and GeminiChatCompletionsCompatService).

    • When a high-thinking Gemini model (including tier-suffixed models such as gemini-4-*-high or gemini-3.8-flash-high invoked without an explicit reasoning.effort field) emits a short initial response and then enters a long interleaved thinking phase (~8k+ thought tokens on large uncached prompts) before a tool call, Google's v1internal endpoint buffers all thought tokens server-side (thoughtsTokenCount > 0 with zero streamed thought parts), leaving the SSE connection completely silent past gateway.stream_data_interval_timeout (default 180s) and terminating the stream with stream_timeout.
    • In addition, prepareAntigravityCompatCall derived bare Gemini thinking variants solely from claudeRequest.Thinking; because ResponsesToAnthropicRequest maps effort=low/minimal/none into output_config.effort while leaving thinking == nil, bare Gemini models with effort="low" fell through to -high instead of -low.
  2. preContentDeadline counted HTTP response-header wait time against the SSE stream pre-content window:
    handleAntigravityCompatStream computed preContentDeadline from call.request.startTime (recorded before antigravityRetryLoop, where waiting for HTTP response headers is already bounded by gateway.response_header_timeout = 600s) and capped maxPreContentWait at 2 minutes even when gateway.stream_data_interval_timeout defaults to 180s. Any request whose header wait exceeded 2 minutes (e.g., 130s–300s prefill on large uncached prompts, or retries) was immediately aborted on arrival of HTTP 200 (stream_read_ms = 0) as an empty stream / pre-content stream timeout.

What Changed

  • backend/internal/service/antigravity_gateway_compat.go:
    • Reuse applyGeminiCompatThinkingConfig(body, claudeBody, mappedModel, claudeRequest != nil && claudeRequest.Stream) in buildAntigravityCompatGeminiBody (matching GeminiChatCompletionsCompatService), enabling includeThoughts: true on streaming requests while keeping non-streaming requests free of forced thought summaries and keeping tier-suffixed model IDs free of redundant thinkingLevel fields.
    • Resolve bare Gemini variant tiers via geminiThinkingLevelFromClaudeBody(request.claudeBody, claudeRequest.Thinking) so output_config.effort (low/minimal/none/medium/high) is honored even when claudeRequest.Thinking is nil.
  • backend/internal/service/antigravity_gateway_compat_stream.go:
    • Anchor preContentDeadline at SSE stream start (when HTTP 200 headers arrive) with max(2m, StreamDataIntervalTimeout) while preserving call.request.startTime for FirstTokenMs (TTFT) measurement, and still bounding comment-only SSE streams.
  • backend/internal/service/antigravity_gateway_compat_test.go:
    • Add unit tests covering tier-suffixed & bare Gemini thinkingConfig contracts (streaming vs non-streaming, effort="low" variant selection, explicit bare model thinkingLevel), interleaved thought chunks resetting StreamDataIntervalTimeout, and post-header stream completion after >2m header waits.

Related Issues & PRs

Validation

  • cd backend && go test -tags=unit ./internal/pkg/antigravity/... -count=1
  • cd backend && go test -tags=unit ./internal/service -run "^(TestAntigravity|TestGemini|TestApplyGeminiCompat)" -count=1
  • cd backend && go test -race -tags=unit ./internal/service -run "^TestAntigravityCompat(GeminiThinkingConfig|HeaderWaitExceedingPreContentLimitDoesNotAbortStream|InterleavedThoughtChunksResetIdleTimeout)$" -count=1
  • cd backend && go build ./...
  • cd backend && golangci-lint run --timeout=5m ./internal/service/... (0 issues)

@Wei-Shaw
Wei-Shaw merged commit 8da0388 into Wei-Shaw:main Nov 16, 2026
8 of 9 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 16, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants