fix(opencode): make long-lived provider streams robust to silent SSE terminations - #39970
Open
hubert-marek wants to merge 2 commits into
Open
fix(opencode): make long-lived provider streams robust to silent SSE terminations#39970hubert-marek wants to merge 2 commits into
hubert-marek wants to merge 2 commits into
Conversation
…inations Three related defects around long-lived SSE completion streams: - The chunkTimeout provider option was only enforced by the fetch-level SSE wrapper, which watches raw bytes: keepalive comments reset its timer while the parsed model stream delivers nothing, and a half-open connection could stall a session indefinitely. The deadline is now also enforced where the AI SDK fullStream is consumed, disarmed while locally executed tool calls are outstanding, and aborts the request on expiry as a terminal provider error. - A stream that hit EOF without a finish frame surfaced as the AI SDK's synthesized finish-step (reason "other"/"unknown", no raw finish reason, zero usage) and concluded the turn as if it had completed, silently dropping the rest of the response. It is now a retryable provider error with a bounded in-turn retry budget; exhausting the budget fails the turn with a nonzero exit instead of exiting 0 with a truncated turn. - Provider rejections whose SDK message is a single generic sentence (for example a bare "Invalid request.") discarded the response body during normalization, making 400-class failures undiagnosable. The bounded body is now retained in the surfaced error message. Co-authored-by: Cursor <cursoragent@cursor.com>
This was referenced Jul 31, 2026
This was referenced Aug 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #39968
Type of change
What does this PR do?
Fixes the three defects described in #39968, observed when a gateway terminates or stalls long-lived SSE completion responses (23 bare-EOF terminations across 13,312 requests in one ~12h run, plus one 9.5h half-open stall):
EOF without a finish frame is no longer a completed turn. When a provider stream ends without a finish frame, the AI SDK synthesizes a fallback finish-step whose signature is unambiguous: unified reason
"other",rawFinishReason: undefined, and null usage (a real finish frame always carries the provider's raw reason, and unusual-but-real reasons carry usage).LLMAISDK.streamFailureconverts exactly that signature into a new retryableProviderError.StreamIncompleteErrorbefore adaptation, so the session-level retry policy retries the request within the turn. The budget is bounded (STREAM_INCOMPLETE_RETRY_LIMIT = 3, existing backoff) because a provider that truncates every response must fail the turn (nonzero exit) rather than retry forever or exit 0 with half a turn. Legitimateunknownfinish reasons (unrecognized raw reasons from real finish frames) flow through unchanged.chunkTimeoutnow bounds the parsed-chunk gap. The existing fetch-levelwrapSSEtimer watches raw bytes, so keepalive comments reset it while the model stream delivers nothing, and stalls above the byte layer never trip it.LLMAISDK.boundChunkGapswraps fullStream consumption insession/llm.tswith a per-event deadline from the samechunkTimeoutprovider option (no new config). The deadline is disarmed while locally executed tool calls are outstanding — their results arrive on the same stream and a long-running local tool is not a stalled provider. On expiry it aborts the request's AbortController and fails the stream withProviderError.ResponseStreamError, surfaced like any provider failure. I considered the AI SDK's nativetimeout: { chunkMs }instead, but its timer spans tool execution inside each step (resetChunkTimeoutonly ticks on stream chunks and is cleared at step flush), which would abort any local tool that runs longer than the chunk deadline.Provider error bodies survive normalization.
ProviderError.message()previously returned the SDK message alone whenever it differed from the bare status text, discardingresponseBody— which is how a 400 surfaced as justInvalid request.. It now appends the response body, bounded to 2000 chars, skipping HTML bodies and bodies already contained in the message; the existing behavior for empty messages, status-text messages, and the HTML 401/403 hints is unchanged.Two existing tests (
unknown stream finish preserves partial output and exits 0and its--format jsontwin) pinned the old exit-0-on-truncation behavior and are updated to the new contract: retry within the turn (exit 0 once the retry succeeds, with the partial output preserved), and a new test asserts nonzero exit when every retry is truncated too.How did you verify your code works?
New regression tests:
test/session/llm.test.ts: mock SSE server streaming partial content then bare EOF → stream fails withStreamIncompleteError; stream that stalls behind 50ms SSE keepalives withchunkTimeout: 400→ aborted at ~400ms with the deadline error (raw-byte timers provably can't catch this one); a local tool call running longer thanchunkTimeout→ no false abort;streamFailureunit cases (raw-reason/usage-carrying finish frames are kept).test/session/processor-effect.test.ts: truncated stream is retried within the turn and the retry's output completes the turn without error.test/session/retry.test.ts: incomplete-stream errors map to retryableAPIErrorwith the marker code; the retry schedule stops afterSTREAM_INCOMPLETE_RETRY_LIMIT; other retryable errors stay unbounded.test/cli/run/run-process.test.ts: subprocess-level — truncated stream retries and exits 0 when the retry succeeds; four consecutive truncations exit nonzero with the finish-frame error on stderr.test/session/message-v2.test.ts: 400 with a JSON body keepsInvalid request.plus the body'scode/paramin the surfaced message; a 10KB body is truncated with a marker.Full
packages/opencodesuite: 3181 pass / 1 fail — the one failure istool.write > sets file permissions when writing sensitive data, which fails identically on the untouched v1.18.2 checkout in this environment (umask-dependent), unrelated to this change.bun run test:httpapigates: 208 pass / 0 fail.bun turbo typecheck: 30/30 packages pass.oxlint: 0 errors.Screenshots / recordings
Not a UI change.
Checklist
Made with Cursor