Skip to content

fix(opencode): make long-lived provider streams robust to silent SSE terminations - #39970

Open
hubert-marek wants to merge 2 commits into
anomalyco:devfrom
hubert-marek:fix/provider-stream-idle-timeout
Open

fix(opencode): make long-lived provider streams robust to silent SSE terminations#39970
hubert-marek wants to merge 2 commits into
anomalyco:devfrom
hubert-marek:fix/provider-stream-idle-timeout

Conversation

@hubert-marek

Copy link
Copy Markdown

Issue for this PR

Closes #39968

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Fixes the three defects described in #39968, observed when a gateway terminates or stalls long-lived SSE completion responses (23 bare-EOF terminations across 13,312 requests in one ~12h run, plus one 9.5h half-open stall):

  1. EOF without a finish frame is no longer a completed turn. When a provider stream ends without a finish frame, the AI SDK synthesizes a fallback finish-step whose signature is unambiguous: unified reason "other", rawFinishReason: undefined, and null usage (a real finish frame always carries the provider's raw reason, and unusual-but-real reasons carry usage). LLMAISDK.streamFailure converts exactly that signature into a new retryable ProviderError.StreamIncompleteError before adaptation, so the session-level retry policy retries the request within the turn. The budget is bounded (STREAM_INCOMPLETE_RETRY_LIMIT = 3, existing backoff) because a provider that truncates every response must fail the turn (nonzero exit) rather than retry forever or exit 0 with half a turn. Legitimate unknown finish reasons (unrecognized raw reasons from real finish frames) flow through unchanged.

  2. chunkTimeout now bounds the parsed-chunk gap. The existing fetch-level wrapSSE timer watches raw bytes, so keepalive comments reset it while the model stream delivers nothing, and stalls above the byte layer never trip it. LLMAISDK.boundChunkGaps wraps fullStream consumption in session/llm.ts with a per-event deadline from the same chunkTimeout provider option (no new config). The deadline is disarmed while locally executed tool calls are outstanding — their results arrive on the same stream and a long-running local tool is not a stalled provider. On expiry it aborts the request's AbortController and fails the stream with ProviderError.ResponseStreamError, surfaced like any provider failure. I considered the AI SDK's native timeout: { chunkMs } instead, but its timer spans tool execution inside each step (resetChunkTimeout only ticks on stream chunks and is cleared at step flush), which would abort any local tool that runs longer than the chunk deadline.

  3. Provider error bodies survive normalization. ProviderError.message() previously returned the SDK message alone whenever it differed from the bare status text, discarding responseBody — which is how a 400 surfaced as just Invalid request.. It now appends the response body, bounded to 2000 chars, skipping HTML bodies and bodies already contained in the message; the existing behavior for empty messages, status-text messages, and the HTML 401/403 hints is unchanged.

Two existing tests (unknown stream finish preserves partial output and exits 0 and its --format json twin) pinned the old exit-0-on-truncation behavior and are updated to the new contract: retry within the turn (exit 0 once the retry succeeds, with the partial output preserved), and a new test asserts nonzero exit when every retry is truncated too.

How did you verify your code works?

New regression tests:

  • test/session/llm.test.ts: mock SSE server streaming partial content then bare EOF → stream fails with StreamIncompleteError; stream that stalls behind 50ms SSE keepalives with chunkTimeout: 400 → aborted at ~400ms with the deadline error (raw-byte timers provably can't catch this one); a local tool call running longer than chunkTimeout → no false abort; streamFailure unit cases (raw-reason/usage-carrying finish frames are kept).
  • test/session/processor-effect.test.ts: truncated stream is retried within the turn and the retry's output completes the turn without error.
  • test/session/retry.test.ts: incomplete-stream errors map to retryable APIError with the marker code; the retry schedule stops after STREAM_INCOMPLETE_RETRY_LIMIT; other retryable errors stay unbounded.
  • test/cli/run/run-process.test.ts: subprocess-level — truncated stream retries and exits 0 when the retry succeeds; four consecutive truncations exit nonzero with the finish-frame error on stderr.
  • test/session/message-v2.test.ts: 400 with a JSON body keeps Invalid request. plus the body's code/param in the surfaced message; a 10KB body is truncated with a marker.

Full packages/opencode suite: 3181 pass / 1 fail — the one failure is tool.write > sets file permissions when writing sensitive data, which fails identically on the untouched v1.18.2 checkout in this environment (umask-dependent), unrelated to this change. bun run test:httpapi gates: 208 pass / 0 fail. bun turbo typecheck: 30/30 packages pass. oxlint: 0 errors.

Screenshots / recordings

Not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Made with Cursor

opencode and others added 2 commits July 15, 2026 16:15
…inations

Three related defects around long-lived SSE completion streams:

- The chunkTimeout provider option was only enforced by the fetch-level SSE
  wrapper, which watches raw bytes: keepalive comments reset its timer while
  the parsed model stream delivers nothing, and a half-open connection could
  stall a session indefinitely. The deadline is now also enforced where the
  AI SDK fullStream is consumed, disarmed while locally executed tool calls
  are outstanding, and aborts the request on expiry as a terminal provider
  error.

- A stream that hit EOF without a finish frame surfaced as the AI SDK's
  synthesized finish-step (reason "other"/"unknown", no raw finish reason,
  zero usage) and concluded the turn as if it had completed, silently
  dropping the rest of the response. It is now a retryable provider error
  with a bounded in-turn retry budget; exhausting the budget fails the turn
  with a nonzero exit instead of exiting 0 with a truncated turn.

- Provider rejections whose SDK message is a single generic sentence (for
  example a bare "Invalid request.") discarded the response body during
  normalization, making 400-class failures undiagnosable. The bounded body
  is now retained in the surfaced error message.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Silent SSE terminations: EOF without a finish frame completes the turn, chunkTimeout misses stalled streams, provider error bodies are discarded

1 participant