Repository navigation
fix(relay): mark Responses streams done on terminal event to avoid client_gone mislabel - #6904
Closed
lichengzhe wants to merge 1 commit into
Closed
lichengzhe wants to merge 1 commit into
lichengzhe wants to merge 1 commit into
Conversation
Responses SSE has no "data: [DONE]" sentinel, so OaiResponsesStreamHandler never signaled completion to the stream scanner: the end reason was decided by a race between upstream EOF and the client closing its socket right after the final event. Clients like codex CLI close immediately after response.completed, and upstreams that linger before EOF lose that race every time, mislabeling finished streams as client_gone (QuantumNous#6649). Calling sr.Done() on response.completed / response.done records the terminal reason at the moment it is known and stops the scanner without waiting for EOF. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
8 of 11 tasks
Contributor
|
Caution Review failedAn error occurred during the review process. Please try again later. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
7 of 11 tasks
7 tasks done
Calcium-Ion
force-pushed
the
main
branch
2 times, most recently
from
August 30, 2026 15:03
51fdfc5 to
2b6f1df
Compare
7 tasks done
8 of 12 tasks
10 tasks done
Calcium-Ion
added a commit
that referenced
this pull request
Oct 22, 2026
…ct errors Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from #6904 and #6951 and narrows the cancellation reconciliation proposed in #6808. Refs #6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
Calcium-Ion
added a commit
that referenced
this pull request
Oct 22, 2026
…ct errors (#7956) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from #6904 and #6951 and narrows the cancellation reconciliation proposed in #6808. Refs #6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
Member
|
已整合至 #7956 并合并,因此关闭此 PR。感谢贡献,合并提交已保留 co-author 署名。 |
cicioln
pushed a commit
to cicioln/new-api
that referenced
this pull request
Oct 25, 2026
…ct errors (QuantumNous#7956) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com> (cherry picked from commit 0c2af97)
kyuhwi-choi
pushed a commit
to attodev/new-api
that referenced
this pull request
Oct 30, 2026
…ct errors (QuantumNous#7956) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com> (cherry picked from commit 0c2af97)
lizhongyi1209
pushed a commit
to lizhongyi1209/new-api
that referenced
this pull request
Nov 1, 2026
…ct errors (QuantumNous#7956) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
woncc
added a commit
to 86669666/new-api
that referenced
this pull request
Nov 1, 2026
…ct errors (QuantumNous#7956) (#92) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649. Co-authored-by: Calcium-Ion <i@caion.me> Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
timwenx
pushed a commit
to timwenx/new-api
that referenced
this pull request
Nov 18, 2026
…ct errors (QuantumNous#7956) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 变更描述 / Description
/v1/responses的 SSE 流没有data: [DONE]哨兵:上游发完response.completed后就静默,直到它自己关闭连接。OaiResponsesStreamHandler此前从不向 scanner 报告"正常完成",所以stream_status.end_reason由两个事件赛跑决定——上游 EOF 先到记eof,客户端先关 socket 就记client_gone。codex CLI 这类客户端收到response.completed后立即关连接,遇到收尾慢的上游时每次都输掉这场竞速,正常完成的流被整批标成client_gone(#6649;评论区提到的 "codex + azure gpt-5.6 必现" 就是这个形状)。本 PR 在
response.completed/response.done分支调用sr.Done():终止原因在事件到达的那一刻记录,scanner 随即停止,不再等上游 EOF。附带的好处是提前释放对收尾迟缓上游的读等待。与 #6808 的关系:互补,不冲突。#6808 允许正常完成(
Done/EOF)纠正先写入的client_gone,但对/v1/responses客户端先断开时,cleanup 强制关闭 body 会让 scanner 以 forced-close error 结束而非干净 EOF,纠正路径无从触发(见 #6808 评论区 walker1211 的分析)。本 PR 让 handler 在竞速开始前就把done写进去,两个 PR 合起来才完整覆盖。response.failed/response.incomplete/response.cancelled分支保持原状(它们不是"正常完成",语义上是否该停 scanner 可另行讨论)。复现数据(生产环境,self-hosted v1.0.0-rc.24)
同一个 codex 0.147.0 客户端、同一时段、两条渠道对照:
同请求用 curl(读到 EOF)得
eof,用 codex(completed 后即关)得client_gone,HTTP 200、内容与计费完全一致。测试 / Tests
TestOaiResponsesStreamHandlerMarksDoneOnCompleted:管道模拟上游,发response.completed后保持连接不关,断言 handler 立即返回、end_reason=done、usage 正确、终止事件仍转发给客户端。该测试在未修复的main上失败(挂到超时),修复后通过。gofmt干净,go test ./relay/...全量通过。🤖 Generated with Claude Code