Skip to content

fix(relay): mark Responses streams done on terminal event to avoid client_gone mislabel - #6904

Closed
lichengzhe wants to merge 1 commit into
QuantumNous:mainfrom
lichengzhe:fix/responses-stream-done-on-completed
Closed

lichengzhe wants to merge 1 commit into
QuantumNous:mainfrom
lichengzhe:fix/responses-stream-done-on-completed

Conversation

@lichengzhe

@lichengzhe lichengzhe commented Aug 17, 2026 •

Copy link
Copy Markdown
Contributor

📝 变更描述 / Description

/v1/responses 的 SSE 流没有 data: [DONE] 哨兵:上游发完 response.completed 后就静默,直到它自己关闭连接。OaiResponsesStreamHandler 此前从不向 scanner 报告"正常完成",所以 stream_status.end_reason 由两个事件赛跑决定——上游 EOF 先到记 eof,客户端先关 socket 就记 client_gone。codex CLI 这类客户端收到 response.completed 后立即关连接,遇到收尾慢的上游时每次都输掉这场竞速,正常完成的流被整批标成 client_gone(#6649;评论区提到的 "codex + azure gpt-5.6 必现" 就是这个形状)。

本 PR 在 response.completed / response.done 分支调用 sr.Done():终止原因在事件到达的那一刻记录,scanner 随即停止,不再等上游 EOF。附带的好处是提前释放对收尾迟缓上游的读等待。

与 #6808 的关系:互补,不冲突。#6808 允许正常完成(Done/EOF)纠正先写入的 client_gone,但对 /v1/responses 客户端先断开时,cleanup 强制关闭 body 会让 scanner 以 forced-close error 结束而非干净 EOF,纠正路径无从触发(见 #6808 评论区 walker1211 的分析)。本 PR 让 handler 在竞速开始前就把 done 写进去,两个 PR 合起来才完整覆盖。

response.failed / response.incomplete / response.cancelled 分支保持原状(它们不是"正常完成",语义上是否该停 scanner 可另行讨论)。

复现数据(生产环境,self-hosted v1.0.0-rc.24)

同一个 codex 0.147.0 客户端、同一时段、两条渠道对照:

上游 最后事件 → EOF 间隔 codex 流量被标 client_gone
聚合商(Azure 系后端) ~0.25s ~100%(含全部实际成功的请求)
本机反代(即时关流) ~0.04s 0.4–2%

同请求用 curl(读到 EOF)得 eof,用 codex(completed 后即关)得 client_gone,HTTP 200、内容与计费完全一致。

测试 / Tests

  • 新增 TestOaiResponsesStreamHandlerMarksDoneOnCompleted:管道模拟上游,发 response.completed 后保持连接不关,断言 handler 立即返回、end_reason=done、usage 正确、终止事件仍转发给客户端。该测试在未修复的 main 上失败(挂到超时),修复后通过。
  • gofmt 干净,go test ./relay/... 全量通过。

🤖 Generated with Claude Code

Responses SSE has no "data: [DONE]" sentinel, so OaiResponsesStreamHandler
never signaled completion to the stream scanner: the end reason was decided
by a race between upstream EOF and the client closing its socket right after
the final event. Clients like codex CLI close immediately after
response.completed, and upstreams that linger before EOF lose that race
every time, mislabeling finished streams as client_gone (QuantumNous#6649). Calling
sr.Done() on response.completed / response.done records the terminal reason
at the moment it is known and stops the scanner without waiting for EOF.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

An error occurred during the review process. Please try again later.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Calcium-Ion
Calcium-Ion force-pushed the main branch 2 times, most recently from 51fdfc5 to 2b6f1df Compare August 30, 2026 15:03
Calcium-Ion added a commit that referenced this pull request Oct 22, 2026
…ct errors

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from #6904 and #6951 and narrows the
cancellation reconciliation proposed in #6808. Refs #6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
Calcium-Ion added a commit that referenced this pull request Oct 22, 2026
…ct errors (#7956)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from #6904 and #6951 and narrows the
cancellation reconciliation proposed in #6808. Refs #6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
@Calcium-Ion

Copy link
Copy Markdown
Member

已整合至 #7956 并合并,因此关闭此 PR。感谢贡献,合并提交已保留 co-author 署名。

cicioln pushed a commit to cicioln/new-api that referenced this pull request Oct 25, 2026
…ct errors (QuantumNous#7956)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the
cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
(cherry picked from commit 0c2af97)
kyuhwi-choi pushed a commit to attodev/new-api that referenced this pull request Oct 30, 2026
…ct errors (QuantumNous#7956)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the
cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
(cherry picked from commit 0c2af97)
lizhongyi1209 pushed a commit to lizhongyi1209/new-api that referenced this pull request Nov 1, 2026
…ct errors (QuantumNous#7956)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the
cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
woncc added a commit to 86669666/new-api that referenced this pull request Nov 1, 2026
…ct errors (QuantumNous#7956) (#92)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the
cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649.

Co-authored-by: Calcium-Ion <i@caion.me>
Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
timwenx pushed a commit to timwenx/new-api that referenced this pull request Nov 18, 2026
…ct errors (QuantumNous#7956)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the
cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants