Skip to content

fix(relay): 正确结束已完成的 Responses 流 - #7956

Merged
Calcium-Ion merged 1 commit into
mainfrom
codex/fix-responses-stream-completion
Oct 22, 2026
Merged

Calcium-Ion merged 1 commit into
mainfrom
codex/fix-responses-stream-completion

Conversation

@Calcium-Ion

@Calcium-Ion Calcium-Ion commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

🔗 关联任务 / Related Issue

🚀 变更类型 / Type of change

  • 🐛 Bug 修复 (Bug fix)

📝 变更描述 / Description

Responses 已发送完整完成事件、客户端在上游 EOF 前断开时,原代码仍记录 client_gone / error,并将健康样本判为 ignored。现在成功发送完成事件后主动结束扫描;所有协程退出后,只纠正明确完成且无错误的并发取消。裸 EOF、真正中途断连、协议错误以及发送失败不被改成成功。

保留当前 usage 累积逻辑,补齐 SSE 写入和 flush 错误传递,避免正常收尾关闭上游 Body 时重复记录 scanner 错误。回归集中在现有 Responses 测试文件,覆盖 15 个完成、断连、写入失败和协议终态场景,并核对 token 和图片调用计数。

由 Codex 按维护者授权整合并复核。参考 #6904 / #6951 的完成事件处理及 #6808 的竞态分析;提交包含 leecz、zkasuran、yinshaohua 的 Co-authored-by。

📸 运行证明 / Proof of Work

环境:macOS arm64,Go 1.26.10,基于 7d8b9126 的独立 worktree。

  • go test ./common ./relay/common ./relay/helper ./relay/channel/openai ./relay/channel/claude ./relay/channel/gemini ./relay/channel/dify ./service ./pkg/perf_metrics -count=1 -timeout=3m:8 个有测试的包通过;dify 无测试文件。
  • go test -race -parallel=1 ./relay/channel/openai ./relay/helper ./relay/common -run 'Test(OaiResponsesStreamHandlerCompletionLifecycle|StreamScannerHandler_|StreamStatus_)' -count=1 -timeout=2m:3 个包通过;仍保留每个请求内部协程的 race 检测。
  • 本机 HTTP/1.1 SSE 对照:临时复现程序 go test -race -v -run '^TestIssue6649HTTPClientDisconnect$' -count=1 -timeout=30s /private/tmp/new-api-6649-hgud6g0c/fixed-repro_test.go 通过。客户端完整收到完成事件后关闭连接,由 client_gone / error / ignored 变为 done / ok / success,usage 保持 7+5=12;中途断开仍为 client_gone / error / ignored。
  • go build ./...、go vet ./common ./relay/helper ./relay/channel/openai、git diff --check:通过。构建所需 web/dist 复用现有构建产物;无前端源码改动。

限制:默认并行的 helper -race 检查触发已有 logger/logger.go:114 的 logCount 竞态,使用未修改主干的 Go overlay 已复现相同告警;本 PR 不改日志计数器。未运行官方镜像或真实供应商请求,无数据库结构或查询变更。

✅ 提交前检查项 / Checklist

  • 人工确认: 人工审阅由维护者确认,本次不代勾。
  • 非重复提交: 已核对三个旧 PR 的实际 diff,本 PR 是维护者要求的整合替代。
  • 新功能关联 Issue: 不适用,修复关联 stream_status.end_reason shows "client_gone" for streams that completed normally (SSE scanner race) #6649。
  • 事前沟通: 维护者已要求整合并保留贡献者署名。
  • 功能范围: 通用 Responses SSE 收尾,不修改 Codex 渠道类型或第三方协议。
  • 非透传: 处理正常解析路径,不针对整个请求体透传。
  • 范围聚焦: 仅流收尾、发送错误传播及回归测试。
  • 本地验证: 命令与对照结果如上。
  • 安全合规: 无凭据、鉴权或数据库变更。

Summary by CodeRabbit

  • Bug Fixes
    • Streaming write and flush failures are now reported instead of being silently ignored.
    • Completed streams are distinguished from client disconnects, while incomplete responses and interrupted streams retain accurate end statuses.
    • Errors encountered while reading or forwarding streamed responses are handled more consistently, improving stream status and billing accuracy.

@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: da0fa8d7-5def-4401-8484-08519693b533
📥 Commits

Reviewing files that changed from the base of the PR and between 7d8b912 and b77674b.

📒 Files selected for processing: 7
  • common/custom-event.go
  • relay/channel/openai/helper.go
  • relay/channel/openai/relay_responses.go
  • relay/channel/openai/relay_responses_billing_test.go
  • relay/helper/common.go
  • relay/helper/stream_result.go
  • relay/helper/stream_scanner.go
💤 Files with no reviewable changes: 1
  • relay/channel/openai/helper.go

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

SSE writes, event rendering, and flushes now return errors. Stream handling records explicit completion and preserves the correct end reason when cancellation or read errors occur during cleanup. The Responses stream handler records events before forwarding them and marks qualifying completed responses as done.

Changes

SSE stream lifecycle

Layer / File(s) Summary
SSE write and flush error propagation
common/custom-event.go, relay/helper/common.go
SSE data writes and flushes return available errors. Event rendering returns the first render error.
Scanner cancellation and completion status
relay/helper/stream_result.go, relay/helper/stream_scanner.go
StreamResult records whether Done() was called. The scanner avoids replacing the stop reason after cancellation and corrects ClientGone only for completed streams with no recorded errors.
Responses forwarding and lifecycle validation
relay/channel/openai/helper.go, relay/channel/openai/relay_responses.go, relay/channel/openai/relay_responses_billing_test.go
The Responses handler records events before forwarding, stops on forwarding errors, and marks qualifying completed responses as done. Tests cover completion, cancellation, I/O failures, billing data, and upstream closure.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Low

Possibly related issues

  • QuantumNous/new-api#6239: The PR applies normal-versus-error completion distinctions to Responses streams; the issue’s Chat timeout path remains outside this change.
  • QuantumNous/new-api#6230: The lifecycle tests check token usage and image-generation call counts for Responses outcomes.
  • QuantumNous/new-api#5222: The scanner changes preserve distinctions among client cancellation, incomplete streams, and recorded stream errors.

Suggested reviewers: seefs001, songquanpeng

Merge Risk: ⚪ Minimal · up to b7767

The reviewed change has no identified issue that needs resolution before merge; complete the normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 6 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly identifies the relay fix for correctly ending completed Responses streams, which matches the primary changes and objectives.
Linked Issues check Passed #6649 is closed and supplies historical context only. No active directly linked issue supplies coding requirements. The PR objectives state this condition and describe changes that address the histori…
Out of Scope Changes check Passed The changes stay within the stated stream-lifecycle scope. relay/helper/stream_scanner.go handles completion and cancellation reconciliation. relay/channel/openai/relay_responses.go and the Respon…
Full details: Linked Issues check

Explanation

#6649 is closed and supplies historical context only. No active directly linked issue supplies coding requirements. The PR objectives state this condition and describe changes that address the historical Responses SSE completion race.

Full details: Out of Scope Changes check

Explanation

The changes stay within the stated stream-lifecycle scope. relay/helper/stream_scanner.go handles completion and cancellation reconciliation. relay/channel/openai/relay_responses.go and the Responses tests cover completion outcomes. common/custom-event.go and relay/helper/common.go propagate SSE write and flush errors. No unrelated product, database, frontend, or protocol changes are identified.

✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit watches streams flow by,
And checks each write before the sky.
When DONE arrives, the end is clear,
No cleanup EOF should cause a fear.
Then hops away with carrots near.

Comment @coderabbitai help to get the list of available commands.

…ct errors

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from #6904 and #6951 and narrows the
cancellation reconciliation proposed in #6808. Refs #6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
@Calcium-Ion
Calcium-Ion force-pushed the codex/fix-responses-stream-completion branch from 9614b33 to b77674b Compare October 22, 2026 04:57
@Calcium-Ion
Calcium-Ion merged commit 0c2af97 into main Oct 22, 2026
3 checks passed
Yorick-Ryu added a commit to Yorick-Ryu/new-api that referenced this pull request Oct 25, 2026
Why: Completed Responses streams can be misclassified after cancellation,
and fixed per-write deadlines cut off slow clients receiving large events.

Change: Merge upstream QuantumNous#7956 and the following stream write budget fix.
Preserve BeiAPI terminal handling for failed, incomplete and cancelled
responses while surfacing downstream write and flush errors. Keep upstream
usage and tool counts, and reconcile cancellation only for delivered
completed responses without errors.

Validation: Relevant common, relay/common, relay/helper, relay/channel/openai,
relay, controller and service package tests passed. The initial lifecycle
fixture assertions raced EOF; hold those terminal fixtures open and rerun
the openai package successfully. Focused race tests, go build ./...,
scoped go vet and git diff --check passed.

Refs: QuantumNous#7956; upstream 7e65967
cicioln pushed a commit to cicioln/new-api that referenced this pull request Oct 25, 2026
…ct errors (QuantumNous#7956)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the
cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
(cherry picked from commit 0c2af97)
kyuhwi-choi pushed a commit to attodev/new-api that referenced this pull request Oct 30, 2026
…ct errors (QuantumNous#7956)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the
cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
(cherry picked from commit 0c2af97)
@Calcium-Ion
Calcium-Ion deleted the codex/fix-responses-stream-completion branch October 30, 2026 15:42
lizhongyi1209 pushed a commit to lizhongyi1209/new-api that referenced this pull request Nov 1, 2026
…ct errors (QuantumNous#7956)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the
cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
woncc added a commit to 86669666/new-api that referenced this pull request Nov 1, 2026
…ct errors (QuantumNous#7956) (#92)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the
cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649.

Co-authored-by: Calcium-Ion <i@caion.me>
Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
shirosoralumie648 added a commit to shirosoralumie648/newapi-rev that referenced this pull request Nov 5, 2026
Sync 75 upstream commit(s) via merge (fork history preserved).

Upstream commits:
- 7061447 fix(auth): 校验 Redis 鉴权缓存完整性 (QuantumNous#8200)
- ba5f736 fix(gemini): bill embeddings from upstream usageMetadata (QuantumNous#8182)
- 94b5216 feat(vertex): add per-channel v1beta1 endpoint switch (QuantumNous#8179)
- 83b18fe feat(channel): return created IDs with input mappings (QuantumNous#8164)
- 52c1730 feat(gemini): add Argon fields and reasoning mappings (QuantumNous#8177)
- 9b473a3 fix(twofa): restart the failure count after a lockout expires (QuantumNous#8172)
- 45746ee feat(relay): add opt-in HTTP/2 health checks (QuantumNous#8059)
- c046559 feat(plugins): bill gemini omni by the tokens vertex reports (QuantumNous#8153)
- 651d5a2 fix(xai): preserve token details in stream usage conversion (QuantumNous#6420)
- 70e6af6 perf: read and edit request JSON and base64 with moego (QuantumNous#8148)
- d0abbf6 feat(plugins): generate video with Gemini Omni on Vertex AI channels (QuantumNous#8142)
- ea343c4 fix(plugins): charge relayed streams even when the reserve or task insert fails (QuantumNous#8121)
- fe57cba fix(plugins): prefer Kling's returned price (list_price) for Kling 4.0 cash billing (QuantumNous#8125)
- 1424ffd fix(plugins): load plugins that declare their own /{key} address (QuantumNous#8119)
- 8b253de fix(plugins): keep Kling 4.0 durations in range and map every video size (QuantumNous#8118)
- c59f6fd feat(plugins): serve task plugins at the vendors' own paths and answer as the vendors do (QuantumNous#8117)
- 37d23dc fix(profile): honor global sidebar visibility (QuantumNous#7429)
- 7743627 fix(deps): upgrade DOMPurify to 3.4.17 (QuantumNous#8116)
- 883eeb8 feat(plugins): warn about task models a plugin cannot charge
- 1e8c707 chore(deps): bump @codemirror/language from 6.12.4 to 6.13.1 in /web (QuantumNous#8102)
- 6e78cb3 chore(deps): bump marked from 18.0.6 to 18.1.2 in /web (QuantumNous#8103)
- 9fc70bb chore(deps-dev): bump @tanstack/react-query-devtools in /web (QuantumNous#8104)
- cef7820 chore(deps-dev): bump knip from 6.27.0 to 6.43.0 in /web (QuantumNous#8105)
- 1695171 chore(deps): bump tailwind-merge from 3.6.0 to 3.7.0 in /web (QuantumNous#8106)
- 7fa815e chore(deps-dev): bump fast-uri from 3.1.7 to 3.1.9 in /electron (QuantumNous#8007)
- ee0f0ac chore(deps-dev): bump electron from 41.10.6 to 42.11.16 in /electron (QuantumNous#8008)
- 837b814 chore(deps): bump undici in /electron (QuantumNous#8006)
- 3150dfe chore(deps): bump github.com/go-jose/go-jose/v4 from 4.1.4 to 4.1.6 (QuantumNous#8112)
- a3851c2 fix: save Turnstile keys before enabling (QuantumNous#7960)
- 4193734 fix(logger): keep gin access and recovery logs on the current log file after rotation (QuantumNous#8098)
- f366017 Merge commit from fork
- 829f082 fix: unmount authenticated layout on sign-out to stop credential-less refetches (QuantumNous#8110)
- 9eda50a Merge commit from fork
- a2defd2 Merge commit from fork
- e449cc7 feat: sample immediate task successes in model performance metrics (QuantumNous#8067)
- fcc241c fix: start affinity fallback at the highest channel priority (QuantumNous#8078)
- b08ee86 fix(cloudflare): preserve embedding response data (QuantumNous#8041)
- 26c488c fix(relay): allow Azure channels to opt in to Responses WebSocket (QuantumNous#8047) (QuantumNous#8075)
- 55515db fix(relay): terminate streams on exact bare done markers (QuantumNous#8074)
- 49271d2 fix(aws): preserve safeguards in Bedrock Invoke requests (QuantumNous#8071)
- 5478811 fix: count completed non-stream relays after context cancellation (QuantumNous#7844)
- ad56112 fix(relay): apply GPT-6 chat parameter rules to gpt-6.1-sol and gpt-6.1-terra (QuantumNous#8064)
- 01da3a5 perf(relay): stop recompiling param override regexes on every request (QuantumNous#8061)
- 34003e9 test: stabilize channel affinity cache keys (QuantumNous#8009)
- 0165fef fix(relaykit): preserve Gemini text alongside tool calls (QuantumNous#7998)
- 4d962e4 fix(relaykit): preserve tool argument numbers in Claude responses (QuantumNous#7996)
- f1569d1 fix(pricing): classify dynamic per-request billing correctly (QuantumNous#8010)
- bc795cc fix:修复stop为非字符串和[]string的协议转换问题 (QuantumNous#7935)
- 9823ebc fix(billing): bill cache_creation_input_tokens reported in prompt_tokens_details (QuantumNous#7909)
- e835b1b chore(deps): bump axios from 1.18.1 to 1.20.0 in /web (QuantumNous#7835)
- 5c8032d chore(deps): bump react-i18next from 17.0.10 to 17.1.0 in /web (QuantumNous#7981)
- 0b667ca chore(deps): bump i18next from 26.3.6 to 26.5.3 in /web (QuantumNous#7979)
- 4105bd0 chore(deps): bump input-otp from 1.4.2 to 1.5.0 in /web (QuantumNous#7980)
- b6b85ca chore(deps): bump react-resizable-panels from 4.12.2 to 4.14.6 in /web (QuantumNous#7982)
- 1dafb19 fix(gemini): preserve top-level imageConfig in chat conversion
- 0034b24 fix(task): wait up to 72 hours before timing out tasks
- a404460 chore(web): upgrade Bun to 1.4.4 and use bun check
- 33c61a2 fix: prevent stale values when switching edit drawer records (QuantumNous#7989)
- 30748c6 fix(logging): 隐藏访问日志中的查询凭据 (QuantumNous#7972)
- 7fb6e57 fix(channels): preserve advanced custom body passthrough settings (QuantumNous#7971)
- e9d4d63 feat(billing): bill OpenAI 12-hour prompt cache writes at their own price (QuantumNous#7965)
- d0b840e fix(relay): keep the client's JSON Schema property order
- d8e6561 feat(web): copy model names from usage log details
- 8802113 feat(task): let ordinary channels compete on shared plugin endpoints
- eb169e7 feat(pricing): adjust synced upstream prices by the site exchange rate
- e2321a5 chore(web): sort locale keys
- db80ac8 fix(task): let task plugins validate their own requests
- dbe97be feat(task): pass client request headers to task plugin drivers
- 556517f feat(task): accept multipart bodies on native plugin routes
- 94cc3ff fix(billing): bill Claude cache writes that arrive without a 5m/1h split
- 7e65967 fix(relay): stop cutting off slow clients during large stream events
- 0c2af97 fix(relay): finish delivered Responses streams without false disconnect errors (QuantumNous#7956)
- 7d8b912 fix: upgrade Go to 1.26.10 for CVE-2026-111943 (QuantumNous#7942)
- 4d36f2c chore(deps-dev): bump vitest from 4.1.10 to 5.0.4 in /web (QuantumNous#7838)
- eea05e0 chore(deps): bump github.com/klauspost/compress from 1.18.3 to 1.18.7 (QuantumNous#7931)

Local tests not run (CI-only).
timwenx pushed a commit to timwenx/new-api that referenced this pull request Nov 18, 2026
…ct errors (QuantumNous#7956)

Stop Responses SSE after its completed event has been written, preserving
upstream token and tool usage. Reconcile a concurrent client cancellation
only after all scanner workers have joined and a completed response was
successfully delivered. Bare EOF, failed writes and protocol errors do not
qualify. Propagate SSE write/flush errors and suppress cleanup read errors.

Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the
cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649.

Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com>
Co-authored-by: zkasuran <zkasuran@gmail.com>
Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

stream_status.end_reason shows "client_gone" for streams that completed normally (SSE scanner race)

1 participant