Repository navigation
fix(relay): 正确结束已完成的 Responses 流 - #7956
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
WalkthroughSSE writes, event rendering, and flushes now return errors. Stream handling records explicit completion and preserves the correct end reason when cancellation or read errors occur during cleanup. The Responses stream handler records events before forwarding them and marks qualifying completed responses as done. ChangesSSE stream lifecycle
Priority: ➖ Normal Estimated code review effort: 3 (Moderate) | ~20 minutes Change: Bug fix · Severity of issue fixed: Low Possibly related issues
Suggested reviewers: Merge Risk: ⚪ Minimal · up to
|
…ct errors Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from #6904 and #6951 and narrows the cancellation reconciliation proposed in #6808. Refs #6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
9614b33 to
b77674b
Compare
Why: Completed Responses streams can be misclassified after cancellation, and fixed per-write deadlines cut off slow clients receiving large events. Change: Merge upstream QuantumNous#7956 and the following stream write budget fix. Preserve BeiAPI terminal handling for failed, incomplete and cancelled responses while surfacing downstream write and flush errors. Keep upstream usage and tool counts, and reconcile cancellation only for delivered completed responses without errors. Validation: Relevant common, relay/common, relay/helper, relay/channel/openai, relay, controller and service package tests passed. The initial lifecycle fixture assertions raced EOF; hold those terminal fixtures open and rerun the openai package successfully. Focused race tests, go build ./..., scoped go vet and git diff --check passed. Refs: QuantumNous#7956; upstream 7e65967
…ct errors (QuantumNous#7956) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com> (cherry picked from commit 0c2af97)
…ct errors (QuantumNous#7956) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com> (cherry picked from commit 0c2af97)
…ct errors (QuantumNous#7956) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
…ct errors (QuantumNous#7956) (#92) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649. Co-authored-by: Calcium-Ion <i@caion.me> Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
Sync 75 upstream commit(s) via merge (fork history preserved). Upstream commits: - 7061447 fix(auth): 校验 Redis 鉴权缓存完整性 (QuantumNous#8200) - ba5f736 fix(gemini): bill embeddings from upstream usageMetadata (QuantumNous#8182) - 94b5216 feat(vertex): add per-channel v1beta1 endpoint switch (QuantumNous#8179) - 83b18fe feat(channel): return created IDs with input mappings (QuantumNous#8164) - 52c1730 feat(gemini): add Argon fields and reasoning mappings (QuantumNous#8177) - 9b473a3 fix(twofa): restart the failure count after a lockout expires (QuantumNous#8172) - 45746ee feat(relay): add opt-in HTTP/2 health checks (QuantumNous#8059) - c046559 feat(plugins): bill gemini omni by the tokens vertex reports (QuantumNous#8153) - 651d5a2 fix(xai): preserve token details in stream usage conversion (QuantumNous#6420) - 70e6af6 perf: read and edit request JSON and base64 with moego (QuantumNous#8148) - d0abbf6 feat(plugins): generate video with Gemini Omni on Vertex AI channels (QuantumNous#8142) - ea343c4 fix(plugins): charge relayed streams even when the reserve or task insert fails (QuantumNous#8121) - fe57cba fix(plugins): prefer Kling's returned price (list_price) for Kling 4.0 cash billing (QuantumNous#8125) - 1424ffd fix(plugins): load plugins that declare their own /{key} address (QuantumNous#8119) - 8b253de fix(plugins): keep Kling 4.0 durations in range and map every video size (QuantumNous#8118) - c59f6fd feat(plugins): serve task plugins at the vendors' own paths and answer as the vendors do (QuantumNous#8117) - 37d23dc fix(profile): honor global sidebar visibility (QuantumNous#7429) - 7743627 fix(deps): upgrade DOMPurify to 3.4.17 (QuantumNous#8116) - 883eeb8 feat(plugins): warn about task models a plugin cannot charge - 1e8c707 chore(deps): bump @codemirror/language from 6.12.4 to 6.13.1 in /web (QuantumNous#8102) - 6e78cb3 chore(deps): bump marked from 18.0.6 to 18.1.2 in /web (QuantumNous#8103) - 9fc70bb chore(deps-dev): bump @tanstack/react-query-devtools in /web (QuantumNous#8104) - cef7820 chore(deps-dev): bump knip from 6.27.0 to 6.43.0 in /web (QuantumNous#8105) - 1695171 chore(deps): bump tailwind-merge from 3.6.0 to 3.7.0 in /web (QuantumNous#8106) - 7fa815e chore(deps-dev): bump fast-uri from 3.1.7 to 3.1.9 in /electron (QuantumNous#8007) - ee0f0ac chore(deps-dev): bump electron from 41.10.6 to 42.11.16 in /electron (QuantumNous#8008) - 837b814 chore(deps): bump undici in /electron (QuantumNous#8006) - 3150dfe chore(deps): bump github.com/go-jose/go-jose/v4 from 4.1.4 to 4.1.6 (QuantumNous#8112) - a3851c2 fix: save Turnstile keys before enabling (QuantumNous#7960) - 4193734 fix(logger): keep gin access and recovery logs on the current log file after rotation (QuantumNous#8098) - f366017 Merge commit from fork - 829f082 fix: unmount authenticated layout on sign-out to stop credential-less refetches (QuantumNous#8110) - 9eda50a Merge commit from fork - a2defd2 Merge commit from fork - e449cc7 feat: sample immediate task successes in model performance metrics (QuantumNous#8067) - fcc241c fix: start affinity fallback at the highest channel priority (QuantumNous#8078) - b08ee86 fix(cloudflare): preserve embedding response data (QuantumNous#8041) - 26c488c fix(relay): allow Azure channels to opt in to Responses WebSocket (QuantumNous#8047) (QuantumNous#8075) - 55515db fix(relay): terminate streams on exact bare done markers (QuantumNous#8074) - 49271d2 fix(aws): preserve safeguards in Bedrock Invoke requests (QuantumNous#8071) - 5478811 fix: count completed non-stream relays after context cancellation (QuantumNous#7844) - ad56112 fix(relay): apply GPT-6 chat parameter rules to gpt-6.1-sol and gpt-6.1-terra (QuantumNous#8064) - 01da3a5 perf(relay): stop recompiling param override regexes on every request (QuantumNous#8061) - 34003e9 test: stabilize channel affinity cache keys (QuantumNous#8009) - 0165fef fix(relaykit): preserve Gemini text alongside tool calls (QuantumNous#7998) - 4d962e4 fix(relaykit): preserve tool argument numbers in Claude responses (QuantumNous#7996) - f1569d1 fix(pricing): classify dynamic per-request billing correctly (QuantumNous#8010) - bc795cc fix:修复stop为非字符串和[]string的协议转换问题 (QuantumNous#7935) - 9823ebc fix(billing): bill cache_creation_input_tokens reported in prompt_tokens_details (QuantumNous#7909) - e835b1b chore(deps): bump axios from 1.18.1 to 1.20.0 in /web (QuantumNous#7835) - 5c8032d chore(deps): bump react-i18next from 17.0.10 to 17.1.0 in /web (QuantumNous#7981) - 0b667ca chore(deps): bump i18next from 26.3.6 to 26.5.3 in /web (QuantumNous#7979) - 4105bd0 chore(deps): bump input-otp from 1.4.2 to 1.5.0 in /web (QuantumNous#7980) - b6b85ca chore(deps): bump react-resizable-panels from 4.12.2 to 4.14.6 in /web (QuantumNous#7982) - 1dafb19 fix(gemini): preserve top-level imageConfig in chat conversion - 0034b24 fix(task): wait up to 72 hours before timing out tasks - a404460 chore(web): upgrade Bun to 1.4.4 and use bun check - 33c61a2 fix: prevent stale values when switching edit drawer records (QuantumNous#7989) - 30748c6 fix(logging): 隐藏访问日志中的查询凭据 (QuantumNous#7972) - 7fb6e57 fix(channels): preserve advanced custom body passthrough settings (QuantumNous#7971) - e9d4d63 feat(billing): bill OpenAI 12-hour prompt cache writes at their own price (QuantumNous#7965) - d0b840e fix(relay): keep the client's JSON Schema property order - d8e6561 feat(web): copy model names from usage log details - 8802113 feat(task): let ordinary channels compete on shared plugin endpoints - eb169e7 feat(pricing): adjust synced upstream prices by the site exchange rate - e2321a5 chore(web): sort locale keys - db80ac8 fix(task): let task plugins validate their own requests - dbe97be feat(task): pass client request headers to task plugin drivers - 556517f feat(task): accept multipart bodies on native plugin routes - 94cc3ff fix(billing): bill Claude cache writes that arrive without a 5m/1h split - 7e65967 fix(relay): stop cutting off slow clients during large stream events - 0c2af97 fix(relay): finish delivered Responses streams without false disconnect errors (QuantumNous#7956) - 7d8b912 fix: upgrade Go to 1.26.10 for CVE-2026-111943 (QuantumNous#7942) - 4d36f2c chore(deps-dev): bump vitest from 4.1.10 to 5.0.4 in /web (QuantumNous#7838) - eea05e0 chore(deps): bump github.com/klauspost/compress from 1.18.3 to 1.18.7 (QuantumNous#7931) Local tests not run (CI-only).
…ct errors (QuantumNous#7956) Stop Responses SSE after its completed event has been written, preserving upstream token and tool usage. Reconcile a concurrent client cancellation only after all scanner workers have joined and a completed response was successfully delivered. Bare EOF, failed writes and protocol errors do not qualify. Propagate SSE write/flush errors and suppress cleanup read errors. Integrates the completion handling from QuantumNous#6904 and QuantumNous#6951 and narrows the cancellation reconciliation proposed in QuantumNous#6808. Refs QuantumNous#6649. Co-authored-by: leecz <11499281+lichengzhe@users.noreply.github.com> Co-authored-by: zkasuran <zkasuran@gmail.com> Co-authored-by: yinshaohua <3753251+yinshaohua@users.noreply.github.com>
🔗 关联任务 / Related Issue
🚀 变更类型 / Type of change
📝 变更描述 / Description
Responses 已发送完整完成事件、客户端在上游 EOF 前断开时,原代码仍记录
client_gone / error,并将健康样本判为ignored。现在成功发送完成事件后主动结束扫描;所有协程退出后,只纠正明确完成且无错误的并发取消。裸 EOF、真正中途断连、协议错误以及发送失败不被改成成功。保留当前 usage 累积逻辑,补齐 SSE 写入和 flush 错误传递,避免正常收尾关闭上游 Body 时重复记录 scanner 错误。回归集中在现有 Responses 测试文件,覆盖 15 个完成、断连、写入失败和协议终态场景,并核对 token 和图片调用计数。
由 Codex 按维护者授权整合并复核。参考 #6904 / #6951 的完成事件处理及 #6808 的竞态分析;提交包含 leecz、zkasuran、yinshaohua 的
Co-authored-by。📸 运行证明 / Proof of Work
环境:macOS arm64,Go 1.26.10,基于
7d8b9126的独立 worktree。go test ./common ./relay/common ./relay/helper ./relay/channel/openai ./relay/channel/claude ./relay/channel/gemini ./relay/channel/dify ./service ./pkg/perf_metrics -count=1 -timeout=3m:8 个有测试的包通过;dify 无测试文件。go test -race -parallel=1 ./relay/channel/openai ./relay/helper ./relay/common -run 'Test(OaiResponsesStreamHandlerCompletionLifecycle|StreamScannerHandler_|StreamStatus_)' -count=1 -timeout=2m:3 个包通过;仍保留每个请求内部协程的 race 检测。go test -race -v -run '^TestIssue6649HTTPClientDisconnect$' -count=1 -timeout=30s /private/tmp/new-api-6649-hgud6g0c/fixed-repro_test.go通过。客户端完整收到完成事件后关闭连接,由client_gone / error / ignored变为done / ok / success,usage 保持 7+5=12;中途断开仍为client_gone / error / ignored。go build ./...、go vet ./common ./relay/helper ./relay/channel/openai、git diff --check:通过。构建所需web/dist复用现有构建产物;无前端源码改动。限制:默认并行的 helper
-race检查触发已有logger/logger.go:114的logCount竞态,使用未修改主干的 Go overlay 已复现相同告警;本 PR 不改日志计数器。未运行官方镜像或真实供应商请求,无数据库结构或查询变更。✅ 提交前检查项 / Checklist
Summary by CodeRabbit