Skip to content

fix(metrics): count completed streams after context cancellation - #7638

Merged
seefs001 merged 1 commit into
QuantumNous:mainfrom
jimmyleocn:fix/metrics-completed-stream-cancellation
Oct 12, 2026
Merged

seefs001 merged 1 commit into
QuantumNous:mainfrom
jimmyleocn:fix/metrics-completed-stream-cancellation

Conversation

@jimmyleocn

@jimmyleocn jimmyleocn commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Agent

  • Tool: OpenAI Codex
  • Tool version: codex-cli 0.159.0
  • Model (full id): gpt-6.1-sol
  • Host (CLI / IDE / GitHub coding agent / other): IDE coding agent
  • Date (UTC): 2026-10-01

Links

User request

  • Verbatim: 深入研究这个问题的本质,和调查是上游哪次提交引入的。
  • Later constraints or corrections from the user (quote, or none): “其实如果bug是9.18日提交后引入的,之前时间的issue和pr其实不能算。”;“正式提交issue的话就必须以正式版为视角来描述”

Out of scope — refuse

  • Matched: no;仅修正 new-api 自身的模型健康分类,不是转发兼容、透传或第三方服务问题。
  • If yes, what was told to the user (stop here; do not open a PR): not applicable

Open gate — do not open unless all are satisfied

  • Out of scope (including pass-through-only forwarding): no
  • Usage / configuration / integration (answered instead of opening): no
  • Required issue facts present without invention: yes;见 [Bug] 已正常完成的流式请求因 context 取消被模型健康统计忽略 #7637 的原版上游源码最小复现。
  • Verification is actual commands or steps and observed results, not only go build or tests passed: yes;EOF/DONE 取消后的分类由 ignored 变为 success,其他边界结果保持不变。
  • Body is short and factual; no unfiltered AI-generated text: yes;已提炼证据并经请求者审阅调整。
  • Open: yes
  • If no, what was told to the user (stop here): not applicable

Kind

  • Bug fix
  • New feature
  • Performance / refactor
  • Docs
  • Other:

Issue facts

  • Actual behavior: 已 completed、正常 EOF/DONE 且无错误的流,在统计前 context 被取消时返回 ignored。
  • Impact: RecordRelayResult 跳过 ignored,正常成功请求及其性能样本未进入模型健康统计桶。
  • Frequency: 指定时序下,EOF/DONE 两种最小复现均必现;不声称真实流量中的总体比例。
  • Evidence that the problem is in new-api rather than the client or upstream: [Bug] 已正常完成的流式请求因 context 取消被模型健康统计忽略 #7637 在未修改的上游 commit 1a4166d8e 上,仅改变 context 状态即可得到 success → ignored。
  • Applicable types and their fields (relay / billing / frontend / deployment; write "not applicable" otherwise): 内部健康分类;最小复现不发送 HTTP 请求、不依赖渠道或模型。计费、前端、部署升级为 not applicable;端到端限制见 Verification。

Change

仅修改 pkg/perf_metrics/outcome.go:先读取已有流终态,再判断 context 取消。

只有同时满足下列条件,才允许已取消 context 的请求继续参与原有分类:

  • IsStream 为 true,StreamStatus 存在;
  • 协议终态为 completed,结束原因为 eof 或 done;
  • API 错误、流错误和结束错误均不存在。

没有提前返回 success;其他分类分支保持原样。生成过程中取消、未完成或存在错误的请求不因该例外而误计成功。

本代码由 AI 辅助完成,已对照实际 diff 和可执行复现审阅验证;提交者不是仓库历史核心开发者。

Research

Duplicate / prior art

Docs and code

  • https://docs.newapi.ai/ : 查看管理指南入口和功能介绍,未找到可纠正该取消时序的配置。
  • https://deepwiki.com/QuantumNous/new-api : 经官方 MCP 阅读 Performance Metrics & Rankings 的采样、聚合和展示说明,未找到必须排除正常完成后取消样本的明确要求。
  • README / repo docs: README 的网关和用量管理说明未解释该分类行为。
  • Code paths and what they imply for this change: ClassifyRelayOutcome 是共享资格判断;RecordRelayResult 在 HTTP Relay 和 Responses WS 收尾时调用。GitNexus 影响分析:1 个直接调用者、7 个上游符号,风险 LOW;分支变更分析定位到 1 个文件、1 个函数。

Alternatives considered

  • Option A: 在共享分类函数中依据已完成且无错误的终态,为 context 取消增加窄例外。
  • Option B: 移除 context 取消过滤,或分别修改 HTTP/WS 调用方。
  • Why this approach: 保留生成中取消的忽略语义,且一次覆盖共享入口,不扩大修复范围。

Files

Path Why
pkg/perf_metrics/outcome.go 区分正常完成后的取消与生成中的取消;仅 +10/-4 行

Behavior

  • Before: completed + EOF/DONE + 无错误的流,在 context 取消后从 success 变为 ignored,随后被跳过采样。
  • After: 同样输入在取消后仍为 success;中途取消、缺终态、业务拒绝和有错误的请求继续遵循原有分类。
  • Explicit non-goals / leftover work: 不修改转发协议、计费、重试、渠道禁用、scanner 结束原因、数据库结构或前端;不重算历史样本。

Verification

  • Commands and results:
    • 从项目根目录运行 [Bug] 已正常完成的流式请求因 context 取消被模型健康统计忽略 #7637 的仓库外独立复现程序;原版 outcome.go 使用 Go overlay 加载。实际对照:

      原版上游:
      end=eof before=success after=ignored
      end=done before=success after=ignored
      
      修复后:
      end=eof before=success after=success
      end=done before=success after=success
      
    • go run -p 1 <仓库外的 verify.go>:16 项确定性边界校验通过;正常 EOF/DONE 在取消后计成功,中途取消、缺终态、异常结束、流错误、API 错误、业务拒绝、非流式取消及 nil 状态不错误地计成功。

    • go test -p 1 -parallel 1 ./pkg/perf_metrics ./relay/common ./relay -count=1 -timeout=2m:三个包通过。

    • go test -p 1 -parallel 1 ./controller -run '^TestResponses(InterruptedStreamHealth|HTTPHealthCountsFinalResult|WebSocketPreRoutingRejectionsFollowHealthClassification)$' -count=1 -timeout=2m:通过;覆盖 HTTP/WS 超时、中途取消、业务拒绝、重试最终结果及路由前拒绝的健康分类。

    • go vet -p 1 ./pkg/perf_metrics、go build -p 1 ./...:通过。

    • git diff --check upstream/main...HEAD:通过;GitNexus detect-changes --scope compare --base-ref upstream/main 仅定位到 ClassifyRelayOutcome,风险 low。

  • Manual steps and observed result: 无额外手工界面操作;确定性复现和实际输出如上。
  • UI: screenshot or recording (or why none): not applicable;无 UI 改动。
  • Tests added or updated, or why none: 本 PR 仅包含生产代码修复;未新增或修改仓库测试,已运行既有回归及仓库外的复现、边界校验。
  • Databases / providers / platforms exercised: Windows amd64 / Go 1.26.6;既有相关测试使用 SQLite;分类最小复现不访问数据库或真实模型服务。
  • Not verified: 未执行未修改官方实例与真实上游的端到端对照;未测量总体频率,未覆盖其他平台及 MySQL/PostgreSQL。controller 全量测试尝试未通过,出现 Windows 下 SQLite 临时文件占用的清理错误;上述健康分类专项测试通过,不宣称全仓测试通过。

Risks

  • Failure modes: 若放宽终态或错误条件,可能将真正中断的请求计成功;当前要求 completed、EOF/DONE 和无错误全部满足,未改变其他分支。
  • Billing / quota / auth impact: 无相关代码改动;只改变健康采样资格,不改变转发、结算、退款或鉴权。
  • Follow-ups: 合并后仅新请求按修正规则采样,历史遗漏样本不会自动补回。

Scope check

  • Single focused change: yes;一个文件、一个分类函数。
  • Secrets included: no
  • Out of scope (Coding Plan / reverse-engineered channel / third-party wrapper / Codex / pass-through-only forwarding): no

Summary by CodeRabbit

  • Bug Fixes
    • Completed streaming responses are now counted consistently when a request context is canceled after the response finishes. Streams with errors or incomplete responses, as well as rejected requests, continue to be excluded from outcome metrics. This improves the accuracy of reported relay outcomes without changing what responses are delivered.

@coderabbitai

coderabbitai Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

🧰 Additional context used
📚 Code guidelines (1)
AGENTS.md — auto-discovered

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: db0b47f1-f06f-43a7-9919-0f96b9ff62bf

📥 Commits

Reviewing files that changed from the base of the PR and between 1a4166d and dbf3045.

📒 Files selected for processing (1)
  • pkg/perf_metrics/outcome.go

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.


Walkthrough

ClassifyRelayOutcome now checks the stream outcome before classifying a canceled context. Normally completed streams without errors remain eligible for sampling after cancellation.

Changes

Stream outcome classification

Layer / File(s) Summary
Cancellation check
pkg/perf_metrics/outcome.go
The function snapshots stream status before checking context cancellation. It does not ignore a canceled context for a completed stream with no API error, snapshot errors, or EndError that ends in EOF or DONE.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to dbf30

Only normally completed, error-free streams remain eligible for health sampling after cancellation; incomplete and errored canceled requests remain excluded. No actionable merge risk remains in the reviewed scope.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: counting completed streams after context cancellation. It matches the reported behavior and scope.
Linked Issues check ✅ Passed For #7637, ClassifyRelayOutcome snapshots the stream outcome and preserves eligibility when info.IsStream is true, apiErr is nil, the response is completed, the end reason is EOF or DONE, `Has…
Out of Scope Changes check ✅ Passed The pull request changes only pkg/perf_metrics/outcome.go. The code and comments modify health-sample classification for normally completed streams after context cancellation. No unrelated billing, …
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the stream at dawn
EOF arrives; the errors are gone
DONE leaves a sample in the queue
A canceled context cannot hide it from view
The rabbit hops, its metrics true

Comment @coderabbitai help to get the list of available commands.

@seefs001
seefs001 merged commit 3f926c3 into QuantumNous:main Oct 12, 2026
3 checks passed
shirosoralumie648 added a commit to shirosoralumie648/newapi-rev that referenced this pull request Oct 17, 2026
Sync 54 upstream commit(s) via merge (fork history preserved).
Behavior changes from upstream:
- 4013729 fix: disable sensitive word filtering by default and clear the default word list (QuantumNous#7814)

Upstream commits:
- 5e95951 feat(channels): add drag mode for channel priority ordering
- 5a59d69 fix(pricing): hide task prices left at zero when another price is set
- 3e2ce2c fix: align model card summary with status bar hours (QuantumNous#7847)
- b0a98ac fix(web): apply theme class before first paint so Dark Reader detects dark mode (QuantumNous#7571)
- f92c576 fix(web): cover Cloudflare clef and NVIDIA model families in provider rules (QuantumNous#7840)
- ff36846 fix(relayconvert): 保留 Chat/Claude → Responses 转换中 input_image 的 detail 字段 (QuantumNous#7855)
- 09109ec fix(middleware): 过载哨兵改用容器口径并加入连续性与新鲜度判定 (QuantumNous#7317)
- d032204 feat(aws): add Mantle Messages API support
- ca2cae6 fix(relaykit): emit summary on reasoning items announced without one (QuantumNous#7833)
- 45fbb14 fix(rate-limit): count each group with its own limit separately
- 6549a17 fix(dashboard): draw flow chart labels in the theme text color (QuantumNous#7821)
- c9179d3 fix(playground): support the reasoning field in chat responses (QuantumNous#7831)
- 908f94d fix(web): hide pages the user cannot open from console search
- 19d435e fix(relay): mask only the custom upstream address in error messages (QuantumNous#7822)
- 718a4e7 fix(vertex): use the multi-region hostnames for us and eu
- f196934 feat(plugins): bind Hailuo, Vidu and Veo plugins to New API channels
- bfdad29 fix(vidu): bill the model, duration and resolution Vidu generates
- 150ab3c chore(licenses): refresh the third-party dependency inventory
- 76cd511 build: upgrade moejs to v0.1.0-alpha.7
- 1b63c28 fix(claude): preserve explicit zero usage and cache-only billing (QuantumNous#7816)
- c9f7b62 fix(web): sort pricing models by input, output and per-request price (QuantumNous#7813)
- a0bf6a8 fix(vertex): pass project_id in API Key mode to avoid 404 on official endpoints (QuantumNous#6330)
- 4013729 fix: disable sensitive word filtering by default and clear the default word list (QuantumNous#7814)
- 7de10f2 feat(token): let an API key read its owner's quota
- ef60ef2 fix(relay): keep the cause of a failed body read for every later reader
- d10eafb fix(relay): report the real cause when a request body cannot be read
- 63cecab feat(channel): support Sub2API balance queries (QuantumNous#7749)
- b4be2a7 feat(ui): show newly added pricing groups at the top (QuantumNous#7782)
- 633b4c5 feat(logs): export usage logs from the common log page (QuantumNous#7774)
- dd59854 chore(deps-dev): bump undici from 6.28.0 to 6.29.0 in /electron (QuantumNous#7624)
- 93cf4ba chore(deps-dev): bump js-yaml from 4.3.1 to 4.3.2 in /electron (QuantumNous#7327)
- 3c9fb2a chore(deps-dev): bump brace-expansion from 1.1.16 to 1.1.21 in /electron (QuantumNous#7623)
- 9f04810 chore(deps): bump ai from 7.0.31 to 7.0.136 in /web (QuantumNous#7767)
- 3fe394c chore(deps): bump github.com/gorilla/websocket from 1.5.0 to 1.5.3 (QuantumNous#7001)
- 1c58f3d chore(deps-dev): bump fast-uri from 3.1.5 to 3.1.7 in /electron (QuantumNous#7169)
- 7f51a97 chore(deps-dev): bump js-yaml from 4.3.1 to 4.3.2 in /electron (QuantumNous#7327)
- 7e25797 chore(deps-dev): bump electron from 39.8.10 to 41.10.6 in /electron (QuantumNous#7625)
- bf4e1eb build(deps-dev): bump oxfmt from 0.57.0 to 0.72.0 in /web (QuantumNous#7766)
- 82306b3 build(deps-dev): bump @tanstack/router-plugin in /web (QuantumNous#7768)
- c730add build(deps): bump stream-markdown-parser from 1.1.3 to 1.2.18 in /web (QuantumNous#7769)
- b8c2dc1 build(deps): bump yace from 1.1.0 to 1.1.1 in /web (QuantumNous#7770)
- ba29f06 chore(deps-dev): bump @xmldom/xmldom from 0.8.13 to 0.8.15 in /electron (QuantumNous#7187)
- 767a82b fix(claude): forward thinking.block_binding and every anthropic-beta value (QuantumNous#7732)
- 2b85a8f fix(auth): show legal consent warning on password login (QuantumNous#7771)
- 2ed0587 fix(ci): stabilize checks and configure Bun dependency updates (QuantumNous#7765)
- 9948b1e fix(pricing): accept declared endpoint arrays in pricing conversion (QuantumNous#7724)
- 4bfcf28 chore(web): bump @lobehub/icons to 5.22.0 (QuantumNous#7764)
- fc08b76 fix(web): preserve recorded billing tiers in usage log previews (QuantumNous#7763)
- ba0e478 fix(relaykit): preserve assistant text with tool calls (QuantumNous#6871)
- 34b1942 fix(pricing): allow root access when the model square is disabled (QuantumNous#7761)
- b22a43b fix: mark task settlement logs with is_task for tiered billing display (QuantumNous#7446)
- bbf60b4 fix(logs): vertically align billing source icons on mobile (QuantumNous#7760)
- 3f926c3 fix(metrics): count completed streams after context cancellation (QuantumNous#7638)
- 3827fd8 fix(web): show cache read price on expression model cards (QuantumNous#7757)

Local tests not run (CI-only).
hydraxman pushed a commit to MindxMotion/MEM-AI that referenced this pull request Oct 28, 2026
Select nine upstream fixes from QuantumNous/new-api main at 49271d2. Preserve OIDC, root-managed integration clients, bounded diagnostics, canonical mem-ai routes, and existing dependencies. Do not include pricing, schema, frontend, or deployment changes.

Keep the AWS safeguards regression without importing unrelated thinking-binding tests from three-way context. Add canceled-context stream coverage alongside the upstream non-stream regression.

Upstream sources:
bc795cc fix:修复stop为非字符串和[]string的协议转换问题 (QuantumNous#7935)
4d962e4 fix(relaykit): preserve tool argument numbers in Claude responses (QuantumNous#7996)
0165fef fix(relaykit): preserve Gemini text alongside tool calls (QuantumNous#7998)
34003e9 test: stabilize channel affinity cache keys (QuantumNous#8009)
01da3a5 perf(relay): stop recompiling param override regexes on every request (QuantumNous#8061)
ad56112 fix(relay): apply GPT-6 chat parameter rules to gpt-6.1-sol and gpt-6.1-terra (QuantumNous#8064)
3f926c3 fix(metrics): count completed streams after context cancellation (QuantumNous#7638)
5478811 fix: count completed non-stream relays after context cancellation (QuantumNous#7844)
49271d2 fix(aws): preserve safeguards in Bedrock Invoke requests (QuantumNous#8071)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] 已正常完成的流式请求因 context 取消被模型健康统计忽略

2 participants