Skip to content

fix(claude): preserve explicit zero usage and cache-only billing - #7816

Merged
Calcium-Ion merged 3 commits into
QuantumNous:mainfrom
paopaoandlingyia:codex/claude-zero-usage
Oct 14, 2026
Merged

Calcium-Ion merged 3 commits into
QuantumNous:mainfrom
paopaoandlingyia:codex/claude-zero-usage

Conversation

@paopaoandlingyia

@paopaoandlingyia paopaoandlingyia commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #7815

Claude 流式响应可能在正常结束时返回零输入、零输出及非零缓存用量。原逻辑把零当作缺失,额外补算普通输入,导致多扣费用。

本修改记录输入、输出计数是否实际返回,保留明确的零值,仅对缺失字段或中断后的未完成输出进行估算。同时保留纯缓存用量的正常计费,不改变拒绝免单、模型定价或 fallback 策略。

验证:

  • 回归覆盖完整拒绝、全零响应、终止零值修正、缺失字段和流中断;原代码错误补算,修复后保留零值。
  • 纯缓存计费测试覆盖 1h 写入、5m 写入、缓存读取和全零用量。
  • go test ./relay/channel/claude ./relay/channel/aws ./relay/channel/vertex ./relay -count=1 通过(Vertex 无测试)。
  • 相关 service 计费测试、go build ./relay/... ./service 和受影响包的 go vet 通过。
  • relaykit 独立构建、DTO/转换测试和静态检查通过。
  • service 全量测试有亲和缓存统计失败;未修改的上游版本也会出现,相关计费回归通过。

代码由 AI 辅助完成。

Summary by CodeRabbit

  • Bug Fixes
    • Improved Claude usage estimates for streamed and interrupted responses, including cached input tokens.
    • Preserved billing for requests that use only cached input tokens, even when ordinary token totals are zero.

@coderabbitai

coderabbitai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Walkthrough

Claude stream finalization now uses converted prompt usage and streamed response text to determine when to estimate usage. Quota checks treat positive Claude cache token counts as billable, including when ordinary token totals are zero.

Changes

Claude Usage Handling

Layer / File(s) Summary
Finalize streamed usage
relay/channel/claude/relay-claude.go, relay/channel/claude/relay_claude_test.go
Finalization checks converted prompt usage and response text before applying local estimates. Tests cover cache-only usage, interrupted streams, streamed text with zero reported usage, and a zero input delta after nonzero start usage.
Recognize cache-only billable usage
service/text_quota.go, service/text_quota_test.go
Billability checks now include positive Claude cache token counts. Tests cover cache writes, cache reads, and empty usage.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: calcium-ion

Merge Risk: ⚪ Minimal · up to 6817d

No confirmed issue blocks merging after normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 6 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main changes: preserving explicit zero usage and retaining billing for cache-only Claude usage.
Linked Issues check Passed Issue #7815 requires explicit zero input and output counts to remain zero. It also requires cache-only usage to remain billable. HandleStreamFinalResponse checks converted input usage, which include…
Out of Scope Changes check Passed The reported changes stay within issue #7815. They update Claude stream usage finalization, Anthropic cache-only billability, and regression tests for those behaviors. No unrelated product behavior is…
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit checks the token stream,
Cache counts join the billing team.
Zero stays zero when it is known,
Estimates fill gaps alone.
The tests hop through each usage case,
Then nibble carrots in their place.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔇 Additional comments: 6
relaykit/dto/claude.go: 1

653-688: LGTM!

relaykit/relayconvert/internal/claude_messages/to_oai_chat_resp.go: 1

26-28: LGTM!

Also applies to: 515-515, 539-542, 557-557

relay/channel/claude/relay-claude.go: 1

260-279: LGTM!

service/text_quota.go: 1

70-78: LGTM!

service/text_quota_test.go: 1

1333-1365: LGTM!

relay/channel/claude/relay_claude_test.go-48-48: 1

48-48: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

⚠️ Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Use httptest.NewRequestWithContext to satisfy the noctx linter.

golangci-lint reports httptest.NewRequest as forbidden on this line. Pass a context to clear the diagnostic.

Proposed fix
-			c.Request = httptest.NewRequest(http.MethodPost, "/v1/messages", nil)
+			c.Request = httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/v1/messages", nil)

Source: Linters/SAST tools


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b300c10d-2d34-4ea9-b0e8-6e40d6d258ed
📥 Commits

Reviewing files that changed from the base of the PR and between 4013729 and 1d47561.

📒 Files selected for processing: 6
  • relay/channel/claude/relay-claude.go
  • relay/channel/claude/relay_claude_test.go
  • relaykit/dto/claude.go
  • relaykit/relayconvert/internal/claude_messages/to_oai_chat_resp.go
  • service/text_quota.go
  • service/text_quota_test.go

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔇 Additional comments: 2
service/text_quota_test.go: 1

1340-1343: LGTM!

relay/channel/claude/relay_claude_test.go: 1

35-44: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

⚠️ Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Name the expected values in the table.

The cases use bare numbers such as 1200, 2, and 20000. These numbers depend on the SetEstimatePromptTokens(1200) call, the tokenizer output for "hello", and the cache fields in the JSON. Define named constants, for example estimatedPromptTokens, helloCompletionTokens, and oneHourCacheWriteTokens. Use them in the table and in the JSON strings. This keeps the test valid if the inputs change.

Source: Learnings


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1835e8a9-06ea-407c-a6d2-88526514b4a8
📥 Commits

Reviewing files that changed from the base of the PR and between 1d47561 and 50bf421.

📒 Files selected for processing: 2
  • relay/channel/claude/relay_claude_test.go
  • service/text_quota_test.go

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

Trusting every explicit zero let an upstream that reports zeros stream
output for free, and a zero in message_delta erased the input counted
at message_start. Decide from the counts instead: input is missing only
when input, cache reads and cache writes are all zero, and output only
when it is zero after streamed content. A broken stream still takes the
larger of the reported and estimated output. A fully cached prompt and
a refusal before any output keep their zeros, so relaykit usage parsing
goes back to its previous form.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔇 Additional comments: 2
relay/channel/claude/relay-claude.go: 1

264-265: LGTM!

relay/channel/claude/relay_claude_test.go-42-42: 1

42-42: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

⚠️ Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Replace httptest.NewRequest with httptest.NewRequestWithContext.

The noctx linter reports this call as an error at Line 42. It can fail the lint step. Pass a context explicitly.

Proposed fix
-			c.Request = httptest.NewRequest(http.MethodPost, "/v1/messages", nil)
+			c.Request = httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/v1/messages", nil)

t.Context() requires Go 1.24 or later. go.mod declares Go 1.26.

Source: Linters/SAST tools


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c004f05b-6cad-4e15-80ff-7f4d092fc762
📥 Commits

Reviewing files that changed from the base of the PR and between 50bf421 and 6817da7.

📒 Files selected for processing: 2
  • relay/channel/claude/relay-claude.go
  • relay/channel/claude/relay_claude_test.go

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@Calcium-Ion
Calcium-Ion merged commit 1b63c28 into QuantumNous:main Oct 14, 2026
3 checks passed
shirosoralumie648 added a commit to shirosoralumie648/newapi-rev that referenced this pull request Oct 17, 2026
Sync 54 upstream commit(s) via merge (fork history preserved).
Behavior changes from upstream:
- 4013729 fix: disable sensitive word filtering by default and clear the default word list (QuantumNous#7814)

Upstream commits:
- 5e95951 feat(channels): add drag mode for channel priority ordering
- 5a59d69 fix(pricing): hide task prices left at zero when another price is set
- 3e2ce2c fix: align model card summary with status bar hours (QuantumNous#7847)
- b0a98ac fix(web): apply theme class before first paint so Dark Reader detects dark mode (QuantumNous#7571)
- f92c576 fix(web): cover Cloudflare clef and NVIDIA model families in provider rules (QuantumNous#7840)
- ff36846 fix(relayconvert): 保留 Chat/Claude → Responses 转换中 input_image 的 detail 字段 (QuantumNous#7855)
- 09109ec fix(middleware): 过载哨兵改用容器口径并加入连续性与新鲜度判定 (QuantumNous#7317)
- d032204 feat(aws): add Mantle Messages API support
- ca2cae6 fix(relaykit): emit summary on reasoning items announced without one (QuantumNous#7833)
- 45fbb14 fix(rate-limit): count each group with its own limit separately
- 6549a17 fix(dashboard): draw flow chart labels in the theme text color (QuantumNous#7821)
- c9179d3 fix(playground): support the reasoning field in chat responses (QuantumNous#7831)
- 908f94d fix(web): hide pages the user cannot open from console search
- 19d435e fix(relay): mask only the custom upstream address in error messages (QuantumNous#7822)
- 718a4e7 fix(vertex): use the multi-region hostnames for us and eu
- f196934 feat(plugins): bind Hailuo, Vidu and Veo plugins to New API channels
- bfdad29 fix(vidu): bill the model, duration and resolution Vidu generates
- 150ab3c chore(licenses): refresh the third-party dependency inventory
- 76cd511 build: upgrade moejs to v0.1.0-alpha.7
- 1b63c28 fix(claude): preserve explicit zero usage and cache-only billing (QuantumNous#7816)
- c9f7b62 fix(web): sort pricing models by input, output and per-request price (QuantumNous#7813)
- a0bf6a8 fix(vertex): pass project_id in API Key mode to avoid 404 on official endpoints (QuantumNous#6330)
- 4013729 fix: disable sensitive word filtering by default and clear the default word list (QuantumNous#7814)
- 7de10f2 feat(token): let an API key read its owner's quota
- ef60ef2 fix(relay): keep the cause of a failed body read for every later reader
- d10eafb fix(relay): report the real cause when a request body cannot be read
- 63cecab feat(channel): support Sub2API balance queries (QuantumNous#7749)
- b4be2a7 feat(ui): show newly added pricing groups at the top (QuantumNous#7782)
- 633b4c5 feat(logs): export usage logs from the common log page (QuantumNous#7774)
- dd59854 chore(deps-dev): bump undici from 6.28.0 to 6.29.0 in /electron (QuantumNous#7624)
- 93cf4ba chore(deps-dev): bump js-yaml from 4.3.1 to 4.3.2 in /electron (QuantumNous#7327)
- 3c9fb2a chore(deps-dev): bump brace-expansion from 1.1.16 to 1.1.21 in /electron (QuantumNous#7623)
- 9f04810 chore(deps): bump ai from 7.0.31 to 7.0.136 in /web (QuantumNous#7767)
- 3fe394c chore(deps): bump github.com/gorilla/websocket from 1.5.0 to 1.5.3 (QuantumNous#7001)
- 1c58f3d chore(deps-dev): bump fast-uri from 3.1.5 to 3.1.7 in /electron (QuantumNous#7169)
- 7f51a97 chore(deps-dev): bump js-yaml from 4.3.1 to 4.3.2 in /electron (QuantumNous#7327)
- 7e25797 chore(deps-dev): bump electron from 39.8.10 to 41.10.6 in /electron (QuantumNous#7625)
- bf4e1eb build(deps-dev): bump oxfmt from 0.57.0 to 0.72.0 in /web (QuantumNous#7766)
- 82306b3 build(deps-dev): bump @tanstack/router-plugin in /web (QuantumNous#7768)
- c730add build(deps): bump stream-markdown-parser from 1.1.3 to 1.2.18 in /web (QuantumNous#7769)
- b8c2dc1 build(deps): bump yace from 1.1.0 to 1.1.1 in /web (QuantumNous#7770)
- ba29f06 chore(deps-dev): bump @xmldom/xmldom from 0.8.13 to 0.8.15 in /electron (QuantumNous#7187)
- 767a82b fix(claude): forward thinking.block_binding and every anthropic-beta value (QuantumNous#7732)
- 2b85a8f fix(auth): show legal consent warning on password login (QuantumNous#7771)
- 2ed0587 fix(ci): stabilize checks and configure Bun dependency updates (QuantumNous#7765)
- 9948b1e fix(pricing): accept declared endpoint arrays in pricing conversion (QuantumNous#7724)
- 4bfcf28 chore(web): bump @lobehub/icons to 5.22.0 (QuantumNous#7764)
- fc08b76 fix(web): preserve recorded billing tiers in usage log previews (QuantumNous#7763)
- ba0e478 fix(relaykit): preserve assistant text with tool calls (QuantumNous#6871)
- 34b1942 fix(pricing): allow root access when the model square is disabled (QuantumNous#7761)
- b22a43b fix: mark task settlement logs with is_task for tiered billing display (QuantumNous#7446)
- bbf60b4 fix(logs): vertically align billing source icons on mobile (QuantumNous#7760)
- 3f926c3 fix(metrics): count completed streams after context cancellation (QuantumNous#7638)
- 3827fd8 fix(web): show cache read price on expression model cards (QuantumNous#7757)

Local tests not run (CI-only).
jahnli pushed a commit to jahnli/new-api that referenced this pull request Oct 17, 2026
Upstream scope: 22 commits / 123 files since the last sync (ef60ef2). 19 conflicted files were resolved one by one with explicit confirmation: 9 manual merges, 3 files taken from upstream (console search refactor plus its new test), and key-set unions in all 7 locale files. Every auto-merged file was reviewed one by one and both sides' changes are preserved (line-level check: no local line and no upstream line lost). No upstream deletions.

Local behavior kept: QuantumNous#2 AIGatewayError naming across relaykit and host error paths, QuantumNous#23/QuantumNous#24 LDAP login and LDAP default login, QuantumNous#27 avatar_url, QuantumNous#29/QuantumNous#31 removed OAuth providers, QuantumNous#37 subscription and violation fee flow, QuantumNous#39 user management table, QuantumNous#45/QuantumNous#69/QuantumNous#71 data overview access and BP roles, QuantumNous#49 pricing currency, QuantumNous#51/QuantumNous#56/QuantumNous#65 profile, audit and demo mode, QuantumNous#54/QuantumNous#55/QuantumNous#57 pricing square and dashboard/four-token stats, QuantumNous#58 online image generation, QuantumNous#60 API key interactions, QuantumNous#61 auto subscribe, QuantumNous#62 admin entry permissions, QuantumNous#77 removed frontend update check, seven-locale custom copy, header-removal convention.

Upstream adopted: QuantumNous#7822 upstream address masking (upstreamURLs plumbing through relaykit error paths, task errors, WebSocket error events, playground/relay/gemini/chat_via_responses/violation fee), QuantumNous#7317 container cgroup overload sentinel with consecutive and freshness sampling, QuantumNous#7816 Claude explicit zero usage and cache-only billing, QuantumNous#7813 pricing sorts by input/output/per-request price, QuantumNous#7840 provider rules for Cloudflare and NVIDIA, QuantumNous#7814 sensitive word filtering off by default, AWS Mantle Messages API, Vertex multi-region hostnames and API key project_id, Hailuo/Vidu/Veo plugin channel bindings and Vidu billing, per-group rate limits, API key owner quota query, moejs v0.1.0-alpha.7, theme class applied before first paint, console search limited to sidebar pages.

Collateral decisions (user approved): removed local test web/src/components/__tests__/command-menu-permissions.test.tsx (superseded by upstream command-menu.test.tsx) and added the newly required allow_user_quota_query field to the web/src/features/keys/components/__tests__/api-key-timestamp-cell.test.tsx fixture.

Verification: go build ./... exit 0; GOWORK=off go build ./... in relaykit exit 0; gofmt clean on modified Go files; web bun run typecheck exit 0; oxlint clean on the 49 merged web files; bun run i18n:sync reports missingCount 0 and extrasCount 0 for all seven locales; no conflict markers left; every upstream commit's added lines are present.

Not verified: SQLite/MySQL/PostgreSQL migration matrix for the new tokens.allow_user_quota_query column, unit tests (not run per project rule), web production build, runtime manual checks.
lizhongyi1209 pushed a commit to lizhongyi1209/new-api that referenced this pull request Oct 17, 2026
…ntumNous#7816)

* fix(claude): preserve explicit zero usage during stream finalization

* test: use synthetic values for Claude usage regressions

* fix(claude): estimate stream usage only when the counts are missing

Trusting every explicit zero let an upstream that reports zeros stream
output for free, and a zero in message_delta erased the input counted
at message_start. Decide from the counts instead: input is missing only
when input, cache reads and cache writes are all zero, and output only
when it is zero after streamed content. A broken stream still takes the
larger of the reported and estimated output. A fully cached prompt and
a refusal before any output keep their zeros, so relaykit usage parsing
goes back to its previous form.

---------

Co-authored-by: CaIon <i@caion.me>
yaser-k added a commit to yaser-k/new-api-fork that referenced this pull request Oct 17, 2026
…illing

Bring in upstream v1.0.0-rc.45 through merge/stream-cut-signal. Upstream's
1b63c28 (Refs QuantumNous#7816) is now in the base, so of f8de1de, its cherry-pick,
nothing remains; relay-claude.go and relay_claude_test.go merge cleanly,
HandleStreamFinalResponse matching this branch's resolution.

service/text_quota_test.go: both sides add the same
TestClaudeCacheOnlyUsageRemainsBillable; keep it once, followed by
TestClaudeRefusalWaiverSettlesCacheOnlyUsageAtZero.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XKGXUoEzejtyhoP7h2nJU6
kyuhwi-choi pushed a commit to attodev/new-api that referenced this pull request Oct 30, 2026
…ntumNous#7816)

* fix(claude): preserve explicit zero usage during stream finalization

* test: use synthetic values for Claude usage regressions

* fix(claude): estimate stream usage only when the counts are missing

Trusting every explicit zero let an upstream that reports zeros stream
output for free, and a zero in message_delta erased the input counted
at message_start. Decide from the counts instead: input is missing only
when input, cache reads and cache writes are all zero, and output only
when it is zero after streamed content. A broken stream still takes the
larger of the reported and estimated output. A fully cached prompt and
a refusal before any output keep their zeros, so relaykit usage parsing
goes back to its previous form.

---------

Co-authored-by: CaIon <i@caion.me>
(cherry picked from commit 1b63c28)
timwenx pushed a commit to timwenx/new-api that referenced this pull request Nov 18, 2026
…ntumNous#7816)

* fix(claude): preserve explicit zero usage during stream finalization

* test: use synthetic values for Claude usage regressions

* fix(claude): estimate stream usage only when the counts are missing

Trusting every explicit zero let an upstream that reports zeros stream
output for free, and a zero in message_delta erased the input counted
at message_start. Decide from the counts instead: input is missing only
when input, cache reads and cache writes are all zero, and output only
when it is zero after streamed content. A broken stream still takes the
larger of the reported and estimated output. A fully cached prompt and
a refusal before any output keep their zeros, so relaykit usage parsing
goes back to its previous form.

---------

Co-authored-by: CaIon <i@caion.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Claude stream finalization replaces explicit zero usage with local estimates

2 participants