Skip to content

Preserve Anthropic cache creation TTL details - #8174

Merged
Wei-Shaw merged 15 commits into
Wei-Shaw:mainfrom
wucm667:fix/issue-8172-anthropic-cache-ttl
Nov 18, 2026
Merged

Wei-Shaw merged 15 commits into
Wei-Shaw:mainfrom
wucm667:fix/issue-8172-anthropic-cache-ttl

Conversation

@wucm667

@wucm667 wucm667 commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

问题

Anthropic 缓存创建信息在响应转换过程中未完整保留,导致缓存 TTL 细节无法稳定传递。

根因

响应类型与网关转发路径对缓存创建字段的表达不一致,部分字段在转换过程中被遗漏。

改动

统一相关响应类型与转换逻辑,保留 Anthropic 缓存创建 TTL 细节,并补充兼容性、转发及用量回归覆盖。

验证

  • 定向兼容性测试通过。
  • 定向服务回归测试通过。
  • 相关包与服务的完整单元测试通过。
  • 相关包与服务的静态检查通过。
  • 依赖安全检查通过,未发现影响调用代码的问题。
  • 代码格式与差异检查通过。
  • 仓库级快捷测试因环境缺少必要命令未执行;全量单元测试尝试超过执行窗口且无最终结果。
  • 额外 lint 检查因本地版本与工具链导出数据不兼容未完成,未将其计为通过。

Fixes #8172

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical token-accounting bug in both parsing paths leaves uncached input unchanged after cache-only updates, inflating reported input totals and double-billing tokens.

1 open issue
1 file not reviewed
  • frontend/pnpm-lock.yaml: Generated file
What changed in this PR

This PR addresses #8172 by preserving Anthropic’s 5-minute and 1-hour cache-creation details through gateway conversion, usage logging, and billing.

Changes:

  • Track missing, null, and explicit-zero cache usage while merging partial streaming snapshots.
  • Carry TTL details into billing and account statistics, with unmatched aggregate tokens billed using the 5-minute fallback.
  • Add conversion and billing regressions, and update the frontend follow-redirects dependency override.
File Description
frontend/​pnpm-lock.yaml Lock follow-redirects to 1.16.2.
frontend/​package.json Raise the follow-redirects override minimum.
backend/​internal/​service/​openai_gateway_usage.go Include TTL details in billing and logs.
backend/​internal/​service/​openai_gateway_service.go Extend OpenAI usage with TTL fields and aggregate presence.
backend/​internal/​service/​openai_gateway_responses_anthropic_native.go Normalize accumulated buffered Responses usage.
backend/​internal/​service/​openai_gateway_native_cache_creation_test.go Test native routes through billing and logging.
backend/​internal/​service/​openai_gateway_messages_anthropic_native.go Preserve TTL fields when converting internal usage.
backend/​internal/​service/​openai_gateway_chat_completions_anthropic_native.go Normalize accumulated buffered Chat usage.
backend/​internal/​service/​kimi_anthropic_usage_test.go Test partial TTL updates and explicit clearing.
backend/​internal/​service/​gateway_usage_billing.go Share effective usage projection across billing and statistics.
backend/​internal/​service/​gateway_upstream_response.go Track aggregate presence and reconcile TTL overrides.
backend/​internal/​service/​gateway_service.go Track explicit cache-creation aggregate presence.
backend/​internal/​service/​gateway_forward_as_responses.go Merge cache details and normalize converted usage.
backend/​internal/​service/​gateway_forward_as_responses_test.go Cover TTL retention and aggregate edge cases.
backend/​internal/​service/​gateway_forward_as_chat_completions.go Normalize buffered Chat usage before conversion.
backend/​internal/​service/​gateway_forward_as_chat_completions_test.go Assert buffered TTL retention.
backend/​internal/​service/​gateway_cache_ttl_override_test.go Test overrides across forwarding and billing paths.
backend/​internal/​service/​gateway_cache_creation_usage_test.go Add parsing, forwarding, and billing regressions.
backend/​internal/​service/​gateway_anthropic_passthrough.go Share presence-aware raw cache usage merging.
backend/​internal/​service/​gateway_anthropic_apikey_passthrough_test.go Cover partial snapshots and fallback behavior.
backend/​internal/​service/​billing_service.go Allocate unclassified cache creation to 5-minute pricing.
backend/​internal/​service/​billing_service_test.go Verify partial-detail billing allocation.
backend/​internal/​service/​antigravity_gateway_upstream.go Preserve complete usage and aggregate presence.
backend/​internal/​service/​antigravity_gateway_service_test.go Update expected aggregate-presence state.
backend/​internal/​service/​anthropic_chat_stream_usage_test.go Cover contradictory, zero, and partial cache usage.
backend/​internal/​service/​account_stats_pricing.go Reuse cache billing allocation for statistics.
backend/​internal/​service/​account_stats_pricing_test.go Test TTL allocation under custom pricing.
backend/​internal/​pkg/​apicompat/​types.go Add TTL fields and presence-aware JSON serialization.
backend/​internal/​pkg/​apicompat/​responses_to_anthropic.go Mark converted aggregate totals as explicit.
backend/​internal/​pkg/​apicompat/​responses_anthropic_cache_creation_test.go Test TTL decoding, merging, and conversion.
backend/​internal/​pkg/​apicompat/​chatcompletions_anthropic_bridge.go Preserve explicit aggregate totals in Chat conversion.
backend/​internal/​pkg/​apicompat/​anthropic_to_responses_response.go Derive missing aggregates and merge streaming TTL details.
backend/​internal/​pkg/​apicompat/​anthropic_cache_creation_usage_test.go Test presence, JSON round trips, and partial snapshots.

Review effort: Balanced

Comment on lines +351 to +352
if !dst.cacheCreationInputTokensPresent && (src.CacheCreation.Ephemeral5mInputTokens != nil || src.CacheCreation.Ephemeral1hInputTokens != nil) {
dst.CacheCreationInputTokens = dst.CacheCreation5mTokens + dst.CacheCreation1hTokens

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已在 9e4c03d 修复:保留权威 prompt 总量,TTL-only 更新后重算非缓存输入;原生独立 input 桶不做额外扣减。同步处理缓存读取显式零、缺失/null 与重复快照,补充 typed/raw/generic 解析、流式终态/EOF、缓冲和计费边界回归。本地完整检查及当前 head 的远端 CI、安全检查均已通过。

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

在当前 82395b0 复核确认:权威 prompt 总量保留,cache-only/TTL-only 更新后重算非缓存输入,原生 Anthropic 独立 input 桶不重复扣减;typed/raw/generic 与计费边界回归通过。本轮另补 Chat 显式零写缓存与 Responses 终态 usage 优先级的一致性回归。精确当前 head 的完整 CI、安全及 CLA 检查均已终态通过(cla-lock 按工作流跳过)。

@Wei-Shaw

Wei-Shaw commented Nov 4, 2026

Copy link
Copy Markdown
Owner

感谢更新。上轮两点已确认修复:明细全 0 不再清零总量;明细之和大于总量不再改写总量,计费以总量为准、明细截断后差额补到 5m,不变式成立。覆盖面也比 #8237 完整。剩余几处计费口径变化需要说明或调整:

1. 上游显式总量为 0、明细非 0 时,现在计 0

billingUsageTokens(gateway_usage_billing.go、openai_gateway_usage.go)在 cacheCreationInputTokensPresent && CacheCreationInputTokens == 0 时把 5m/1h 清零,最终计费 0。main 在这种情况下按明细计费(bc4a9ae43 的 "zero aggregate unchanged" 有意保留,透传路径还会用 5m+1h 回填)。

另外 PR 删掉了 TestGatewayService_ParseSSEUsagePassthrough_MessageStartFallbacks、TestParseClaudeUsageFromResponseBody 输入中的 "cache_creation_input_tokens":0,断言值未变——这让行为变化在既有测试中不可见。请说明是否有上游真实这样返回;若无证据,建议保留 main 的处理(总量 ≤0 时按明细),并恢复这两个测试输入。

2. message_delta 中显式 0 现在会覆盖之前的正值

main 对 delta 中的 cache_creation_input_tokens/cache_read_input_tokens 要求 v > 0(gateway_upstream_response.go:1249-1256,透传与 typed 路径同理),PR 改为 >= 0。如果某个兼容上游在 message_start 报正值、delta 里用 0 占位,缓存读取/写入会被覆盖为 0。官方 Anthropic 的 delta 是累计值不受影响。请说明这一改动的依据(#8407 讨论的是 input_tokens 显式 0)。

此外几处规则不一致:delta 上 input_tokens 仍是 v > 0;antigravity_gateway_upstream.go 中 creation 改为 >= 0、read 仍是 > 0。

3. 客户端可见 JSON 的变化

AnthropicUsage.MarshalJSON 只在存在标记或非 0 时输出 cache_read_input_tokens,而各转换器(Chat/Responses → Anthropic)没有设置读取的存在标记,message_start 上连 creation 标记也没设,读取为 0 时字段被省略(main 总是输出)。rewriteCacheCreationJSON 在原本没有 cache_creation 对象时会新建一个。建议转换器出口保持与 main 一致总是输出这两个字段。

4. 与 #8237 互斥

两者修同一个 issue(#8172),都给 AnthropicUsage 加了同名 CacheCreation 字段但类型不同,无法同时合入,需二选一。

@wucm667

wucm667 commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

已核对 71532bf 的缓存口径修复:aggregate≤0 按 TTL 明细回填;恢复既有显式零输入;delta 零占位不覆盖正值;合成 Anthropic 输出零值缓存字段,不凭空新增 cache_creation。当前 4656364 同步 main 后修正严格 EOF 回归,断流必须报 502 且保留已采集 usage;隔离分配守卫避免并行噪声但不放宽断言。本地完整检查、当前 head 远端 CI 与安全检查均通过。#8237 仍为互斥实现,未混入其改动,需维护者择一。

@wucm667

wucm667 commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

已在 661770b 整合最新 main 并解决缓存计费冲突:保留新增 OpenAI 30m/12h 口径,同时保留本 PR 的 Anthropic 5m/1h 明细与差额分配,新增两套 TTL 计费和统计并存回归。此前关于零总量、delta 零占位及客户端 JSON 的修复保留。独立复核、本地完整检查及当前 head 远端 CI/安全检查通过。#8237 仍为互斥候选,等待维护者择一。

@wucm667

wucm667 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

已在 82395b0 完成本轮复核与补充:

  • 上轮 owner 指出的 aggregate≤0 按 TTL 明细回填、恢复显式零 fixture、delta 零占位保留正值,以及合成 Anthropic JSON 的零值缓存字段规则均保留;这些不是本轮重新发现的未修复项。
  • 补齐两个相邻边界:Chat 显式 cache_write_tokens:0 不再被冲突别名覆盖;Responses 终态同时带顶层及嵌套 usage 时,采集计费与客户端输出一致采用嵌套快照。增加直接/适配转换、流式/缓冲、wire/采集/计费一致性回归。
  • Go 升级到 1.27.4,保持与 main 的安全基线一致。针对合并 main 后的 EOF 回归,区分出字前 502 failover 与出字后缺失终态错误,不放宽断流要求,也不伪造成功终态。

本地在 Go 1.27.4 串行执行完整 unit/integration、边界回归、vet、lint、安全扫描、前端检查/构建、后端构建、依赖审计和发布辅助检查均通过;Apple 脚本本地使用 Linux BSD 兼容辅助验证,远端 macOS shell 检查通过。当前精确 head 的 CI(44228220057)、Security Scan(44228220027)及 CLA 检查已终态通过,cla-lock 按工作流跳过。

#8237 仍为互斥候选,未混入其实现,需维护者择一。未合并或开启自动合并。

@Wei-Shaw
Wei-Shaw merged commit 2354c78 into Wei-Shaw:main Nov 18, 2026
9 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Anthropic→Chat/Responses 转换丢失 5m/1h 缓存创建明细,1h 写入回退按 5m 计费

3 participants