Repository navigation
Preserve Anthropic cache creation TTL details - #8174
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
A critical token-accounting bug in both parsing paths leaves uncached input unchanged after cache-only updates, inflating reported input totals and double-billing tokens.
1 open issue
1 file not reviewed
frontend/pnpm-lock.yaml: Generated file
What changed in this PR
This PR addresses #8172 by preserving Anthropic’s 5-minute and 1-hour cache-creation details through gateway conversion, usage logging, and billing.
Changes:
- Track missing, null, and explicit-zero cache usage while merging partial streaming snapshots.
- Carry TTL details into billing and account statistics, with unmatched aggregate tokens billed using the 5-minute fallback.
- Add conversion and billing regressions, and update the frontend
follow-redirectsdependency override.
| File | Description |
|---|---|
| frontend/pnpm-lock.yaml | Lock follow-redirects to 1.16.2. |
| frontend/package.json | Raise the follow-redirects override minimum. |
| backend/internal/service/openai_gateway_usage.go | Include TTL details in billing and logs. |
| backend/internal/service/openai_gateway_service.go | Extend OpenAI usage with TTL fields and aggregate presence. |
| backend/internal/service/openai_gateway_responses_anthropic_native.go | Normalize accumulated buffered Responses usage. |
| backend/internal/service/openai_gateway_native_cache_creation_test.go | Test native routes through billing and logging. |
| backend/internal/service/openai_gateway_messages_anthropic_native.go | Preserve TTL fields when converting internal usage. |
| backend/internal/service/openai_gateway_chat_completions_anthropic_native.go | Normalize accumulated buffered Chat usage. |
| backend/internal/service/kimi_anthropic_usage_test.go | Test partial TTL updates and explicit clearing. |
| backend/internal/service/gateway_usage_billing.go | Share effective usage projection across billing and statistics. |
| backend/internal/service/gateway_upstream_response.go | Track aggregate presence and reconcile TTL overrides. |
| backend/internal/service/gateway_service.go | Track explicit cache-creation aggregate presence. |
| backend/internal/service/gateway_forward_as_responses.go | Merge cache details and normalize converted usage. |
| backend/internal/service/gateway_forward_as_responses_test.go | Cover TTL retention and aggregate edge cases. |
| backend/internal/service/gateway_forward_as_chat_completions.go | Normalize buffered Chat usage before conversion. |
| backend/internal/service/gateway_forward_as_chat_completions_test.go | Assert buffered TTL retention. |
| backend/internal/service/gateway_cache_ttl_override_test.go | Test overrides across forwarding and billing paths. |
| backend/internal/service/gateway_cache_creation_usage_test.go | Add parsing, forwarding, and billing regressions. |
| backend/internal/service/gateway_anthropic_passthrough.go | Share presence-aware raw cache usage merging. |
| backend/internal/service/gateway_anthropic_apikey_passthrough_test.go | Cover partial snapshots and fallback behavior. |
| backend/internal/service/billing_service.go | Allocate unclassified cache creation to 5-minute pricing. |
| backend/internal/service/billing_service_test.go | Verify partial-detail billing allocation. |
| backend/internal/service/antigravity_gateway_upstream.go | Preserve complete usage and aggregate presence. |
| backend/internal/service/antigravity_gateway_service_test.go | Update expected aggregate-presence state. |
| backend/internal/service/anthropic_chat_stream_usage_test.go | Cover contradictory, zero, and partial cache usage. |
| backend/internal/service/account_stats_pricing.go | Reuse cache billing allocation for statistics. |
| backend/internal/service/account_stats_pricing_test.go | Test TTL allocation under custom pricing. |
| backend/internal/pkg/apicompat/types.go | Add TTL fields and presence-aware JSON serialization. |
| backend/internal/pkg/apicompat/responses_to_anthropic.go | Mark converted aggregate totals as explicit. |
| backend/internal/pkg/apicompat/responses_anthropic_cache_creation_test.go | Test TTL decoding, merging, and conversion. |
| backend/internal/pkg/apicompat/chatcompletions_anthropic_bridge.go | Preserve explicit aggregate totals in Chat conversion. |
| backend/internal/pkg/apicompat/anthropic_to_responses_response.go | Derive missing aggregates and merge streaming TTL details. |
| backend/internal/pkg/apicompat/anthropic_cache_creation_usage_test.go | Test presence, JSON round trips, and partial snapshots. |
Review effort: Balanced
| if !dst.cacheCreationInputTokensPresent && (src.CacheCreation.Ephemeral5mInputTokens != nil || src.CacheCreation.Ephemeral1hInputTokens != nil) { | ||
| dst.CacheCreationInputTokens = dst.CacheCreation5mTokens + dst.CacheCreation1hTokens |
There was a problem hiding this comment.
已在 9e4c03d 修复:保留权威 prompt 总量,TTL-only 更新后重算非缓存输入;原生独立 input 桶不做额外扣减。同步处理缓存读取显式零、缺失/null 与重复快照,补充 typed/raw/generic 解析、流式终态/EOF、缓冲和计费边界回归。本地完整检查及当前 head 的远端 CI、安全检查均已通过。
There was a problem hiding this comment.
在当前 82395b0 复核确认:权威 prompt 总量保留,cache-only/TTL-only 更新后重算非缓存输入,原生 Anthropic 独立 input 桶不重复扣减;typed/raw/generic 与计费边界回归通过。本轮另补 Chat 显式零写缓存与 Responses 终态 usage 优先级的一致性回归。精确当前 head 的完整 CI、安全及 CLA 检查均已终态通过(cla-lock 按工作流跳过)。
|
感谢更新。上轮两点已确认修复:明细全 0 不再清零总量;明细之和大于总量不再改写总量,计费以总量为准、明细截断后差额补到 5m,不变式成立。覆盖面也比 #8237 完整。剩余几处计费口径变化需要说明或调整: 1. 上游显式总量为 0、明细非 0 时,现在计 0
另外 PR 删掉了 2. message_delta 中显式 0 现在会覆盖之前的正值main 对 delta 中的 此外几处规则不一致:delta 上 3. 客户端可见 JSON 的变化
4. 与 #8237 互斥两者修同一个 issue(#8172),都给 |
|
已在 82395b0 完成本轮复核与补充:
本地在 Go 1.27.4 串行执行完整 unit/integration、边界回归、vet、lint、安全扫描、前端检查/构建、后端构建、依赖审计和发布辅助检查均通过;Apple 脚本本地使用 Linux BSD 兼容辅助验证,远端 macOS shell 检查通过。当前精确 head 的 CI(44228220057)、Security Scan(44228220027)及 CLA 检查已终态通过,cla-lock 按工作流跳过。 #8237 仍为互斥候选,未混入其实现,需维护者择一。未合并或开启自动合并。 |

问题
Anthropic 缓存创建信息在响应转换过程中未完整保留,导致缓存 TTL 细节无法稳定传递。
根因
响应类型与网关转发路径对缓存创建字段的表达不一致,部分字段在转换过程中被遗漏。
改动
统一相关响应类型与转换逻辑,保留 Anthropic 缓存创建 TTL 细节,并补充兼容性、转发及用量回归覆盖。
验证
Fixes #8172