Repository navigation
fix: correct cache rate calculation by excluding cache_creation_tokens from denominator - #8018
Open
yikang-shen wants to merge 1 commit into
Open
yikang-shen wants to merge 1 commit into
yikang-shen wants to merge 1 commit into
Conversation
…s from denominator Current channel monitor v2 calculates cache rate as: cache_read / (input + cache_creation + cache_read) This incorrectly penalizes requests that write new cache entries. Cache creation represents tokens being indexed for future reuse, not cache misses. A request can simultaneously read from existing cache AND write new cache entries. Example with current (incorrect) formula: cache_read: 132,627 | input: 50 | cache_creation: 47,869 Displayed rate: 132,627 / (50 + 47,869 + 132,627) = 58.4% ❌ Misleading - this request actually had 99.96% cache coverage Standard cache hit rate definition: cache_read / (cache_read + input) This correctly measures what percentage of required input came from cache. Changes: - Remove cache_creation from denominator in metricAccumulator.metric() - Denominator now: input + cache_read (tokens that needed to be processed) - Cache creation tokens still tracked separately for cost analysis Impact: - Cache rates will increase significantly for workloads with active cache writing - Metrics now align with Anthropic's usage model and standard cache semantics - No breaking changes to data collection or API schema Tested with production data (Response API, 2026-10-12): Before: 82.87% displayed | After: 99.65% (matches actual cache coverage) Reference: Anthropic Prompt Caching tracks cache_creation_input_tokens and cache_read_input_tokens separately - creation is not a "miss".
riofutab
added a commit
to riofutab/sub2api
that referenced
this pull request
Oct 14, 2026
riofutab
added a commit
to riofutab/sub2api
that referenced
this pull request
Oct 14, 2026
- OpenAI/Codex 稳定性:OAuth 缺过期时间时按 JWT 判定并在 WS 上恢复 401(Wei-Shaw#8034,替换 fork 原有的 Wei-Shaw#7826 实现)、模型拒绝不计入调度错误率(Wei-Shaw#8013)、首包前预算耗尽换号(Wei-Shaw#8005)、Sol 别名误判修正(Wei-Shaw#8063) - 流与协议:原生 Claude 转 Chat 转发心跳(Wei-Shaw#8052)、Codex 流式出图兼容 JSON 响应(Wei-Shaw#8028)、Antigravity 工具结果图片(Wei-Shaw#8045) - 用量显示:5h 换窗保留未到期周额度(Wei-Shaw#7990)、缓存命中率口径修正与单次命中率显示(Wei-Shaw#8018、Wei-Shaw#7995) 测试:go test -tags=unit ./... 全部通过,golangci-lint 0 issues;前端 make test-frontend 通过;集成测试交 CI。
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Current channel monitor v2 calculates cache rate incorrectly:
This penalizes requests that write new cache entries. Cache creation represents tokens being indexed for future reuse, not cache misses. A request can simultaneously read from existing cache AND write new cache entries.
Example (production data, 2026-10-12)
Request with:
cache_read: 132,627 tokensinput: 50 tokenscache_creation: 47,869 tokensCurrent (incorrect) calculation:
❌ Misleading - this request actually had 99.96% cache coverage!
Correct calculation:
✅ Accurately represents that 99.96% of required input came from cache
Solution
Use standard cache hit rate definition:
This correctly measures what percentage of required input tokens came from cache.
Changes
cache_creationfrom denominator inmetricAccumulator.metric()input + cache_read(tokens that needed to be processed)Impact
CacheRateNumeratorandCacheRateDenominatorfields updated consistentlyValidation
Tested with production Response API data (2026-10-12, 19 requests):
Aggregated totals:
cache_read: 1,951,088input: 6,942cache_creation: 396,240Reference
Anthropic Prompt Caching tracks
cache_creation_input_tokensandcache_read_input_tokensseparately - creation is not a "miss".The current formula incorrectly treats cache creation as a penalty rather than recognizing it as building the cache for future requests.