Skip to content

feat(gemini): support cached token billing - #2798

Merged
seefs001 merged 1 commit into
QuantumNous:mainfrom
RedwindA:feat/GeminiCacheBilling
Feb 2, 2026
Merged

seefs001 merged 1 commit into
QuantumNous:mainfrom
RedwindA:feat/GeminiCacheBilling

Conversation

@RedwindA

@RedwindA RedwindA commented Feb 1, 2026 •

Copy link
Copy Markdown
Contributor

PR 类型

  • Bug 修复
  • 新功能
  • 文档更新
  • 其他

PR 是否包含破坏性更新?

  • 是
  • 否

PR 描述

958510e7-2aaa-4a14-b5ea-658f0d25678c cafbdab4ae6dc7899cfc5310568369ec

为 Gemini 实现缓存计费:新增 cachedContentTokenCount 字段解析,并在 Gemini 非流式/流式响应中写入 PromptTokensDetails.CachedTokens 以参与
用量计费;同时为 gemini-3-flash-preview 与 gemini-3-pro-preview 添加默认缓存计费比例,确保后台计费逻辑可直接使用。

Summary by CodeRabbit

  • New Features
    • Added cached content token tracking to usage metrics for Gemini models to provide more detailed token consumption information.
    • Added support for two new Gemini 3 preview model variants with optimized cache configurations.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai

coderabbitai Bot commented Feb 1, 2026 •

Copy link
Copy Markdown
Contributor

Walkthrough

The changes add support for tracking cached content tokens in Gemini API responses. A new field is introduced to the usage metadata DTO, handlers are updated to propagate this cached token data, and cache ratios are configured for new Gemini 3 model variants.

Changes

Cohort / File(s) Summary
Gemini Usage Metadata
dto/gemini.go
Added CachedContentTokenCount field to GeminiUsageMetadata struct with JSON serialization support.
Token Propagation in Handlers
relay/channel/gemini/relay-gemini-native.go, relay/channel/gemini/relay-gemini.go
Multiple assignments added across handlers to populate usage.PromptTokensDetails.CachedTokens from the new CachedContentTokenCount field in Gemini responses.
Cache Ratio Configuration
setting/ratio_setting/cache_ratio.go
Added cache ratio entries (0.25) for "gemini-3-flash-preview" and "gemini-3-pro-preview" model variants.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

Poem

🐰 A cached token hops along the stream,
Through Gemini's handlers, a metric dream!
From metadata to details it flows,
With flashy models in tidy rows. ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main feature: adding support for cached token billing in Gemini integration by parsing and propagating cached token counts.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@seefs001
seefs001 merged commit 76050e6 into QuantumNous:main Feb 2, 2026
ennnnny pushed a commit to ennnnny/new-api that referenced this pull request Mar 17, 2026
…ling

feat(gemini): support cached token billing
salem-2007 added a commit to salem-2007/new-api that referenced this pull request Sep 10, 2026
…ling

feat(gemini): support cached token billing
junjundesk pushed a commit to junjundesk/new-api that referenced this pull request Oct 3, 2026
…ling

feat(gemini): support cached token billing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants