Skip to content

feat: add cache hit display to dashboard and rankings - #5842

Open
kozora wants to merge 1 commit into
QuantumNous:mainfrom
kozora:main
Open

kozora wants to merge 1 commit into
QuantumNous:mainfrom
kozora:main

Conversation

@kozora

@kozora kozora commented Jul 1, 2026 •

Copy link
Copy Markdown

Add cache_read_tokens and cache_write_tokens to QuotaData model, extracting from Log.other on the write path. Propagate through rankings service. Dashboard gets Cache Read and Cache Hit Rate stat cards plus enriched chart tooltips. Rankings leaderboard shows cache total and hit rate percentage per model.

📝 变更描述 / Description

  • 添加/dashboard/models数据看板缓存命中情况

  • 添加排行榜界面模型缓存命中情况

🚀 变更类型 / Type of change

  • ✨ 新功能 (New feature) -

✅ 提交前检查项 / Checklist

  • 人工确认: 我已亲自整理并撰写此描述,没有直接粘贴未经处理的 AI 输出。
  • 非重复提交: 我已搜索现有的 Issues 与 PRs,确认不是重复提交。
  • 变更理解: 我已理解这些更改的工作原理及可能影响。
  • 范围聚焦: 本 PR 未包含任何与当前任务无关的代码改动。
  • 本地验证: 已在本地运行并通过测试或手动验证,维护者可以据此复核结果。
  • 安全合规: 代码中无敏感凭据,且符合项目代码规范。

📸 运行证明 / Proof of Work

image image

Summary by CodeRabbit

  • New Features

    • Added cache read/write token metrics across dashboards and rankings.
    • Introduced cache hit rate views and cache-related labels in charts, cards, and leaderboards.
    • Expanded ranking and history displays to show richer token breakdowns.
  • Bug Fixes

    • Improved aggregation so cache token totals are counted consistently in summaries and reports.
    • Refined number formatting and responsive card layout for clearer dashboard display.

Add cache_read_tokens and cache_write_tokens to QuotaData model,
extracting from Log.other on the write path. Propagate through
rankings service. Dashboard gets Cache Read and Cache Hit Rate stat
cards plus enriched chart tooltips. Rankings leaderboard shows cache
total and hit rate percentage per model.
@coderabbitai

coderabbitai Bot commented Jul 1, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Adds cache read/write token tracking end-to-end: consume-log recording extracts cache tokens, quota data models persist and aggregate them, ranking queries and the rankings service propagate the values, and dashboard/rankings frontend components with new i18n strings display cache hit rates and totals.

Changes

Cache Token Metrics End-to-End

Layer / File(s) Summary
Capture cache tokens in consume log
model/log.go
Adds toIntValue coercion helper and updates RecordConsumeLog to compute cacheReadTokens/cacheWriteTokens from params.Other, with fallback to cache_creation_tokens.
Persist and aggregate in QuotaData model
model/usedata.go
Adds CacheReadTokens/CacheWriteTokens fields, updates cache merging, LogQuotaData, DB increments, and centralized aggregate SELECT queries.
Aggregate cache tokens in ranking queries
model/usedata_rankings.go
Adds cache fields to RankingQuotaTotal/RankingQuotaBucket and sums them in their respective queries.
Propagate through rankings service
service/rankings.go
Extends RankedModel, RankedVendor, ModelHistoryPoint, VendorSharePoint/VendorShareVendor, vendorAggregate with cache fields; updates history/vendor builders to compute and populate them.
Dashboard types and stats aggregation
web/default/src/features/dashboard/types.ts, .../lib/stats.ts, .../lib/charts.ts
Adds optional cache token fields to dashboard item types, aggregates cache totals in stats, and extends chart rank-bar/tooltip logic with cache metrics.
Dashboard stat cards UI
.../hooks/use-dashboard-config.tsx, .../components/models/log-stat-cards.tsx
Adds cacheRead/cacheHitRate stat card configs with new icons, and updates card formatting/grid layout to show cache metrics.
Rankings frontend types and leaderboard display
web/default/src/features/rankings/types.ts, .../components/model-leaderboard.tsx
Adds optional cache fields to ranking/history types and renders cache hit rate/totals conditionally in leaderboard rows.
Localization strings
web/default/src/i18n/locales/en.json, .../zh.json
Adds English/Chinese translations for Cache Hit Rate, cache read share, and cached input tokens.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Possibly related PRs

  • QuantumNous/new-api#3400: Refactors text quota billing to compute cache-token breakdown and pass it into RecordConsumeLog's Other metadata, directly aligned with the same consume-log cache token data flow.

Suggested reviewers: seefs001

A rabbit hops through caches deep,
Counting reads and writes it keeps,
Tokens tallied, charts now bloom,
Hit-rate glowing in the room. 🐇📊
From logs to boards, the cache trail flows,
Hop hop — every metric shows!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 14.29% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main user-facing change: adding cache hit display to dashboard and rankings.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
web/default/src/features/dashboard/lib/charts.ts (1)

729-752: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cache hit rate formula duplicated between this tooltip and the dashboard stat card.

The (CacheRead / Tokens) * 100 calculation here is re-implemented independently in use-dashboard-config.tsx's cacheHitRate card (via safeDivide). Both need to stay semantically identical (cache-read share of total tokens); extracting a small shared helper (e.g. calculateCacheHitRate(read, total)) would prevent the two from silently drifting if precision or the guard condition changes later.

♻️ Proposed shared helper
// e.g. in a shared lib file
export function calculateCacheHitRatePct(cacheRead: number, totalTokens: number): number {
  if (totalTokens <= 0) return 0
  return (cacheRead / totalTokens) * 100
}

Then in charts.ts:

-              value: (datum: Record<string, unknown>) => {
-                const tokens = Number(datum?.Tokens) || 0
-                const read = Number(datum?.CacheRead) || 0
-                if (tokens <= 0) return '0%'
-                return `${((read / tokens) * 100).toFixed(1)}%`
-              },
+              value: (datum: Record<string, unknown>) => {
+                const tokens = Number(datum?.Tokens) || 0
+                const read = Number(datum?.CacheRead) || 0
+                return `${calculateCacheHitRatePct(read, tokens).toFixed(1)}%`
+              },
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@web/default/src/features/dashboard/lib/charts.ts` around lines 729 - 752, The
cache hit rate logic is duplicated in the tooltip formatter inside charts.ts and
the cacheHitRate card in use-dashboard-config.tsx, so both should use the same
shared helper. Extract the percentage calculation into a small function such as
calculateCacheHitRatePct(cacheRead, totalTokens) and call it from both the
charts tooltip value formatter and the dashboard stat card logic (including any
safeDivide-based path) so the guard condition and precision stay identical.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@web/default/src/features/rankings/components/model-leaderboard.tsx`:
- Around line 113-116: The vendor prefix text in model-leaderboard’s row
rendering is hardcoded, so this React component is still partially untranslated.
Update the component to use react-i18next by calling useTranslation() in
model-leaderboard.tsx and render the “by” prefix with t() alongside the existing
VendorLink/row.vendor output, matching the surrounding localized cache labels.

---

Nitpick comments:
In `@web/default/src/features/dashboard/lib/charts.ts`:
- Around line 729-752: The cache hit rate logic is duplicated in the tooltip
formatter inside charts.ts and the cacheHitRate card in
use-dashboard-config.tsx, so both should use the same shared helper. Extract the
percentage calculation into a small function such as
calculateCacheHitRatePct(cacheRead, totalTokens) and call it from both the
charts tooltip value formatter and the dashboard stat card logic (including any
safeDivide-based path) so the guard condition and precision stay identical.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6b6e3902-59e2-40bf-a0ba-da58fcaa662c

📥 Commits

Reviewing files that changed from the base of the PR and between 52858ad and e7142af.

📒 Files selected for processing (13)
  • model/log.go
  • model/usedata.go
  • model/usedata_rankings.go
  • service/rankings.go
  • web/default/src/features/dashboard/components/models/log-stat-cards.tsx
  • web/default/src/features/dashboard/hooks/use-dashboard-config.tsx
  • web/default/src/features/dashboard/lib/charts.ts
  • web/default/src/features/dashboard/lib/stats.ts
  • web/default/src/features/dashboard/types.ts
  • web/default/src/features/rankings/components/model-leaderboard.tsx
  • web/default/src/features/rankings/types.ts
  • web/default/src/i18n/locales/en.json
  • web/default/src/i18n/locales/zh.json

Comment on lines +113 to +116
by{' '}
<VendorLink vendor={row.vendor}>
{row.vendor.toLowerCase()}
</VendorLink>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Localize the vendor prefix.

by is user-facing text in this React component, so this row stays partially untranslated even though the surrounding cache labels already go through t().

Suggested fix
-                by{' '}
+                {t('by')}{' '}

As per coding guidelines, "Frontend UI text must support i18n with i18next/react-i18next" and "web/default/**/*.{ts,tsx}: React components must call useTranslation() and render user-facing text with t()."

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
by{' '}
<VendorLink vendor={row.vendor}>
{row.vendor.toLowerCase()}
</VendorLink>
{t('by')}{' '}
<VendorLink vendor={row.vendor}>
{row.vendor.toLowerCase()}
</VendorLink>
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@web/default/src/features/rankings/components/model-leaderboard.tsx` around
lines 113 - 116, The vendor prefix text in model-leaderboard’s row rendering is
hardcoded, so this React component is still partially untranslated. Update the
component to use react-i18next by calling useTranslation() in
model-leaderboard.tsx and render the “by” prefix with t() alongside the existing
VendorLink/row.vendor output, matching the surrounding localized cache labels.

Source: Coding guidelines

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant