Skip to content

feat(dashboard): add cache token stats and analytics - #5195

Open
ChenXu233 wants to merge 20 commits into
QuantumNous:mainfrom
ChenXu233:feat/cache-tokens
Open

ChenXu233 wants to merge 20 commits into
QuantumNous:mainfrom
ChenXu233:feat/cache-tokens

Conversation

@ChenXu233

@ChenXu233 ChenXu233 commented May 30, 2026 •

Copy link
Copy Markdown

Summary

  • Add cache token fields (cache_tokens, cache_creation_tokens, cache_creation_tokens_5m, cache_creation_tokens_1h) to quota data model and logging
  • Add historical cache token backfill migration
  • Add cache token aggregation to dashboard queries
  • Add Cache Tokens stat card with i18n support
  • Add CacheTokenCharts component with 3 tabs (by model, by day, by hour)
  • Add cache token dimensions to dashboard charts
  • Redesign stat cards with core and derived tiers
  • Add cache token stats to classic dashboard

Closes #5069

Summary by CodeRabbit

  • New Features

    • Comprehensive cache analytics: trend, ranking, and non-cache token charts.
    • Tracks cache reads and cache-creation tokens (including 5m/1h breakdowns).
    • New "Cache Statistics" section showing totals and detailed breakdowns.
    • Dashboard cards and charts updated to display cache metrics alongside existing quota data.
  • Chores

    • Background backfill to populate historical cache metrics for accurate past reporting.

Review Change Stack

ChenXu233 added 17 commits May 30, 2026 15:22
Extend the chart data aggregation to include cache read and cache
creation token counts from QuotaDataItem. Both lineValues and
areaValues data points now carry cacheRead/cacheCreation fields so
charts can surface cache token breakdowns.
Cache Tokens shows breakdown: Read / 5m / 1h creation tokens.
Average TPM shows cache-excluded TPM value.
Copilot AI review requested due to automatic review settings May 30, 2026 07:22
@coderabbitai

coderabbitai Bot commented May 30, 2026 •

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 09403756-0e5c-46fd-8742-c3e4602ab2f0

📥 Commits

Reviewing files that changed from the base of the PR and between 8373215 and 4e704d7.

📒 Files selected for processing (1)
  • main.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • main.go

Walkthrough

This PR tracks cache read/creation token metrics end-to-end: log extraction, in-memory aggregation, DB increments, a migration to backfill existing rows, and UI/charting in both classic and default dashboards.

Changes

Cache Token Analytics & Dashboard

Layer / File(s) Summary
Backend cache metrics schema and aggregation
model/usedata.go
QuotaData gains cache fields; LogQuotaData cache accumulator, SaveQuotaDataCache flush, and increaseQuotaData DB increments extended to include cache token metrics; aggregation queries and ranking totals include cache sums.
Log export: extract cache metrics
model/log.go
RecordConsumeLog reads cache metrics from params.Other, converts numeric JSON types via toIntOrDefault, and forwards them to LogQuotaData; imports encoding/json.
Backfill migration for existing quota_data
model/migrate_cache_tokens.go, main.go
MigrateCacheTokens() pages quota_data rows with zero cache fields, aggregates per-row cache totals from logs (1h window) using DB-specific JSON SQL, and updates quota_data when totals are non-zero; started from InitResources.
Classic dashboard: cache metrics state and display
web/classic/src/hooks/dashboard/useDashboardData.js, web/classic/src/hooks/dashboard/useDashboardStats.jsx, web/classic/src/components/dashboard/index.jsx
Add consumeCacheRead/consumeCacheCreation state and expose them; extend useDashboardStats to accept and render a "缓存统计" section showing cache read and creation metrics; wire through Dashboard.
Default dashboard types, constants, and preferences
web/default/src/features/dashboard/types.ts, web/default/src/features/dashboard/constants.ts, web/default/src/features/dashboard/lib/filters.ts
Add CacheAnalyticsTab/ProcessedCacheChartData, extend QuotaDataItem with cache fields, add cacheAnalyticsChart preference and options, and validate/load persisted cacheAnalyticsChart.
Default dashboard stats aggregation and card config
web/default/src/features/dashboard/lib/stats.ts, web/default/src/features/dashboard/hooks/use-dashboard-config.tsx
calculateDashboardStats accumulates cache totals and time-window variants; StatCardConfig supports getDetail; core hook renamed to useCoreStatCards; derived cards provide cache breakdown/getDetail.
Default dashboard stat cards UI
web/default/src/features/dashboard/components/models/log-stat-cards.tsx
Refactor to use core/derived hooks, expand stats state with cache fields, add formatValue, and split core vs derived card rendering with conditional details.
Cache chart data processing
web/default/src/features/dashboard/lib/charts.ts, web/default/src/features/dashboard/lib/index.ts
Extend chart aggregators to include cacheRead/cacheCreation; add processCacheChartData to emit cache trend, ranking, and non-cache specs; re-export processor.
Cache token charts React component
web/default/src/features/dashboard/components/models/cache-token-charts.tsx
New CacheTokenCharts component with tabs (trend/ranking/nonCache), lazy theme manager, memoized processing, and VChart rendering guarded by theme readiness.
Default dashboard layout integration
web/default/src/features/dashboard/index.tsx
Lazy-load LazyCacheTokenCharts in models section with Suspense/FadeIn; wire data, loading, preference, and time granularity; adjust skeleton layout.
Localization for cache metrics
web/default/src/i18n/locales/en.json, web/default/src/i18n/locales/zh.json
Add translations for "Cache read + creation tokens", "Cache Tokens", and "Non-cache Tokens".

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • QuantumNous/new-api#2155: Extends cache-creation 5m/1h metrics elsewhere in the pipeline; related to propagation of those fields.
  • QuantumNous/new-api#3400: Also touches consume-log usage fields and cached creation token handling across logging/billing.

Suggested labels

enhancement

Suggested reviewers

  • seefs001
  • Calcium-Ion

Poem

🐰 Hop-hop, the logs now sing,
Cache reads and creations counted in spring.
Totals tallied, charts take flight,
Dashboards glow through day and night.
A rabbit cheers the metrics' new light!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 7.14% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding cache token statistics and analytics to the dashboard, which matches the changeset's primary objective.
Linked Issues check ✅ Passed All code requirements from issue #5069 are met: cache token fields added to quota_data, backfill migration implemented, aggregation included in queries, and frontend dashboard components created for cache stats visualization.
Out of Scope Changes check ✅ Passed All changes directly support the linked issue objectives: backend model/migration/logging changes enable cache token persistence, and frontend changes display these metrics per the requirements.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds cache-token visibility to the dashboard by extending quota aggregation (backend + frontend) and introducing new cache analytics charts/metrics.

Changes:

  • Extend quota_data aggregation to include cache read + cache creation token counters (API + UI types).
  • Add “Cache Token Analytics” charts (trend, ranking, non-cache tokens) and new stat card details.
  • Add a background backfill job to populate cache token columns in existing quota_data rows from usage logs.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
web/default/src/i18n/locales/en.json Adds new cache-related UI strings.
web/default/src/i18n/locales/zh.json Adds Chinese translations for new cache-related UI strings.
web/default/src/features/dashboard/types.ts Extends dashboard types for cache tokens and cache chart specs.
web/default/src/features/dashboard/lib/stats.ts Aggregates cache token totals in dashboard stats.
web/default/src/features/dashboard/lib/index.ts Exposes new processCacheChartData helper.
web/default/src/features/dashboard/lib/filters.ts Adds persisted preference parsing for cache analytics tab.
web/default/src/features/dashboard/lib/charts.ts Adds cache-token chart processing + extends model aggregation to include cache metrics.
web/default/src/features/dashboard/index.tsx Adds lazy-loaded cache token charts section.
web/default/src/features/dashboard/hooks/use-dashboard-config.tsx Splits stat cards into core + derived cards and adds cache-derived details.
web/default/src/features/dashboard/constants.ts Adds cache analytics chart options + default preference value.
web/default/src/features/dashboard/components/models/log-stat-cards.tsx Shows cache-related derived stats and details in the model stat cards UI.
web/default/src/features/dashboard/components/models/cache-token-charts.tsx New component rendering cache analytics charts.
web/classic/src/hooks/dashboard/useDashboardStats.jsx Adds classic dashboard section for cache read/creation.
web/classic/src/hooks/dashboard/useDashboardData.js Tracks cache read/creation counters in classic dashboard state.
web/classic/src/components/dashboard/index.jsx Wires new cache counters into classic dashboard stats hook.
model/usedata_rankings.go Adds cache token totals to model ranking query results.
model/usedata.go Extends QuotaData schema + aggregation/logging to include cache token fields.
model/migrate_cache_tokens.go New backfill routine to populate cache token columns from logs.
model/main.go Starts cache-token backfill during DB migration.
model/log.go Captures cache token values from consume logs into quota_data.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread model/log.go Resolved
Comment thread model/usedata.go Resolved
Comment thread model/usedata.go Resolved
Comment thread model/usedata_rankings.go Resolved
Comment thread model/migrate_cache_tokens.go Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (2)
web/default/src/features/dashboard/types.ts (1)

122-131: 💤 Low value

Reuse the existing VChartSpec alias instead of re-inlining Record<string, any>.

This file already defines type VChartSpec = Record<string, any> (line 105) and ProcessedChartData uses it. Reusing it here removes three repeated eslint-disable lines and keeps the chart-spec typing consistent.

♻️ Proposed cleanup
 export interface ProcessedCacheChartData {
-  // eslint-disable-next-line `@typescript-eslint/no-explicit-any`
-  spec_cache_trend: Record<string, any>
-  // eslint-disable-next-line `@typescript-eslint/no-explicit-any`
-  spec_cache_rank: Record<string, any>
-  // eslint-disable-next-line `@typescript-eslint/no-explicit-any`
-  spec_non_cache: Record<string, any>
+  spec_cache_trend: VChartSpec
+  spec_cache_rank: VChartSpec
+  spec_non_cache: VChartSpec
   totalCacheDisplay: string
   totalNonCacheDisplay: string
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@web/default/src/features/dashboard/types.ts` around lines 122 - 131, The
ProcessedCacheChartData interface is re-inlining Record<string, any> for
spec_cache_trend, spec_cache_rank, and spec_non_cache; replace those three types
with the existing VChartSpec alias and remove the corresponding
eslint-disable-next-line comments so the interface reads spec_cache_trend:
VChartSpec, spec_cache_rank: VChartSpec, spec_non_cache: VChartSpec within the
ProcessedCacheChartData declaration.
web/default/src/features/dashboard/components/models/log-stat-cards.tsx (1)

125-125: 💤 Low value

Mark the decorative stat-card icons as aria-hidden.

These lucide icons are purely decorative (the metric is conveyed by the adjacent title/value text), so they should be hidden from assistive tech.

♿ Proposed change
-          <Icon className='text-muted-foreground/60 size-3.5 shrink-0' />
+          <Icon aria-hidden='true' className='text-muted-foreground/60 size-3.5 shrink-0' />

(apply to both the core-grid icon at line 125 and the derived-strip icon at line 170)

As per coding guidelines: "add aria-hidden="true" to decorative icons; provide text equivalents for important information".

Also applies to: 170-170

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@web/default/src/features/dashboard/components/models/log-stat-cards.tsx` at
line 125, The decorative lucide icons in log-stat-cards.tsx (the core-grid Icon
used in the stat card header at the <Icon className='text-muted-foreground/60
size-3.5 shrink-0' /> instance and the derived-strip Icon used later) should be
hidden from assistive tech; update those JSX Icon elements to include
aria-hidden="true" so they are ignored by screen readers while leaving the
visible styling intact.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@model/main.go`:
- Around line 288-289: The MigrateCacheTokens goroutine is started too early in
migrateDB/InitDB and can dereference LOG_DB (used in queryCacheTokensFromLogs)
before InitLogDB sets it; either move the call to go MigrateCacheTokens() so it
runs only after InitLogDB completes (i.e., start backfill after both DBs are
initialized), or add a nil-check guard inside
MigrateCacheTokens/queryCacheTokensFromLogs (check LOG_DB != nil and/or
loop-wait until LOG_DB non-nil) and respect common.DataExportEnabled, ensuring
you still only run the backfill when quota_data rows exist.

In `@model/migrate_cache_tokens.go`:
- Around line 20-54: The loop uses offset pagination over DB.Table("quota_data")
with a mutating filter which causes skipped rows and repeated full scans; change
to keyset pagination by replacing Offset/offset logic with an id-based
high-water mark: query records with Where("cache_tokens = 0 ... AND id > ?",
lastID).Order("id ASC").Limit(batchSize), update lastID = record.Id as you
iterate (or set lastID to the max id of the batch) and remove offset/offset
increment; keep using queryCacheTokensFromLogs and the same Updates call, and
consider persisting the lastID high-water mark (or storing a completion flag) so
restarts resume after last processed id rather than rescanning from the
beginning.

In `@web/default/src/features/dashboard/components/models/cache-token-charts.tsx`:
- Line 136: The Database icon is decorative and should not be announced by
screen readers; update the JSX element that renders the Database component (the
Database icon) to include the aria-hidden="true" attribute so it is ignored by
assistive technology, e.g., add aria-hidden="true" on the Database component
usage in cache-token-charts.tsx.

In `@web/default/src/features/dashboard/lib/stats.ts`:
- Around line 45-49: The aggregation for totalCacheCreation is double-counting
because it always sums cache_creation_tokens + cache_creation_tokens_5m +
cache_creation_tokens_1h; change the logic in the reducer that computes
totalCacheCreation (and the analogous aggregator in charts.ts) to follow backend
normalization: if either cache_creation_tokens_5m or cache_creation_tokens_1h is
non-zero, add only (Number(item.cache_creation_tokens_5m) || 0) +
(Number(item.cache_creation_tokens_1h) || 0); otherwise add
Number(item.cache_creation_tokens) || 0. Ensure you update both the
function/variable that computes totalCacheCreation in stats.ts and the matching
aggregation in charts.ts so the UI matches cacheWriteTokensTotal() semantics.

---

Nitpick comments:
In `@web/default/src/features/dashboard/components/models/log-stat-cards.tsx`:
- Line 125: The decorative lucide icons in log-stat-cards.tsx (the core-grid
Icon used in the stat card header at the <Icon
className='text-muted-foreground/60 size-3.5 shrink-0' /> instance and the
derived-strip Icon used later) should be hidden from assistive tech; update
those JSX Icon elements to include aria-hidden="true" so they are ignored by
screen readers while leaving the visible styling intact.

In `@web/default/src/features/dashboard/types.ts`:
- Around line 122-131: The ProcessedCacheChartData interface is re-inlining
Record<string, any> for spec_cache_trend, spec_cache_rank, and spec_non_cache;
replace those three types with the existing VChartSpec alias and remove the
corresponding eslint-disable-next-line comments so the interface reads
spec_cache_trend: VChartSpec, spec_cache_rank: VChartSpec, spec_non_cache:
VChartSpec within the ProcessedCacheChartData declaration.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3fced897-a8a2-4052-ae2f-81bf7056731e

📥 Commits

Reviewing files that changed from the base of the PR and between 1588027 and bdbd65d.

📒 Files selected for processing (20)
  • model/log.go
  • model/main.go
  • model/migrate_cache_tokens.go
  • model/usedata.go
  • model/usedata_rankings.go
  • web/classic/src/components/dashboard/index.jsx
  • web/classic/src/hooks/dashboard/useDashboardData.js
  • web/classic/src/hooks/dashboard/useDashboardStats.jsx
  • web/default/src/features/dashboard/components/models/cache-token-charts.tsx
  • web/default/src/features/dashboard/components/models/log-stat-cards.tsx
  • web/default/src/features/dashboard/constants.ts
  • web/default/src/features/dashboard/hooks/use-dashboard-config.tsx
  • web/default/src/features/dashboard/index.tsx
  • web/default/src/features/dashboard/lib/charts.ts
  • web/default/src/features/dashboard/lib/filters.ts
  • web/default/src/features/dashboard/lib/index.ts
  • web/default/src/features/dashboard/lib/stats.ts
  • web/default/src/features/dashboard/types.ts
  • web/default/src/i18n/locales/en.json
  • web/default/src/i18n/locales/zh.json

Comment thread model/main.go Resolved
Comment thread model/migrate_cache_tokens.go Resolved
ChenXu233 added 2 commits May 30, 2026 15:37
- replace unsafe v.(float64) type assertions with safe toIntOrDefault helper
- wrap SUM(cache_tokens) with COALESCE(..., 0) to prevent NULL scan issues
- switch backfill from OFFSET to keyset pagination (WHERE id > lastID)
- fix nonCache calculation to exclude both cacheRead and cacheCreation
- optimize non-cache chart aggregation from O(times x models) to O(entries)
- use VChartSpec type for ProcessedCacheChartData instead of Record<string, any>
- add missing i18n keys: Cache Token Analytics, Cache Trend, Cache Ranking, Non-cache Tokens
- move MigrateCacheTokens call after InitLogDB to avoid LOG_DB nil panic
- fix cache_creation_tokens double-counting in stats.ts and charts.ts
- add aria-hidden="true" to decorative Database icon

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@main.go`:
- Around line 304-305: The backfill goroutine is started unconditionally which
can cause every node to run MigrateCacheTokens and a panic there would crash the
process; wrap the call site in main.go with a guard that checks
common.IsMasterNode() (and optional common.DataExportEnabled) before launching
the goroutine, and start it as a protected goroutine that recovers from panics
and logs them instead of letting them propagate; also add a defensive
defer/recover inside model.MigrateCacheTokens (or its top-level entry) to ensure
any internal panic is caught and logged so it cannot bring down the whole
process.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 72539e78-2f84-4278-9f59-0ff1f200d360

📥 Commits

Reviewing files that changed from the base of the PR and between e11761b and 8373215.

📒 Files selected for processing (4)
  • main.go
  • web/default/src/features/dashboard/components/models/cache-token-charts.tsx
  • web/default/src/features/dashboard/lib/charts.ts
  • web/default/src/features/dashboard/lib/stats.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • web/default/src/features/dashboard/components/models/cache-token-charts.tsx
  • web/default/src/features/dashboard/lib/charts.ts

Comment thread main.go Resolved
- wrap backfill goroutine with common.IsMasterNode check
- use gopool.Go instead of raw go for built-in panic recovery
@Calcium-Ion
Calcium-Ion force-pushed the main branch 2 times, most recently from 51fdfc5 to 2b6f1df Compare August 30, 2026 15:03

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: data panel tokens 统计增加 cache_tokens 和 cache_creation_tokens

2 participants