Skip to content

fix(anthropic): read the cumulative usage of message_delta - #8663

Merged
longcw merged 1 commit into
livekit:mainfrom
Caezarr:fix/anthropic-cumulative-usage
Nov 19, 2026
Merged

longcw merged 1 commit into
livekit:mainfrom
Caezarr:fix/anthropic-cumulative-usage

Conversation

@Caezarr

@Caezarr Caezarr commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

The usage on Anthropic's message_delta is a cumulative total, but the LLM added its output_tokens to the message_start count and ignored the input and cache counts it carries. completion_tokens was too high on every request, and with a server tool prompt_tokens missed the input the tool added.

Fixes #8662

  • on message_delta, take output_tokens as is and replace input_tokens, cache_creation_input_tokens and cache_read_input_tokens when present, as the SDK's stream accumulator does
  • add test_usage_takes_the_cumulative_message_delta_counts (plain text, server tool, cache read cases; all three fail without the change)

Testing:

  • pytest tests/test_plugin_anthropic_request.py tests/test_plugin_anthropic.py: 17 passed
  • ruff check, ruff format --check, mypy -p livekit.plugins.anthropic: clean
  • the reproduction in the issue now prints 25 / 15 and 9200 / 310. Not run against the live Anthropic API.

@Caezarr
Caezarr requested a review from a team as a code owner November 18, 2026 23:55

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@longcw
longcw merged commit 3ed9a35 into livekit:main Nov 19, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic LLM: message_delta usage is added instead of taken as the cumulative total

2 participants