Skip to content

feat(budget): surface cached/cache-write/reasoning token counts on last_call (§3) - #377

Merged
tommasobolis merged 2 commits into
developfrom
feat/307-usage-tokens
Sep 15, 2026
Merged

tommasobolis merged 2 commits into
developfrom
feat/307-usage-tokens

Conversation

@tommasobolis

Copy link
Copy Markdown
Collaborator

Closes #307.

What

Surfaces the per-call usage token detail counts from the live LLM-proxy
response body on donkey.last_call and as OTel span attributes:

  • cached_tokens, cache_write_tokens (from usage.input_tokens_details)
  • reasoning_tokens (from usage.output_tokens_details)
  • plus the totals input_tokens / output_tokens / total_tokens

Why this is a consumption task, not a verification one

The usage shape is already live-verified (docs/verified-apis.md §2,
responses.success.body.json). This PR reads an already-confirmed shape, so
there is no UnverifiedValueWarning on the read path (asserted by test).

Design

  • One shared parse_usage / usage_from_response / usage_mapping in
    core/lastcall.py reads both the Responses-API (input_tokens*) and
    Chat-Completions (prompt_tokens* / completion_tokens*) wire shapes.
  • LastCall gains six int | None usage fields; from_response fills them.
    An absent detail field (or an absent usage object entirely) reports None,
    never a fabricated 0 — a real 0 is a distinct, kept observation.
  • observe_usage() merges the streamed terminal-event counts into the
    OBSERVED record on stream finalize (the body is unread at _on_response).
  • build_genai_attributes emits the three detail counts under the stable
    donkey.usage.* namespace
    — the semconv pins no key for them at the pinned
    version — and they are allowlisted so span() cannot silently drop them.
    gen_ai.usage.* is unchanged.
  • Carried identically on async + sync transports and the streaming SSE path
    (attach-to-the-hook, per BG §1.6; no send() override).

Tests

  • Fixture-driven parse test feeds responses.success.body.json verbatim
    (its detail counts are all 0, which exercises the 0-vs-None contract).
  • Async + sync transport last_call fill; end-to-end streaming test
    asserting the span donkey.usage.* attrs and last_call fill from a terminal
    usage event; identity-only response leaves usage None.
  • No-UnverifiedValueWarning assertion on the read path.

Docs (cross-surface lockstep)

  • docs/verified-apis.md §2 (detail sub-fields named on the token-accounting
    row) and §3 (a paragraph recording the budget: surface cached and reasoning token counts from the live usage payload #307 surfacing).
  • website/: observability (span-attribute table + streaming note), telemetry
    (span example + prose), attribution (usage-block field list).
  • TypeScript parity is deferred to Phase 5 (the TS SDK is not yet
    shipped) — a recorded, intentional deferral, not a silent omission.

Gates

pytest -q (372 passed, 9 skipped), mypy --strict clean, ruff check .
clean, lint-imports 4/4 KEPT.

🤖 Generated with Claude Code

tommasobolis and others added 2 commits September 15, 2026 00:07
…st_call (§3)

Parse the per-call usage token detail counts from the live proxy response
body — `usage.input_tokens_details.{cached_tokens,cache_write_tokens}` and
`usage.output_tokens_details.reasoning_tokens` — and surface them (with the
totals) on `donkey.last_call` and as OTel span attributes. Closes #307.

The usage shape is already live-verified (§2), so this is a consumption task,
not a verification one: no UnverifiedValueWarning on the read path. Both the
Responses-API (`input_tokens*`) and Chat-Completions (`prompt_tokens*`) wire
shapes are read via one shared `parse_usage`/`usage_from_response` in
`core/lastcall.py`.

- `LastCall` gains six `int | None` usage fields; `from_response` fills them
  from the body. Absent detail fields (or an absent `usage` object) report
  `None`, never a fabricated `0` — a real `0` is a distinct, kept observation.
- `observe_usage()` merges the streamed terminal-event counts into the OBSERVED
  record on stream finalize (the body is unread at `_on_response` time).
- `build_genai_attributes` emits the three detail counts under the stable
  `donkey.usage.*` namespace (the semconv pins no key for them at the pinned
  version); allowlisted so `span()` cannot drop them. `gen_ai.usage.*` unchanged.
- Carried identically on async + sync transports and the streaming SSE path.

Tests: fixture-driven parse test uses `responses.success.body.json` verbatim
(detail counts are all `0` there — exercises the 0-vs-None contract directly);
async + sync last_call fill; end-to-end streaming span + last_call fill from a
terminal usage event; no-UnverifiedValueWarning assertion.

Docs (cross-surface lockstep): docs/verified-apis.md §2/§3, website
observability / telemetry / attribution pages. TypeScript parity is deferred to
Phase 5 (the TS SDK is not yet shipped) — a recorded, intentional deferral.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@tommasobolis
tommasobolis merged commit 7155dc2 into develop Sep 15, 2026
7 checks passed
@tommasobolis
tommasobolis deleted the feat/307-usage-tokens branch September 15, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant