otel: stop exporting fabricated token counts; add cost_unavailable

[brentrager]

Follow-up to #471 (already merged). Three items from the lead, all one theme: don't export numbers we can't stand behind.

Task 3 — input_tokens = 0 on every prod turn. The cause is not this crate.

Core's collect_stream fabricates a usage struct whenever no StreamEvent::Usage arrives — it hardcodes prompt_tokens = 0 and estimates completion_tokens as content.len() / 4:

// smooai-smooth-operator-core 1.8.5, src/llm.rs:2337-2350
if usage.prompt_tokens == 0 && usage.completion_tokens == 0 {
    // Streaming path doesn't have the outgoing request in
    // scope; estimate prompt_tokens at zero and only
    // capture completion (~4 chars/token rule of thumb).
    let completion_chars = content.len() + total_args_chars;
    usage.completion_tokens = u32::try_from(completion_chars / 4).unwrap_or(u32::MAX);

LiteLLM at llm.smoo.ai drops the usage chunk for smooth-* aliases (core's own pearl th-eff0d0 says so), so this is the common path, not an edge case.

This is worse than the ticket describes. The "plausible" output counts in prod — 62, 74, 81, 76 — were never measurements either. They are character-count estimates. Only the input half looked obviously wrong.

The old gate was prompt_tokens > 0 || completion_tokens > 0, which is exactly what let that pair through: a fabricated struct has completion > 0, so the || published input = 0 beside it. Now gated on prompt_tokens > 0 alone — both counts or neither.

A test reproduces the prod signature and fails on the old gate:

input_tokens: "0", output_tokens: "13", cost_usd: "0.00013"

That third value is a cost derived from the fabricated usage — a guess multiplied by a guess.

The real fix is in core (stop fabricating, or mark the struct estimated). Until then this instrumentation declines to export it.

smooai.gen_ai.cost_unavailable

Set to "unpriced" instead of the cost when none could be established — the exact attribute name and value the TS lane (#4245) ships, so a consumer never special-cases per engine.

I checked the ingest before adopting it: there is no column for it, and no cost-provenance column either. It lands in the attributes JSONB blob, readable as attributes->>'smooai.gen_ai.cost_unavailable' — unindexed and string-typed. Worth knowing before a dashboard is built on it.

Core collapses TS's "unparseable" case into the same None as unpriced, so this layer genuinely cannot emit that value. Documented rather than guessed at.

Cost is judged independently of the token counts

I first suppressed cost whenever usage was fabricated, reasoning that a gateway sending no usage chunk sent no cost header either. An existing test proved that wrong: cost arrives in an HTTP header, usage in an SSE chunk — two separate channels, so a turn can legitimately have an authoritative cost and no usage. Suppressing it there would have recreated the all-zero-rows bug this whole set exists to fix. Backed out.

The residual hazard is the other direction: a locally-priced model with fabricated usage yields ModelPricing × 0 input, an undercount that still looks authoritative. It's pinned by an assertion explicitly labelled as today's gap, not desired behaviour — if it ever starts failing, provenance landed. Production is not exposed: its model isn't in the local pricing table, so that path yields exactly 0 and is dropped.

telemetry::record_turn_usage now owns this whole policy in one place, so the streaming runner and KnowledgeChatRuntime can't drift apart on it.

gen_ai.response.id — cannot be done here

The Rust operator does not set it, and cannot without a core change. Core never captures the chatcmpl-… id at all: neither ChatResponse (llm.rs:558) nor StreamChunk (llm.rs:650) deserializes an id field, and LlmResponse has nowhere to put one. This belongs in the same core PR as cost provenance — see the report back to the lead.

Correcting my own claim in #471

#471's description said the cost header "survives streaming". That was accurate about the code path (core reads resp.headers() before consuming the SSE body) but I let it read as a claim about the data, which I did not verify and which the TS lane disputes. I have no evidence either way on whether LiteLLM populates the header on a streamed response, and this PR asserts nothing about it. Both implementations fail safe, so neither emits a wrong number.

Verification

Reverting the token gate to || and re-running fails with the prod signature quoted above; restoring passes.

gate	result
cargo test -p smooai-smooth-operator-server -p smooai-smooth-operator	564 passed, 0 failed, 1 ignored, 45 suites (563 before, +1 new test)
cargo clippy --all-targets -- -D warnings	clean (exit 0)
cargo fmt --all --check	clean (exit 0)

Rebased onto current main (post-#472). docs/Operations/Observability.md updated in the same commit; no overlap with the files tsworker-cost touched.

No changeset — the lead is driving the release.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LkCz96UfUxcai5RU4LwPnG

[changeset-bot]

⚠️ No Changeset found

Latest commit: 88e69e4

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR