[unbug]

Overview

Fix incorrect usage reporting for OpenAI requests with n > 1.

Fixes #29451.

When multiple choices are requested, the server only keeps the usage from the first result. This causes completion_tokens and total_tokens to be undercounted.

Sum completion_tokens across all choices and recalculate total_tokens, while counting prompt_tokens once.

Additional information

The issue can be reproduced with the script provided in #29451. With n=4, for example, four completions are returned but completion_tokens only reports the tokens from one choice.

Requirements

• I have read and agree with the [contributing guidelines](https://github.com/ggml-org/llama.cpp/blob/master/CONTRIBUTING.md)

• AI usage disclosure: YES — AI was used to help review the code and run the test.

[ggml-gh-bot]

Hi @unbug, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[unbug]

Hi @unbug, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

DONE.

[ngxson]

/bot review

[ggml-gh-bot]

Automated code review

Review of PR #29486 (server: fix usage.completion_tokens undercount with n > 1). Checklists run: scope/quick-reject, security, general, server, tests (changes under tools/server/).

Scope gate: clean. Single-purpose fix, linked issue #29451, minimal diff, AI disclosure filled in. No security concerns: the summed values are server-computed token counts, not client-controlled input, and each element of arr is a server_task_result_cmpl_final serialized through usage_json_oaicompat(), so the ["usage"]["completion_tokens"] lookups cannot hit a missing key or wrong type.

The non-streaming fix itself is correct. I verified that json & usage = arr[0]["usage"] stays valid through the choices-merge loop (only choices is moved out, arr is never resized), and that summing over all elements including arr[0], counting prompt_tokens once, and recomputing total_tokens matches OAI semantics. The test additions are in an existing test function, use the existing fixture, and the hard-coded 16 is safe because the surrounding loop already asserts finish_reason == "length" (so each choice produced exactly max_tokens).

Findings:

1. (Follow-up, incomplete coverage) The undercount persists in streaming mode: with stream: true, n > 1, and stream_options.include_usage: true, each choice is a separate task that emits its own final usage chunk (to_json_oaicompat_chat_stream, tools/server/server-task.cpp around line 502), each reporting only that choice's completion_tokens. A client reading the last usage chunk (the usual convention) still sees a single choice's count, and it gets n separate empty-choices usage chunks instead of one aggregated one per the OAI spec. Please either handle the streaming aggregation here or note it as a known remaining gap in the PR description (e.g. a comment in the issue) so it is tracked.

2. (Nit) tools/server/server-context.cpp:4382 — the merged usage keeps prompt_tokens_details.cached_tokens from the first result only; children may run on different slots with different cache hits. Acceptable for now (OAI has no per-choice cache accounting), but worth being aware of; no change required.

3. (Nit) The sum loop can be const auto & item — the loop only reads. Very minor.

Style is fine: comments are concise and explain the non-obvious intent, ASCII only, alignment matches surrounding code. No blocking issues on the code as written; point 1 is the only substantive item and can be addressed either in this PR or as a follow-up.

This review was generated automatically by pi coding agent using zai-org/GLM-5.3. It may contain mistakes. Maintainers make the final call.