[gdevenyi]

Problem and result

On /v1/messages, Strata dropped tool_use_id and rendered tool_result blocks in the order they arrived. The templates pair results with calls by position. So when a client sent results in a different order from the calls, each result landed on the wrong call, with no error. When both calls use the same tool, the model cannot tell them apart.

A listed client hits this. Claude Code 2.1.316 sends results in the order its tools finish. With two parallel Grep calls against a local mock of /v1/messages:

• Slow call first: wire order [toolu_B_fast, toolu_A_slow], call order [toolu_A_slow, toolu_B_fast], in 3 of 3 runs.
• Fast call first: wire order matches call order, in 3 of 3 runs.

The Anthropic docs allow this order and say how to pair results (parallel tool use):

The API doesn't prescribe an execution order: you can run the calls concurrently (Promise.all, asyncio.gather), sequentially in the order they appear, or in any combination that suits your tools.

Match each result to its call with tool_use_id, and put every tool_result block before any text content in that message.

Before and after, on the captured Claude Code request. The calls are slow Grep, then fast Grep:

• Before: the prompt has 1:FASTFILE (the fast result) first, then No files found.
• After: No files found first, then 1:FASTFILE, which is call order.

Related discussion

• [Other] preserve Chat Completions tool result order #3395 and fix(serve): order chat tool results by tool_call_id #2952: the same problem on Chat Completions. Related, not duplicates. Commit 11237f8 is the Chat fix.
• No scope thread. This is a bug fix and changes no default.

Approach

Chat already has a reorder pass, _order_chat_tool_results. This change gives it the ids on the Anthropic path and calls it there. The helper itself is not changed.

• It only reorders a complete, unambiguous set of ids. A missing, unknown, duplicate or non-string id keeps the received order, and so does an incomplete set.
• The ids are removed before the template renders, so a request already in call order renders as before.
• /v1/messages/count_tokens uses the same converter. On the captured request it returned 51508 tokens on both the base and the branch, because only the order moves.

Validation

Revision: b669ca1 on base c62d5e3. Linux, Python 3.12.3. MockEngine, except the real-engine check, which comes first below.

Passed:

• Real engine: RTX 5080 16 GB, engine 0.1.55 from release v0.2.9, Qwen3.8 IQ3_S, context 65536, anthropic SDK 2.3.0, non-stream, thinking off, temperature 0. A two-call get_weather round trip, Paris then Tokyo, with invented temperatures that change every run and a different sky per city. An answer counts as swapped when each city has the other's temperature.

  Results sent	Base c62d5e3	Branch b669ca1
  Reversed, 5 runs	5 swapped, 0 correct	5 correct, 0 swapped
  In order, 3 runs	3 correct	3 correct

  On the branch, the model made both calls itself in all 8 runs. On the base it did so in 7 of 8. In reversed run 0 it made only the Paris call, so that assistant turn was constructed, with both calls. That run still swapped. Base and branch ran in separate server sessions.

• python -m unittest serve.test_anthropic_tool_results: 10 tests, OK. The same file on c62d5e3: 6 of the 10 tests fail. unittest reports failures=7 because the HTTP test has a non-stream and a stream subtest. The tests cover both templates by direct render and non-stream and stream over HTTP.

• Base against branch over HTTP, using a local script that is not in the repo. It compares status, content type, normalized body and the prompt the engine received. 12 requests, each sent non-stream and stream, so 24 variants: /v1/messages with text, system, thinking on and off, tools, ordered results, ordered results with text, incomplete results and an unknown id; /v1/chat/completions with text and ordered results; and one reversed request.

  • 22 of 24 are identical, all HTTP 200.
  • The other 2 are the reversed request. Only the prompt differs. Result A comes before result B: false on the base, true on the branch.

• anthropic SDK 2.3.0 sending reversed results to a MockEngine server, using a local script: on the branch, non-stream and stream both reach the engine in call order. On the base, both stay reversed.

• Replay of the captured Claude Code request, using a local script: HTTP 200 on the default template, with the before and after shown above. The DeepSeek template, rendered directly from the converted messages and not through the server, gives the same before and after.

Failed:

python -m unittest serve.test_server serve.test_responses serve.test_anthropic_base_url serve.test_chat_tool_results serve.test_context_overflow serve.test_images serve.test_anthropic_tool_results serve.test_frontend serve.test_env_to_user

Ran 479 tests: FAILED (failures=1, skipped=1). The one failure is serve.test_server.GpuChoice.test_hip_ordinal: AssertionError: '0,1' != '1,0'. That test alone fails the same way on c62d5e3.

Not run:

• On the real engine: streaming, thinking on, a live Claude Code session, the DeepSeek template and other models, more than two parallel calls, and any other card.
• Windows and macOS.
• CUDA, HIP, SYCL and Metal builds. The change is Python only.
• Clients other than Claude Code. Whether OpenCode or pi send results out of order is unknown.

Limits

• The MockEngine checks show the prompt only, not how a model's answer changes. The real-engine check shows the answer for one model and one short request, over 8 runs on each side.
• The Claude Code capture came from a local mock server, not a live Strata session. The captured request was then replayed through MockEngine.
• Results split across several user messages with nothing between them count as one run and are reordered together, as on Chat.
• A system message between the calls and the results stops the reorder, and the received order stays. The Claude Code capture has no such message there.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W53iURzvDQMjnYemag8uvv

[Tsury]

Field data point, and a rebase in case it saves time.

RTX 5090 Laptop 24 GB, Ubuntu, Strata v0.2.9 / engine 0.1.55 built from source with this PR applied. Live /v1/messages, two parallel calls with results sent in reverse order: paired correctly 0 of 5 before, 5 of 5 after. In-order results stayed 5 of 5.

v0.2.10 still needs it, and the branch now conflicts. The conflicts are small: the tool_result block now goes through _tool_references, and tools is built by _anthropic_tool. I put a resolved copy here, with your authorship kept: rebase-4832-on-v0.2.10. serve.test_anthropic_tool_results passes on it: 10 tests, mock engine, no GPU run. I am not opening a second PR; this is yours to use or ignore.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

[gdevenyi]

Thank you, @Tsury, for the run on the 5090 and for the rebase.

I rebased the branch onto current main (ffa80d7) and force-pushed it (771f714). My resolution came out identical to yours in serve/: the result block keeps _tool_references(...) and adds _call_id, and _order_chat_tool_results(messages) runs before tools = [_anthropic_tool(t) ...].

Passed (mock engine, no GPU): serve.test_anthropic_tool_results and serve.test_server, 369 tests, all but one.

Failed: GpuChoice.test_hip_ordinal. It fails the same way on main without this PR.

Not run: the rebased branch on a real engine.

Written with an AI agent (Claude Code), which also ran the steps above.