Skip to content

feat(apicompat): carry Anthropic thinking blocks in Chat reasoning_details - #8809

Open
Anankke wants to merge 5 commits into
Wei-Shaw:mainfrom
Anankke:feat/chat-bridge-thinking-replay
Open

Anankke wants to merge 5 commits into
Wei-Shaw:mainfrom
Anankke:feat/chat-bridge-thinking-replay

Conversation

@Anankke

@Anankke Anankke commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

On /v1/chat/completions served by an Anthropic account, signed thinking and redacted_thinking blocks never reach the next tool round, Chat has no field for them.

Each block now leaves as one reasoning.encrypted entry in reasoning_details, format anthropic-claude-v1, streamed as one chunk when the block ends. data is the base64 anthropic-thinking-v1 envelope of /v1/responses, so the tool name rewrite on response bytes cannot touch signed text. Entries sent back go upstream as the original blocks ahead of text and tool_use, in place of the <thinking> copy of reasoning_content that main puts in the text. Other entries are ignored. Same model gate as /v1/responses, read on the mapped upstream model.

Other Chat paths that share the conversion emit no entries: native Anthropic, Antigravity, Gemini compat.

Conflicts with #8750 and #6633. Neither carries Anthropic signatures or redacted_thinking.

Against Anthropic at this head, two tool rounds with the entries echoed back return 200, streamed and not. So does a second round on another model or another account, and an assistant turn that holds only thinking.

…tails

The Chat request has no field for the signed thinking blocks that /v1/responses already round-trips as anthropic-thinking-v1 envelopes, so a tool round reached the upstream without them.
Chat responses built from Anthropic output now carry each block as an OpenRouter style reasoning_details entry, and the bridge rebuilds the blocks from the assistant message's entries, in index order, ahead of its text and tool calls.
Replay and the suppression of the <thinking> text copy share the model gate of ResponsesToAnthropicRequest, so other models keep the old conversion. Responses take the same gate from the mapped model on both the streaming and buffered paths.
Requests without reasoning_details convert as before.
Signed thinking blocks now travel as the anthropic-thinking-v1 envelope in the data of a reasoning.encrypted entry, so tool name reversal on the response bytes cannot change text the signature covers. The plaintext stays in reasoning_content only.
reasoning_details stays raw JSON on Chat messages and deltas and is decoded entry by entry on replay, so a malformed value is ignored as before instead of failing the request.
Responses carry the entries only for the models whose requests replay them, through ResponsesToChatOptions and a stream state flag, so the other Chat paths keep their previous output.
@r266-tech

Copy link
Copy Markdown
Contributor

AI-assisted local review of b148b90bd386565ad2f255fcb70463fafcd6d2f1: the new reasoning_details pass through the response-side tool-name byte rewrite, which can change a thinking block before it is replayed.

With an Anthropic OAuth account declaring sessions_list, the real ForwardAsChatCompletions request declares cc_sess_list upstream. Using the new thinkingToolTurnSSE() fixture with thinking text I will call cc_sess_list., I get this in both streamed and buffered two-round requests:

upstream thinking:                       "I will call cc_sess_list."
Chat reasoning_details[0].text:           "I will call sessions_list."
next upstream messages[1] thinking text: "I will call sessions_list."

The signature is unchanged. reverseToolNamesIfPresent rewrites the entire serialized Chat response/chunk, including the new detail's text; replay then encodes that changed text. Anthropic's thinking-block preservation contract calls for complete, unmodified blocks during tool use.

Could the response rewrite protect reasoning_details, or target tool-call name fields, with a two-round regression? A local Go overlay protecting only the details makes the same requests preserve the original block while still restoring the actual tool call to sessions_list.

Validation on Go 1.27.4/macOS arm64: 2 failing cases, 6 passing controls (ordinary thinking text, omitted text, and API Key model mapping); the existing Responses-envelope path preserves the same text in both modes. The full apicompat package and this PR's 8 focused service tests pass. With the protective overlay all 8 Chat cases and the existing focused tests pass. Synthetic HTTP/SSE only: this demonstrates changed replay content, not a live signature-verification failure.

… tools

Runs ForwardAsChatCompletions twice on an OAuth account whose tool names are rewritten, streaming and buffered, with a stub upstream that signs thinking text containing the rewritten tool name.
The response side maps tool names back for the client, so the second request must still carry the thinking text and signature the stub sent, which the test asserts on the body sent upstream.
The stream layout follows a recorded real Anthropic stream with placeholder values.
@Anankke

Anankke commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

99f6b0f carries signed thinking as the base64 anthropic-thinking-v1 envelope in data, so the rewrite has no text to change there. eb563d4 adds the two-round test, TestForwardAsChatCompletions_TwoRoundsReplaySignedThinkingWithToolNameRewrite, streamed and buffered.

@r266-tech

Copy link
Copy Markdown
Contributor

Rechecked eb563d4e9581b1cc6697c16a2c509ff425249ea8. The new TestForwardAsChatCompletions_TwoRoundsReplaySignedThinkingWithToolNameRewrite passes in both streamed and buffered modes.

My original eight two-round scenarios also pass after adapting the probe to decode the new envelope. In the cc_sess_list case, the second upstream request now preserves I will call cc_sess_list. and its signature verbatim, while the client receives the actual tool call as sessions_list. The redacted block, empty-thinking control, and API-key mapped-model control retain their original values too. This resolves the finding in my earlier comment.

The full internal/pkg/apicompat package, focused service reasoning-details/thinking tests, and both existing Responses-envelope controls pass on Go 1.27.4/macOS arm64. AI-assisted local checks with synthetic HTTP/SSE; no live Anthropic or full-backend validation.

xiaoguai1818 pushed a commit to xiaoguai1818/sub2api that referenced this pull request Nov 21, 2026
Restore Antigravity Responses custom-tool roundtrips and correct Embeddings platform scheduling. Harden OAuth probe fencing, request-scoped Gemini 403 behavior, OpenAI stream failover and WebSocket current-turn safety.

Preserve Claude/GPT-6.2/Grok protocol details and model observations; correct Fable/Kimi quota refreshes and user-vs-account cost reporting. Bound scheduler fallback, request memory retention and shutdown/stream lifecycles without unapproved pricing or schema changes.

Local checks: complete Go unit packages, PostgreSQL 18.1/Redis 8.4 repository and audit integration, focused -race, frontend typecheck/lint/1858 Vitest/build, backend lint 0 issues and embedded server build. Remote CI/Security Scan remain pending.

Upstream references: Wei-Shaw/sub2api Wei-Shaw#8833 Wei-Shaw#8875 Wei-Shaw#8856 Wei-Shaw#8730 Wei-Shaw#8990 Wei-Shaw#8806 Wei-Shaw#8832 Wei-Shaw#8809 Wei-Shaw#8164 Wei-Shaw#8804 Wei-Shaw#8773 Wei-Shaw#8309.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants