Repository navigation
Conversation
Contributor
|
All contributors have signed the CLA. ✅ |
… on the Codex path Chat Completions clients that reach Codex through /v1/chat/completions lose the Responses `reasoning` items between tool turns: the protocol has no field for them, so an assistant tool_calls message comes back without the reasoning that produced it. Under store=false the upstream expects those items (with encrypted_content) to be replayed in front of the function_call (the contract Wei-Shaw#3588 preserves for native Codex clients); without them the model restarts each tool round from scratch and multi-step agent quality drops even though every request returns 200. The bridge also echoed the plaintext reasoning summary back as a <thinking> block inside assistant output_text, so the model saw its own summary as spoken output. Close the gap on the gateway side, mirroring the existing Responses→Chat reasoning cache (Wei-Shaw#5729) and the Anthropic thinking.signature round-trip: - Response direction: while the Responses stream is translated to Chat chunks, completed reasoning items are bound to the next completed function_call / custom_tool_call and stored in the shared reasoning cache under the tool call_id, tagged with the account that produced them. Later calls of the same parallel batch get a marker pointing at the batch owner so lookups can tell "covered by a sibling" from a real miss. The buffered (stream=false) path is covered too. - Request direction: ChatCompletionsToResponsesWithOptions asks for cached items by tool_calls[].id and inserts them as `reasoning` input items (with encrypted_content and an explicit empty summary) right before the matching function_call, dropping the <thinking> echo when real reasoning is restored. Items cached by another account are never injected. - If the upstream still rejects the ciphertext (invalid_encrypted_content), the request is retried once without replay. Only the Codex OAuth protocol is affected; a cache miss leaves the request byte-for-byte as before. Per-request summaries log at debug; warn on cache failures and on the rejected-ciphertext retry. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QCvmU3omdT9VvUGq5S6r7M
wolfhead
force-pushed
the
feat/cc-codex-reasoning-replay
branch
from
September 7, 2026 01:46
918ca87 to
820e873
Compare
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Chat Completions clients that reach Codex through
/v1/chat/completionslose the Responsesreasoningitems between tool turns. Chat Completions has no field for them, so when such a client replaysassistant.tool_callsthe gateway forwards thefunction_callitems without the reasoning that produced them. Understore=falsethe upstream expects those items (withencrypted_content) to be replayed in front of thefunction_call(the same contract #3588 preserves for native Codex clients); without them the model restarts every tool round from scratch. Every request still returns 200, so the only symptoms are degraded multi-step agent quality and a lower prompt-cache hit rate.The bridge also echoed the plaintext
reasoning_contentback to the upstream as a<thinking>block inside assistantoutput_text, so the model saw its own reasoning summary as spoken output.This PR closes the gap on the gateway side, mirroring the existing Responses→Chat reasoning cache (#5729) and the Anthropic
thinking.signatureround-trip:chatReasoningReplayRecorder): while the Responses stream is translated to Chat chunks, completedreasoningitems are bound to the next completedfunction_call/custom_tool_calland stored in the shared reasoning cache under the toolcall_id, tagged with the account that produced them. Later calls of the same parallel batch get a marker record pointing at the batch owner. The buffered (stream=false) path is covered too.apicompat.ChatCompletionsToResponsesWithOptions): on the Codex OAuth path the converter looks uptool_calls[].idand inserts the cached items asreasoninginput items (withencrypted_contentand an explicit emptysummary) right before the matchingfunction_call. When real reasoning is restored the<thinking>echo is dropped. Records written by another account are never injected (ciphertext is account-bound).invalid_encrypted_content), the request is retried once without replay.A cache miss leaves the request byte-for-byte as before. Only
account.UsesOpenAICodexProtocol()accounts are affected; the other six callers ofChatCompletionsToResponseskeep the nil-options path. No config, schema or API changes; the cache reusesGatewayCache.Set/GetReasoningContentwith acc_tool_call:key prefix and the existing 7-day TTL.Production data
Deployed on our gateway (upstream v0.1.183 + this patch) with a Chat Completions agent client (WorkBuddy) driving Codex
gpt-5.6on a chatgpt.com OAuth account:invalid_encrypted_contentretries / account-mismatch skipsEach encrypted reasoning item is ~2.2 KB (max 3.8 KB seen); Redis growth was negligible.
Logging
Per-request summaries are
Debug(reasoning replay recorded/reasoning replay lookupwith hit/miss/covered/injected counts).Infoonly when a cached record from another account is skipped,Warnon cache read/write failures and on the rejected-ciphertext retry.Tests
apicompat: hit inserts items before the call and drops<thinking>; partial hit keeps assistant text; miss/blank keeps the legacy shape; blank ids skipped.service: recorder binding (stream + buffered, parallel batches with sibling markers), account-scoped lookup, round trip, enable gating, error matcher.serviceend-to-end throughForwardAsChatCompletionswith a fake Codex stream: two-turn replay, miss, rejected-cipher retry.go test ./internal/pkg/apicompat/,go test -tags unit ./internal/service/,go vet, golangci-lint v2.13 (CI config): clean.Closes #6632