Skip to content

feat(openai): replay encrypted reasoning for Chat Completions clients on the Codex path - #6633

Open
wolfhead wants to merge 1 commit into
Wei-Shaw:mainfrom
wolfhead:feat/cc-codex-reasoning-replay
Open

wolfhead wants to merge 1 commit into
Wei-Shaw:mainfrom
wolfhead:feat/cc-codex-reasoning-replay

Conversation

@wolfhead

@wolfhead wolfhead commented Sep 5, 2026

Copy link
Copy Markdown

Summary

Chat Completions clients that reach Codex through /v1/chat/completions lose the Responses reasoning items between tool turns. Chat Completions has no field for them, so when such a client replays assistant.tool_calls the gateway forwards the function_call items without the reasoning that produced them. Under store=false the upstream expects those items (with encrypted_content) to be replayed in front of the function_call (the same contract #3588 preserves for native Codex clients); without them the model restarts every tool round from scratch. Every request still returns 200, so the only symptoms are degraded multi-step agent quality and a lower prompt-cache hit rate.

The bridge also echoed the plaintext reasoning_content back to the upstream as a <thinking> block inside assistant output_text, so the model saw its own reasoning summary as spoken output.

This PR closes the gap on the gateway side, mirroring the existing Responses→Chat reasoning cache (#5729) and the Anthropic thinking.signature round-trip:

  • Response direction (chatReasoningReplayRecorder): while the Responses stream is translated to Chat chunks, completed reasoning items are bound to the next completed function_call / custom_tool_call and stored in the shared reasoning cache under the tool call_id, tagged with the account that produced them. Later calls of the same parallel batch get a marker record pointing at the batch owner. The buffered (stream=false) path is covered too.
  • Request direction (apicompat.ChatCompletionsToResponsesWithOptions): on the Codex OAuth path the converter looks up tool_calls[].id and inserts the cached items as reasoning input items (with encrypted_content and an explicit empty summary) right before the matching function_call. When real reasoning is restored the <thinking> echo is dropped. Records written by another account are never injected (ciphertext is account-bound).
  • Recovery: if the upstream still rejects the ciphertext (invalid_encrypted_content), the request is retried once without replay.

A cache miss leaves the request byte-for-byte as before. Only account.UsesOpenAICodexProtocol() accounts are affected; the other six callers of ChatCompletionsToResponses keep the nil-options path. No config, schema or API changes; the cache reuses GatewayCache.Set/GetReasoningContent with a cc_tool_call: key prefix and the existing 7-day TTL.

Production data

Deployed on our gateway (upstream v0.1.183 + this patch) with a Chat Completions agent client (WorkBuddy) driving Codex gpt-5.6 on a chatgpt.com OAuth account:

before after
tool-continuation requests: fresh input tokens ~10k avg 250–1300
tool-continuation requests: cache hit 84.6% 95.6–99.2%
avg latency (tool-heavy task) 23.2 s 7.0 s
avg TTFT 3.7 s 1.0 s
invalid_encrypted_content retries / account-mismatch skips – 0 in ~100 requests, context up to 125k tokens

Each encrypted reasoning item is ~2.2 KB (max 3.8 KB seen); Redis growth was negligible.

Logging

Per-request summaries are Debug (reasoning replay recorded / reasoning replay lookup with hit/miss/covered/injected counts). Info only when a cached record from another account is skipped, Warn on cache read/write failures and on the rejected-ciphertext retry.

Tests

  • apicompat: hit inserts items before the call and drops <thinking>; partial hit keeps assistant text; miss/blank keeps the legacy shape; blank ids skipped.
  • service: recorder binding (stream + buffered, parallel batches with sibling markers), account-scoped lookup, round trip, enable gating, error matcher.
  • service end-to-end through ForwardAsChatCompletions with a fake Codex stream: two-turn replay, miss, rejected-cipher retry.
  • go test ./internal/pkg/apicompat/, go test -tags unit ./internal/service/, go vet, golangci-lint v2.13 (CI config): clean.

Closes #6632

@github-actions

github-actions Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA. ✅
Posted by the CLA Assistant Lite bot.

… on the Codex path

Chat Completions clients that reach Codex through /v1/chat/completions lose
the Responses `reasoning` items between tool turns: the protocol has no
field for them, so an assistant tool_calls message comes back without the
reasoning that produced it. Under store=false the upstream expects those
items (with encrypted_content) to be replayed in front of the function_call
(the contract Wei-Shaw#3588 preserves for native Codex clients); without them the
model restarts each tool round from scratch and multi-step agent quality
drops even though every request returns 200. The bridge also echoed the
plaintext reasoning summary back as a <thinking> block inside assistant
output_text, so the model saw its own summary as spoken output.

Close the gap on the gateway side, mirroring the existing Responses→Chat
reasoning cache (Wei-Shaw#5729) and the Anthropic thinking.signature round-trip:

- Response direction: while the Responses stream is translated to Chat
  chunks, completed reasoning items are bound to the next completed
  function_call / custom_tool_call and stored in the shared reasoning cache
  under the tool call_id, tagged with the account that produced them. Later
  calls of the same parallel batch get a marker pointing at the batch owner
  so lookups can tell "covered by a sibling" from a real miss. The buffered
  (stream=false) path is covered too.
- Request direction: ChatCompletionsToResponsesWithOptions asks for cached
  items by tool_calls[].id and inserts them as `reasoning` input items (with
  encrypted_content and an explicit empty summary) right before the matching
  function_call, dropping the <thinking> echo when real reasoning is
  restored. Items cached by another account are never injected.
- If the upstream still rejects the ciphertext (invalid_encrypted_content),
  the request is retried once without replay.

Only the Codex OAuth protocol is affected; a cache miss leaves the request
byte-for-byte as before. Per-request summaries log at debug; warn on cache
failures and on the rejected-ciphertext retry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCvmU3omdT9VvUGq5S6r7M
@wolfhead
wolfhead force-pushed the feat/cc-codex-reasoning-replay branch from 918ca87 to 820e873 Compare September 7, 2026 01:46
@wolfhead

wolfhead commented Sep 7, 2026

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Chat Completions clients lose reasoning items between tool turns on the Codex OAuth path (quality regression, no error)

1 participant