Skip to content

fix(apicompat): Responses→Anthropic 不再以 assistant / thinking 结尾 / Never end converted history on assistant or a thinking block - #8832

Open
william-drakemond wants to merge 2 commits into
Wei-Shaw:mainfrom
william-drakemond:fix/responses-anthropic-no-prefill-up
Open

william-drakemond wants to merge 2 commits into
Wei-Shaw:mainfrom
william-drakemond:fix/responses-anthropic-no-prefill-up

Conversation

@william-drakemond

Copy link
Copy Markdown
Contributor

问题 / Problem

/v1/responses(Codex 等客户端)经 ResponsesToAnthropicRequest 转成 Anthropic Messages 后,会因为历史末尾的形态被 Claude 5.5 上游以 400 拒绝,客户端重试发送的是同样的历史,所以一直失败。我们生产上 7 天内出现过 38 次,两种报错:

  1. This model does not support assistant message prefill. The conversation must end with a user message.
    转换后的 messages 以 assistant 结尾。常见于 Codex 在一条 assistant commentary 之后断流重连,或末尾一个没有 function_call_output 的 function_call 被 normalizeAnthropicToolPairing 剥掉后只剩前面的 assistant 文本。OpenAI Responses 协议没有 assistant prefill 语义:请求永远是"继续生成下一轮"。
  2. messages.N: The final block in an assistant message cannot be thinking.
    同一轮里 signed thinking(来自 reasoning item)后面跟着一个未配对的 function_call,normalizeAnthropicToolPairing 删掉 tool_use 后,这条 assistant 消息的末尾块就成了 thinking。

Responses → Anthropic conversion could produce histories Claude 5.5 rejects with a 400, and since the client retries the same history it keeps failing:

  1. The converted messages end on assistant (e.g. Codex reconnects after an assistant commentary message, or a trailing unanswered function_call is stripped by normalizeAnthropicToolPairing, leaving the preceding assistant text). Responses has no prefill semantics, so this is never intended.
  2. An assistant message ends with a thinking block when the tool_use after it was dropped as unanswered.

改动 / Change

  • 新增 stripTrailingThinkingFromAssistantMessages:在工具配对之后,去掉每条 assistant 消息末尾的 thinking / redacted_thinking 块;整条只剩 thinking 的消息直接丢弃,再合并相邻同角色消息。

  • ResponsesToAnthropicRequest:转换结果仍以 assistant 结尾时,追加一条最小的 user 消息 Continue.。放在 ResponsesToAnthropicRequest 而不是只对自适应模型生效,因为模型别名(如带档位后缀的名字)不一定能被识别,而 Responses 协议本身永远不需要 prefill。保留最后那条 assistant 内容(而不是删掉它),模型不会丢失刚输出的上下文。

  • Add stripTrailingThinkingFromAssistantMessages, run after tool pairing: trailing thinking / redacted_thinking blocks are removed from every assistant message; a message that held only thinking is dropped and adjacent same-role messages are re-merged.

  • ResponsesToAnthropicRequest appends a minimal user turn (Continue.) when the converted messages still end on assistant. It is not limited to adaptive models, since aliased model names may not be recognized and Responses never needs prefill. The last assistant content is kept rather than dropped, so the model keeps what it just said.

测试 / Tests

先写测试:新增 responses_to_anthropic_adaptive_invariants_test.go,两个用例在没有修复时都失败,修复后通过。go test ./internal/pkg/apicompat/... ./internal/handler/... ./internal/service/... ./internal/pkg/antigravity/... 通过(Go 1.27.3),apicompat 在 -tags unit 下也通过。已在我们的生产环境运行,用以 assistant 消息结尾的 claude-opus-5-5 请求实测返回 200。

Test-first: both new cases fail without the fix and pass with it. The listed packages pass (Go 1.27.3); apicompat also passes with -tags unit. TestFilterThinkingBlocksSignedHistorySkipsDecode (an allocation count) occasionally fails in a full internal/service run under load and passes alone; it does not touch this code. Running in our production gateway: an assistant-terminated claude-opus-5-5 request now returns 200.

@r266-tech

Copy link
Copy Markdown
Contributor

There is a two-request replay gap in the appended Continue. turn at 322f6b4d.

  1. Input ending in user text + assistant commentary is sent upstream as U(original), A(commentary), U("Continue.").
  2. The upstream returns a signed thinking block + text. The Responses client appends those output items to its original input and adds its next real user message.
  3. On conversion of that replay, the prefix before the signed block is only U(original), A(commentary): the synthetic user turn has disappeared, and the new assistant content merges into the preceding assistant message.

I reproduced the missing turn through ForwardAsResponses with a local HTTP/SSE fixture in both buffered and streaming client modes. Both continuation cases fail the prefix-preservation assertion; all four controls (ordinary user history and explicitly retaining the synthetic turn) pass. A direct converter/response-encoder round trip gives the same result for all five locally selected model names in the test below (5 failures, 10 controls passing). The existing apicompat unit package also passes.

The documented preserved-thinking prefix check includes the messages preceding each signed block. This local prefix change therefore appears to invalidate the replayed block's binding; that consequence is inferred from the documented contract. The signatures here are opaque fixtures: I did not call a live provider or observe an upstream 400.

The inserted user turn needs a representation that survives subsequent input replay. Explicitly retaining it is the passing control below.

Offline reproduction: save as backend/internal/pkg/apicompat/continuation_replay_test.go
package apicompat

import (
	"encoding/json"
	"github.com/stretchr/testify/require"
	"testing"
)

func oss8832Message(role, text string) ResponsesInputItem {
	content, _ := json.Marshal(text)
	return ResponsesInputItem{Type: "message", Role: role, Content: content}
}
func oss8832Convert(t *testing.T, model string, items []ResponsesInputItem) *AnthropicRequest {
	t.Helper()
	input, err := json.Marshal(items)
	require.NoError(t, err)
	req, err := ResponsesToAnthropicRequest(&ResponsesRequest{Model: model, Input: input})
	require.NoError(t, err)
	return req
}
func oss8832Prefix(t *testing.T, messages []AnthropicMessage, signature string) []AnthropicMessage {
	t.Helper()
	for i, m := range messages {
		blocks := parseContentBlocks(m.Content)
		for j, b := range blocks {
			if b.Type == "thinking" && b.Signature == signature {
				prefix := append([]AnthropicMessage(nil), messages[:i]...)
				if j > 0 {
					prefix = append(prefix, anthropicMessageFromBlocks(m.Role, blocks[:j]))
				}
				return prefix
			}
		}
	}
	t.Fatal("signed fixture block not replayed")
	return nil
}

func oss8832CanonicalPrefix(messages []AnthropicMessage) []AnthropicMessage {
	messages = mergeConsecutiveMessages(messages)
	out := make([]AnthropicMessage, 0, len(messages))
	for _, message := range messages {
		out = append(out, anthropicMessageFromBlocks(message.Role, parseContentBlocks(message.Content)))
	}
	return out
}

// This compares the actual conversation prefix sent on two conversions. The
// signature is an opaque fixture; this does not emulate Anthropic cryptography
// or claim to have observed an upstream HTTP 400.
func TestOSS8832ContinuationSurvivesSignedThinkingReplay(t *testing.T) {
	for _, model := range []string{"claude-opus-5-5", "claude-sonnet-5-5", "claude-haiku-5-5", "claude-fable-5-2", "claude-mythos-5-2"} {
		for _, shape := range []string{"normal_user_history", "assistant_continuation", "recorded_continuation_control"} {
			t.Run(model+"/"+shape, func(t *testing.T) {
				items := []ResponsesInputItem{oss8832Message("user", "Original user request")}
				if shape != "normal_user_history" {
					items = append(items, oss8832Message("assistant", "Partial commentary before reconnect"))
				}
				sent := oss8832Convert(t, model, items)
				require.Equal(t, "user", sent.Messages[len(sent.Messages)-1].Role)

				if shape == "recorded_continuation_control" {
					items = append(items, oss8832Message("user", "Continue."))
				}
				response := AnthropicToResponsesResponse(&AnthropicResponse{
					Model: model,
					Content: []AnthropicContentBlock{
						{Type: "thinking", Thinking: "opaque fixture summary", Signature: "fixture-bound-to-first-sent-prefix"},
						{Type: "text", Text: "Completed assistant reply"},
					},
				})
				encoded, err := json.Marshal(response.Output)
				require.NoError(t, err)
				var output []ResponsesInputItem
				require.NoError(t, json.Unmarshal(encoded, &output))
				items = append(items, output...)
				items = append(items, oss8832Message("user", "Next real user request"))
				replayed := oss8832Convert(t, model, items)
				got := oss8832Prefix(t, replayed.Messages, "fixture-bound-to-first-sent-prefix")
				wantJSON, err := json.Marshal(oss8832CanonicalPrefix(sent.Messages))
				require.NoError(t, err)
				gotJSON, err := json.Marshal(oss8832CanonicalPrefix(got))
				require.NoError(t, err)
				if model == "claude-opus-5-5" {
					t.Logf("first sent prefix: %s; replayed prefix: %s", wantJSON, gotJSON)
				}
				require.JSONEq(t, string(wantJSON), string(gotJSON), "the prefix of a replayed signed block must still contain the synthetic continuation sent before it was generated")
			})
		}
	}
}

Run from backend:

go test -tags=unit ./internal/pkg/apicompat -run '^TestOSS8832ContinuationSurvivesSignedThinkingReplay$' -count=1

The comparison normalizes string/array text representations and adjacent same-role messages, so the failing distinction is the missing user turn.

AI-assisted local analysis and fixture testing.

@william-drakemond
william-drakemond force-pushed the fix/responses-anthropic-no-prefill-up branch from 322f6b4 to 0fab930 Compare November 19, 2026 19:25
xiaoguai1818 pushed a commit to xiaoguai1818/sub2api that referenced this pull request Nov 21, 2026
Restore Antigravity Responses custom-tool roundtrips and correct Embeddings platform scheduling. Harden OAuth probe fencing, request-scoped Gemini 403 behavior, OpenAI stream failover and WebSocket current-turn safety.

Preserve Claude/GPT-6.2/Grok protocol details and model observations; correct Fable/Kimi quota refreshes and user-vs-account cost reporting. Bound scheduler fallback, request memory retention and shutdown/stream lifecycles without unapproved pricing or schema changes.

Local checks: complete Go unit packages, PostgreSQL 18.1/Redis 8.4 repository and audit integration, focused -race, frontend typecheck/lint/1858 Vitest/build, backend lint 0 issues and embedded server build. Remote CI/Security Scan remain pending.

Upstream references: Wei-Shaw/sub2api Wei-Shaw#8833 Wei-Shaw#8875 Wei-Shaw#8856 Wei-Shaw#8730 Wei-Shaw#8990 Wei-Shaw#8806 Wei-Shaw#8832 Wei-Shaw#8809 Wei-Shaw#8164 Wei-Shaw#8804 Wei-Shaw#8773 Wei-Shaw#8309.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants