Repository navigation
fix(gemini): 兼容入口 chat 请求转发 response_format 结构化输出 - #8662
Merged
Merged
Conversation
/v1/chat/completions 走 Gemini 账号时,response_format 在 chat→Responses→Anthropic 转换中丢失,上游只能靠 prompt 约束 JSON 格式。现在按客户端原始 response_format 写 generationConfig:json_object 写 responseMimeType=application/json;json_schema 另把 json_schema.schema 原样写入 responseJsonSchema。请求带 tools 或上游模型为图片输出 模型时不写。 Code Assist(OAuth + project_id)包装改为原字节嵌入 request,避免 Unmarshal→Marshal 重排 schema 属性顺序、截断大整数。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
Gemini 平台的
/v1/chat/completions兼容入口按 chat → Responses → Anthropic → Gemini 的链路转换请求。response_format在 chat → Responses 这一步会转成text.format,但到 Responses → Anthropic(apicompat.ResponsesToAnthropicRequest)时被丢弃,Gemini 请求的generationConfig里既没有responseMimeType,也没有 schema。结果是客户端即使指定了
response_format: {"type":"json_schema", ...},上游也只能靠 prompt 里的文字约束输出格式。生产上能看到这类请求返回非法 JSON:未转义的引号、尾逗号、套上```json代码块,或者干脆输出 Markdown 文字。改动
新增
applyGeminiChatResponseFormat。在forwardClaudeBodyAsChatCompletions里,紧接思考档位处理之后,根据客户端原始的response_format写入generationConfig:response_format.typejson_objectresponseMimeType: "application/json"json_schemaresponseMimeType,并把json_schema.schema原样写入responseJsonSchema;没有 schema 时只写responseMimeTypetext、缺省、无法解析以下两种情况不写,请求保持原样:
tools:Gemini 2.x 不接受工具与 JSONresponseMimeType同用。isGeminiImageOutputModel,与图片生成门禁同一判定)。schema 写入
responseJsonSchema(JSON Schema 原样),不转换成 OpenAPI 子集的responseSchema,避免有损转换。实测官方 Vertex、AI Studio 以及几家第三方中转,对含$schema、additionalProperties: false、anyOf(含 null)、minLength/maxLength、minimum/maximum、minItems/maxItems、enum的 schema 都返回 200。官方上游会把 schema 计入promptTokenCount,输出为紧凑的合法 JSON。同一个 chat 文件里,Code Assist(OAuth +
project_id)分支原来先把 Gemini 请求 Unmarshal 成any,再与model、project一起 Marshal。这样会把 schema 的属性按字母序重排,客户端有意写成"先 reasoning 后 answer"的顺序会丢失;超过 2^53 的整数也会丢精度。现在改为用sjson.SetRawBytes原样嵌入request,顶层 key 的顺序保持不变。未改动
apicompat.ResponsesToAnthropicRequest。它也被 Claude 平台的 chat/responses 转换共用,改它会让 Claude 上游收到output_config.format。/v1/messages兼容入口(GeminiMessagesCompatService.Forward)同样会丢弃 Anthropic 的output_config.format/output_format。这是另一种协议形状,本 PR 不处理。ForwardNative(/v1beta)的 Code Assist 分支也是同款 Unmarshal → Marshal 包装,客户端原生发送的responseJsonSchema同样会被重排。这个问题在本 PR 之前就存在,本 PR 不处理。测试
TestApplyGeminiChatResponseFormat:覆盖 json_schema、json_object、text、null、非法 JSON、带 tools、图片模型等情况。TestGeminiForwardAsChatCompletions_ForwardsResponseFormat:非流式 json_schema 与流式 json_object 的上游请求体。TestGeminiForwardAsChatCompletions_ResponseFormatSkippedWithToolsTestGeminiForwardAsChatCompletions_CodeAssistKeepsResponseSchemaBytes:断言属性顺序与大整数逐字节保留。CodeAssistKeepsResponseSchemaBytes失败。go test -tags=unit ./internal/service/ ./internal/handler/... ./internal/pkg/apicompat/...go test ./internal/service/ ./internal/handler/... ./internal/pkg/apicompat/...golangci-lint run ./internal/service/... ./internal/handler/...