Repository navigation
fix: retry requeset body incorrect and fix rerank - #1642
Conversation
WalkthroughAdds a generic DeepCopy utility and adopts it across relay handlers to operate on copied request objects. Extends dto.Request with SetModelName and implements setters across DTOs. Refactors model mapping to a centralized helper using dto.Request. Expands ImageRequest fields and adds indirect dependencies. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Client
participant Handler as Relay Handler
participant Copier as common.DeepCopy
participant Mapper as ModelMappedHelper
participant Req as dto.Request
participant Upstream as Adaptor/Upstream
Client->>Handler: Incoming request (concrete DTO)
Handler->>Copier: DeepCopy(original DTO)
alt copy ok
Copier-->>Handler: copied DTO (request)
Handler->>Mapper: ModelMappedHelper(c, info, request)
Mapper->>Req: request.SetModelName(info.UpstreamModelName)
Mapper-->>Handler: mapping result
Handler->>Upstream: Convert & forward (*request)
Upstream-->>Handler: Response
Handler-->>Client: Response
else copy error
Copier-->>Handler: error
Handler-->>Client: API error (InvalidRequest, SkipRetry)
end
sequenceDiagram
autonumber
participant Mapper as ModelMappedHelper
participant Config as model_mapping JSON
participant Info as RelayInfo
participant Req as dto.Request
Mapper->>Config: Load mapping map
Mapper->>Mapper: Resolve chain from OriginModelName (detect cycles)
alt cycle detected
Mapper-->>Info: return model_mapping_contains_cycle
else resolved
Mapper->>Info: Set UpstreamModelName, IsModelMapped
opt request provided
Mapper->>Req: SetModelName(UpstreamModelName)
end
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. ✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 9
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
go.mod (1)
3-4: Fix invalid Go version directives and align toolchain hintThe
godirective must be major.minor only;go 1.23.4is invalid and can break tooling. Also, the Heroku hint pinsgo1.18, which conflicts with modulego 1.23. Align both.Apply one of these diffs:
Option A — simplest, bump Heroku hint and fix go directive:
-// +heroku goVersion go1.18 +// +heroku goVersion go1.23 -go 1.23.4 +go 1.23Option B — if you want to pin a patch toolchain (Go 1.22+ supports this):
-// +heroku goVersion go1.18 +// +heroku goVersion go1.23 -go 1.23.4 +go 1.23 +toolchain go1.23.4Note: Keep the
godirective at1.23; usetoolchainif patch pinning is required. If Heroku doesn’t consume the comment anymore, drop it instead.relay/gemini_handler.go (2)
162-166: Possible panic: unsafe type assertion on resp without nil guard
resp.(*http.Response)will panic if an adaptor ever returnsnilresp withnilerror. Other handlers passhttpResp, which can be nil safely. Mirror that here.Apply this diff:
- usage, openaiErr := adaptor.DoResponse(c, resp.(*http.Response), info) + usage, openaiErr := adaptor.DoResponse(c, httpResp, info)
258-262: *Repeat the nil-guard fix for embedding: avoid resp.(http.Response)Same potential panic exists here; pass the
httpRespvariable instead.Apply this diff:
- usage, openaiErr := adaptor.DoResponse(c, resp.(*http.Response), info) + usage, openaiErr := adaptor.DoResponse(c, httpResp, info)relay/compatible_handler.go (1)
52-66: IncludeUsage may desync from ForceStreamOption; compute after stream-options normalization.If the request supplied
StreamOptions.IncludeUsage=falseandForceStreamOption=true,includeUsageremains false, diverging from the forced true.Apply this diff to recalculate
includeUsageafter the StreamOptions block:- includeUsage := true - // 判断用户是否需要返回使用情况 - if request.StreamOptions != nil { - includeUsage = request.StreamOptions.IncludeUsage - } + includeUsage := true @@ - if !info.SupportStreamOptions || !request.Stream { - request.StreamOptions = nil - } else { - // 如果支持StreamOptions,且请求中没有设置StreamOptions,根据配置文件设置StreamOptions - if constant.ForceStreamOption { - request.StreamOptions = &dto.StreamOptions{ - IncludeUsage: true, - } - } - } + if !info.SupportStreamOptions || !request.Stream { + request.StreamOptions = nil + } else if constant.ForceStreamOption { + // 如果支持StreamOptions,且请求中没有设置StreamOptions,根据配置文件设置StreamOptions + request.StreamOptions = &dto.StreamOptions{IncludeUsage: true} + } + // 归一化后再计算 includeUsage + if request.StreamOptions != nil { + includeUsage = request.StreamOptions.IncludeUsage + } else { + includeUsage = true + }
🧹 Nitpick comments (16)
go.mod (1)
47-51: Remove manually pinned indirect dependencies; let Go derive them
- The only package directly imported by your code is
github.com/antlabs/pcopy(incommon/copy.go). All other entries (copystructure,reflectwalk,sprig/v3,semver/v3,goutils,xstrings,mergo,cast) aren’t directly referenced and will be pulled in transitively bygo mod tidy.- Change the
pcopyline ingo.modto a regular (direct) requirement (remove the// indirectcomment).- Delete the other eight manually listed indirect dependencies (lines 47–51, 76–78, 88–91, 95).
- Run:
to let Go recompute indirect modules automatically.go mod tidy -v- (Optional) After tidying, check if there are newer patch/minor releases for
github.com/antlabs/pcopyand other direct deps, and bump them as appropriate.dto/embedding.go (1)
51-55: EmbeddingRequest model setter is straightforward and safeMatches the interface and respects the empty-model no-op convention. No functional issues.
Minor future tidy: elsewhere in this file, type switches re-assert the same types (e.g.,
r.Input.(string)inside aswitch r.Input.(type)); you can bind to a variable in the switch header to avoid repeated assertions. Not required for this PR.dto/openai_request.go (2)
186-190: SetModelName: trim whitespace; also note a likely field typo
- Suggest trimming whitespace before checking emptiness to avoid setting a whitespace-only model name.
- Minor nit: field name "ParallelTooCalls" (Line 51) looks misspelled; consider renaming to "ParallelToolCalls" in a follow-up that coordinates downstream references.
Apply within the selected range:
func (r *GeneralOpenAIRequest) SetModelName(modelName string) { - if modelName != "" { - r.Model = modelName + if s := strings.TrimSpace(modelName); s != "" { + r.Model = s } }Also applies to: 51-52
850-854: SetModelName: trim whitespace to avoid setting invalid model namesSame rationale as above; keep behavior consistent across DTOs.
func (r *OpenAIResponsesRequest) SetModelName(modelName string) { - if modelName != "" { - r.Model = modelName + if s := strings.TrimSpace(modelName); s != "" { + r.Model = s } }common/copy.go (1)
17-19: Remove impossible nil check on address of local variable
&dstcan never be nil in Go; this branch is dead code and should be removed.- if &dst == nil { - return nil, fmt.Errorf("copy result cannot be nil") - }If helpful, I can add a table-driven unit test to validate deep copy semantics (structs with maps/slices/pointers) and ensure mutations to source don’t affect the copy. Do you want me to draft it?
dto/openai_image.go (1)
66-70: SetModelName: trim whitespacePrevent propagating accidental whitespace-only model names.
func (i *ImageRequest) SetModelName(modelName string) { - if modelName != "" { - i.Model = modelName + if s := strings.TrimSpace(modelName); s != "" { + i.Model = s } }dto/gemini.go (2)
324-335: Token count meta: good coverage; optional enhancementCurrent logic aggregates text parts. Optional: include Title (when present) to better reflect embedding inputs for search-oriented tasks.
func (r *GeminiEmbeddingRequest) GetTokenCountMeta() *types.TokenCountMeta { var inputTexts []string for _, part := range r.Content.Parts { if part.Text != "" { inputTexts = append(inputTexts, part.Text) } } + if r.Title != "" { + inputTexts = append(inputTexts, r.Title) + } inputText := strings.Join(inputTexts, "\n") return &types.TokenCountMeta{ CombineText: inputText, } }Also applies to: 352-364
337-341: SetModelName: trim whitespace; batch propagation looks good
- Trim whitespace before assignment for consistency.
- Batch propagation over Requests is correct and aligns with the requirement that each item carries its own model.
func (r *GeminiEmbeddingRequest) SetModelName(modelName string) { - if modelName != "" { - r.Model = modelName + if s := strings.TrimSpace(modelName); s != "" { + r.Model = s } }Also applies to: 366-372
dto/audio.go (1)
29-33: SetModelName: trim whitespace; add importTrim whitespace before setting to avoid invalid model names. This file doesn’t currently import strings; add it.
func (r *AudioRequest) SetModelName(modelName string) { - if modelName != "" { - r.Model = modelName + if s := strings.TrimSpace(modelName); s != "" { + r.Model = s } }Outside the selected range (add missing import):
import ( "one-api/types" "github.com/gin-gonic/gin" "strings" // add )relay/responses_handler.go (1)
28-31: Fix error message: wrong type name in copy failureMessage says “GeneralOpenAIRequest” but we’re copying an OpenAIResponsesRequest. Keep error text accurate.
Apply this diff:
- return types.NewError(fmt.Errorf("failed to copy request to GeneralOpenAIRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry()) + return types.NewError(fmt.Errorf("failed to copy OpenAIResponsesRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry())relay/audio_handler.go (1)
20-23: Make the error message more specific (optional)Include the expected and actual types to aid diagnostics; optionally set status 400.
Apply this diff:
- return types.NewError(errors.New("invalid request type"), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry()) + return types.NewErrorWithStatusCode( + fmt.Errorf("invalid request type: expected *dto.AudioRequest, got %T", info.Request), + types.ErrorCodeInvalidRequest, + http.StatusBadRequest, + types.ErrOptionWithSkipRetry(), + )relay/gemini_handler.go (1)
72-88: Optional: keep request’s model in sync after thinking-adapter retargetingAfter adjusting
info.OriginModelName/info.UpstreamModelNameto a “-nothinking” variant, callrequest.SetModelName(info.UpstreamModelName)to keep the request object consistent for adaptors that may consult it.Proposed snippet (outside the shown range where you switch the names):
// After updating info.(Origin|Upstream)ModelName if request != nil { request.SetModelName(info.UpstreamModelName) }relay/rerank_handler.go (1)
27-30: Fix incorrect error message: refers to ImageRequest in rerank handler.The returned error says “copy request to ImageRequest” but this is a rerank path, which will confuse operators and users.
Apply this diff:
- request, err := common.DeepCopy(rerankReq) - if err != nil { - return types.NewError(fmt.Errorf("failed to copy request to ImageRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry()) - } + request, err := common.DeepCopy(rerankReq) + if err != nil { + return types.NewError(fmt.Errorf("failed to copy request to RerankRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry()) + }relay/image_handler.go (1)
62-66: Use common.Marshal for consistency with other handlers.Other handlers use
common.Marshal(centralized JSON settings). Keep this consistent to avoid subtle encoding differences.Apply this diff:
- jsonData, err := json.Marshal(convertedRequest) + jsonData, err := common.Marshal(convertedRequest)Additionally remove the now-unused import:
-import ( - "bytes" - "encoding/json" +import ( + "bytes"relay/claude_handler.go (1)
50-70: Thinking adapter: minor robustness and rounding nit.
- Consider rounding
BudgetTokensinstead of truncation to avoid off-by-one under-allocation on percentages.- The suffix trim and assignment look good; they operate after model mapping and only when the effective upstream model still has “-thinking”.
Apply this diff:
- request.Thinking = &dto.Thinking{ - Type: "enabled", - BudgetTokens: common.GetPointer[int](int(float64(request.MaxTokens) * model_setting.GetClaudeSettings().ThinkingAdapterBudgetTokensPercentage)), - } + request.Thinking = &dto.Thinking{ + Type: "enabled", + BudgetTokens: func() *int { + bt := int(math.Round(float64(request.MaxTokens) * + model_setting.GetClaudeSettings().ThinkingAdapterBudgetTokensPercentage)) + return &bt + }(), + }Note: you’ll need to import
math.relay/helper/model_mapped.go (1)
18-20: Use errors.New instead of fmt.Errorf for constant strings; drop fmt import.Minor cleanup to avoid unnecessary import and align with Go idioms.
Apply this diff:
- err := json.Unmarshal([]byte(modelMapping), &modelMap) - if err != nil { - return fmt.Errorf("unmarshal_model_mapping_failed") - } + if err := json.Unmarshal([]byte(modelMapping), &modelMap); err != nil { + return errors.New("unmarshal_model_mapping_failed") + }And adjust imports:
-import ( - "encoding/json" - "errors" - "fmt" - "github.com/gin-gonic/gin" - "one-api/dto" - "one-api/relay/common" -) +import ( + "encoding/json" + "errors" + "github.com/gin-gonic/gin" + "one-api/dto" + "one-api/relay/common" +)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (20)
common/copy.go(1 hunks)dto/audio.go(1 hunks)dto/claude.go(1 hunks)dto/embedding.go(1 hunks)dto/gemini.go(2 hunks)dto/openai_image.go(2 hunks)dto/openai_request.go(2 hunks)dto/request_common.go(2 hunks)dto/rerank.go(1 hunks)go.mod(3 hunks)relay/audio_handler.go(3 hunks)relay/claude_handler.go(3 hunks)relay/common/relay_info.go(1 hunks)relay/compatible_handler.go(2 hunks)relay/embedding_handler.go(2 hunks)relay/gemini_handler.go(2 hunks)relay/helper/model_mapped.go(2 hunks)relay/image_handler.go(3 hunks)relay/rerank_handler.go(2 hunks)relay/responses_handler.go(1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2025-08-05T17:14:17.246Z
Learnt from: neotf
PR: QuantumNous/new-api#1511
File: setting/ratio_setting/model_ratio.go:118-123
Timestamp: 2025-08-05T17:14:17.246Z
Learning: Claude models handle "-thinking" variants differently from Gemini models. For Claude models, only the base model (without "-thinking") gets an entry in defaultModelRatio map. The "-thinking" variants rely on the Claude relay handler stripping the suffix using strings.TrimSuffix(textRequest.Model, "-thinking") before looking up the ratio, so they automatically use the base model's ratio.
Applied to files:
relay/claude_handler.go
📚 Learning: 2025-08-08T17:12:43.157Z
Learnt from: RedwindA
PR: QuantumNous/new-api#1537
File: relay/gemini_handler.go:330-342
Timestamp: 2025-08-08T17:12:43.157Z
Learning: In the new-api repository, the `GeminiEmbeddingHandler` function in `relay/gemini_handler.go` is designed specifically for native Gemini embedding requests and therefore does not require the `ConvertGeminiRequest` step that is used in the chat handler. The embedding requests are already in the native Gemini format and don't need conversion.
Applied to files:
dto/gemini.gorelay/embedding_handler.gorelay/gemini_handler.go
📚 Learning: 2025-08-04T14:36:34.528Z
Learnt from: antecanis8
PR: QuantumNous/new-api#1500
File: relay/channel/gemini/adaptor.go:0-0
Timestamp: 2025-08-04T14:36:34.528Z
Learning: In the Gemini API batchEmbedContents endpoint, each request item in the "requests" array should include its own "model" field, as shown in the official documentation at https://ai.google.dev/api/embeddings?hl=zh-cn#method:-models.batchembedcontents. The model field is not a top-level parameter.
Applied to files:
dto/gemini.go
🧬 Code graph analysis (14)
dto/openai_request.go (1)
model/model_meta.go (1)
Model(22-42)
dto/rerank.go (1)
model/model_meta.go (1)
Model(22-42)
relay/rerank_handler.go (7)
dto/request_common.go (1)
Request(8-12)dto/rerank.go (1)
RerankRequest(10-18)common/sys_log.go (1)
FatalLog(20-24)common/copy.go (1)
DeepCopy(8-21)types/error.go (3)
NewError(181-193)ErrorCodeInvalidRequest(38-38)ErrOptionWithSkipRetry(290-294)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)relay/channel/ali/rerank.go (1)
ConvertRerankRequest(15-32)
relay/claude_handler.go (8)
dto/request_common.go (1)
Request(8-12)dto/claude.go (2)
ClaudeRequest(187-203)Thinking(399-402)common/sys_log.go (1)
FatalLog(20-24)common/copy.go (1)
DeepCopy(8-21)types/error.go (3)
NewError(181-193)ErrorCodeInvalidRequest(38-38)ErrOptionWithSkipRetry(290-294)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)setting/model_setting/claude.go (1)
GetClaudeSettings(41-47)common/utils.go (1)
GetPointer(218-220)
relay/common/relay_info.go (1)
dto/request_common.go (1)
Request(8-12)
dto/gemini.go (1)
types/request_meta.go (1)
TokenCountMeta(20-31)
relay/audio_handler.go (5)
dto/request_common.go (1)
Request(8-12)dto/audio.go (1)
AudioRequest(9-15)types/error.go (2)
NewError(181-193)ErrOptionWithSkipRetry(290-294)common/copy.go (1)
DeepCopy(8-21)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)
relay/responses_handler.go (6)
dto/request_common.go (1)
Request(8-12)dto/openai_request.go (1)
OpenAIResponsesRequest(762-784)common/sys_log.go (1)
FatalLog(20-24)common/copy.go (1)
DeepCopy(8-21)types/error.go (3)
NewError(181-193)ErrorCodeInvalidRequest(38-38)ErrOptionWithSkipRetry(290-294)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)
relay/embedding_handler.go (6)
dto/request_common.go (1)
Request(8-12)dto/embedding.go (1)
EmbeddingRequest(21-32)common/sys_log.go (1)
FatalLog(20-24)common/copy.go (1)
DeepCopy(8-21)types/error.go (3)
NewError(181-193)ErrorCodeInvalidRequest(38-38)ErrOptionWithSkipRetry(290-294)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)
relay/image_handler.go (6)
dto/request_common.go (1)
Request(8-12)dto/openai_image.go (1)
ImageRequest(11-28)common/sys_log.go (1)
FatalLog(20-24)common/copy.go (1)
DeepCopy(8-21)types/error.go (3)
NewError(181-193)ErrorCodeInvalidRequest(38-38)ErrOptionWithSkipRetry(290-294)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)
relay/helper/model_mapped.go (2)
relay/common/relay_info.go (1)
RelayInfo(73-118)dto/request_common.go (1)
Request(8-12)
relay/gemini_handler.go (6)
dto/request_common.go (1)
Request(8-12)dto/gemini.go (1)
GeminiChatRequest(12-18)common/sys_log.go (1)
FatalLog(20-24)common/copy.go (1)
DeepCopy(8-21)types/error.go (3)
NewError(181-193)ErrorCodeInvalidRequest(38-38)ErrOptionWithSkipRetry(290-294)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)
relay/compatible_handler.go (7)
dto/request_common.go (1)
Request(8-12)dto/openai_request.go (3)
GeneralOpenAIRequest(25-72)WebSearchOptions(756-759)StreamOptions(223-225)common/sys_log.go (1)
FatalLog(20-24)common/copy.go (1)
DeepCopy(8-21)types/error.go (4)
NewError(181-193)ErrorCodeInvalidRequest(38-38)ErrOptionWithSkipRetry(290-294)ErrorCodeChannelModelMappedError(53-53)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)constant/env.go (1)
ForceStreamOption(6-6)
dto/openai_image.go (1)
dto/openai_request.go (1)
ResponseFormat(13-16)
🔇 Additional comments (25)
dto/request_common.go (1)
11-11: Manual Verification Required: ConfirmSetModelNameon All Request TypesThe automated checks encountered parsing issues and did not reliably report which DTOs implement
SetModelNameor embedBaseRequest. Please manually verify that every*Requeststruct in your codebase either:
- Defines an explicit
method, orfunc (r *XRequest) SetModelName(modelName string)- Embeds
BaseRequest(which provides the no-op default setter).Key places to inspect:
- All
...Requesttypes underdto/(or wherever your request DTOs live).- Ensure none of your request structs are left without a setter or embedded default.
Once confirmed, consider adding compile-time assertions to guard against future omissions, for example:
var _ Request = (*YourRequestType)(nil)dto/rerank.go (1)
40-44: Setter aligns rerank model propagation with the rest of DTOsThe
SetModelNameimplementation is consistent with other DTOs and guards on empty strings.Add/extend tests to cover:
- Incoming request with empty
Modeland non-empty propagated model.- Pre-set
Modelnot overridden whenmodelName == "".- Retry path with DeepCopy uses the copied request, not the original (to avoid mutating the caller during retries).
dto/claude.go (2)
324-326: Param rename avoids receiver shadowing; returns the intended stream flagLGTM. Returning
c.Streamis the correct behavior here.
328-331: ClaudeRequest gains SetModelName; consistent with DTO interfaceNo issues. The empty-string guard mirrors other DTOs.
dto/gemini.go (2)
76-79: No-op SetModelName is correct for GeminiChatRequestGemini chat payload has no top-level model field; a no-op implementation is appropriate and satisfies the new interface.
319-323: IsStream correctly returns false for embeddingsNon-streaming semantics for both single and batch embeddings are correct.
Also applies to: 347-351
relay/common/relay_info.go (1)
166-168: Good: align request’s model to the origin earlySetting the request’s model to
OriginModelNameup front reduces surprises downstream and makes DeepCopy-based flows consistent.relay/responses_handler.go (2)
33-34: LGTM: map model on the copied requestSwitching
ModelMappedHelperto operate on the copied request avoids mutating the original input.
51-51: LGTM: pass the copied request to the adaptorPassing
*requestmaintains the copy-only mutation contract established earlier.relay/embedding_handler.go (2)
26-31: LGTM: copy-then-map flow is correctDeep-copying first, then model-mapping via the
dto.Requestinterface is consistent with the PR-wide pattern and keeps the original request immutable.
42-42: LGTM: use the copied request in conversionUsing
*requesthere ensures downstream mutations don’t touch the original object.relay/audio_handler.go (2)
20-23: Good change: return a typed error instead of crashingSwitching to
types.NewError(..., invalid_request, SkipRetry)on type mismatch is the right behavior for a server.
25-31: LGTM: adopt DeepCopy + model mapping on the copyThis mirrors the pattern in other handlers and avoids mutating
info.Request.relay/gemini_handler.go (2)
61-68: LGTM: copy before mapping; operate on the copyThis aligns with the new DeepCopy-first contract and keeps the original immutable.
178-179: LGTM: tighten type to dto.Request for embedding handlerUsing
dto.Requestinstead ofanyimproves type-safety and integrates withSetModelName.relay/rerank_handler.go (2)
32-33: Typed ModelMappedHelper usage looks correct.Passing the copied
*dto.RerankRequestasdto.Requestensures model mapping updates propagate viaSetModelNamewithout mutating the original payload.
51-52: ConvertRerankRequest signature already correctAll
ConvertRerankRequestmethods in theAdaptorinterface and every channel implementation take adto.RerankRequestby value, and the handler correctly dereferences the pointer (*request) to pass that value. No changes are needed to avoid build breaks.relay/image_handler.go (2)
30-36: DeepCopy + invalid_request on copy failure — good.This addresses the “retry request body incorrect” class of bugs by preventing in-place mutations of the original DTO.
107-112: Token and log fields now reference the copied request — LGTM.Using
request.N,request.Quality, andrequest.Sizemaintains internal consistency post-copy.Also applies to: 115-117, 121-123
relay/claude_handler.go (3)
30-36: DeepCopy path and error classification are correct.Returning
invalid_requestwithSkipRetryon copy failure matches the PR’s intent and avoids retry loops.
46-48: Default MaxTokens fallback is reasonable.Deriving defaults from model settings when omitted is aligned with existing patterns.
80-81: *All ConvertClaudeRequest signatures match dto.ClaudeRequest – no action neededI’ve checked every
ConvertClaudeRequestimplementation underrelay/channel/*/adaptor.go, and they all use the exact signature:func (a *Adaptor) ConvertClaudeRequest( *gin.Context, *relaycommon.RelayInfo, *dto.ClaudeRequest, ) (any, error)so there is no risk of a signature mismatch when passing a
*dto.ClaudeRequest. The call site inrelay/claude_handler.goalready passesrequest(which is declared as a*dto.ClaudeRequest), so everything lines up correctly.relay/compatible_handler.go (2)
36-40: DeepCopy + typed ModelMappedHelper usage — LGTM.The flow isolates mutations to the copied request and centralizes model mapping, which should address retry-body mutation issues.
Also applies to: 45-48
87-88: I’ve broadened the search to locate both the definition and all usages of ConvertOpenAIRequest and references to GeneralOpenAIRequest across the repo. Once we have those results, we can confirm if the signature changed to a pointer and whether all adaptors align with it.relay/helper/model_mapped.go (1)
12-12: Typed dto.Request improves safety and simplifies call sites.The new signature removes brittle type switches and standardizes model propagation via
SetModelName.
|
|
||
| info.ChannelMeta = channelMeta | ||
|
|
||
| // reset some fields based on channel meta | ||
| // 重置某些字段,例如模型名称等 | ||
| if info.Request != nil { | ||
| info.Request.SetModelName(info.OriginModelName) | ||
| } |
There was a problem hiding this comment.
Prevent blanking model name during mapping: initialize UpstreamModelName before SetModelName
If no model-mapping is provided, info.UpstreamModelName remains empty. Later, ModelMappedHelper unconditionally does request.SetModelName(info.UpstreamModelName), which will overwrite the request’s model with an empty string. Initialize info.UpstreamModelName to info.OriginModelName here to make the default safe.
Apply this diff:
info.ChannelMeta = channelMeta
+ // initialize upstream model to origin by default to avoid empty overwrite later
+ info.UpstreamModelName = info.OriginModelName
+
// reset some fields based on channel meta
// 重置某些字段,例如模型名称等
if info.Request != nil {
info.Request.SetModelName(info.OriginModelName)
}Additionally, harden the mapper to avoid writing empty names when no mapping occurs (change in another file, shown for completeness):
--- a/relay/helper/model_mapped.go
+++ b/relay/helper/model_mapped.go
@@
- if request != nil {
- request.SetModelName(info.UpstreamModelName)
- }
+ if request != nil {
+ target := info.UpstreamModelName
+ if target == "" {
+ target = info.OriginModelName
+ }
+ request.SetModelName(target)
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| info.ChannelMeta = channelMeta | |
| // reset some fields based on channel meta | |
| // 重置某些字段,例如模型名称等 | |
| if info.Request != nil { | |
| info.Request.SetModelName(info.OriginModelName) | |
| } | |
| info.ChannelMeta = channelMeta | |
| // initialize upstream model to origin by default to avoid empty overwrite later | |
| info.UpstreamModelName = info.OriginModelName | |
| // reset some fields based on channel meta | |
| // 重置某些字段,例如模型名称等 | |
| if info.Request != nil { | |
| info.Request.SetModelName(info.OriginModelName) | |
| } |
🤖 Prompt for AI Agents
In relay/common/relay_info.go around lines 161 to 168, info.UpstreamModelName is
left empty when no mapping exists which allows later mapping logic to overwrite
the request's model with an empty string; initialize info.UpstreamModelName =
info.OriginModelName before calling request.SetModelName(info.OriginModelName)
so the default is safe, and also update the mapper to only call
request.SetModelName(...) when the mapped name is non-empty to prevent blanking
when no mapping occurs.
| request.SetModelName(info.UpstreamModelName) | ||
| } |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Ensure UpstreamModelName is set before calling SetModelName.
If InitChannelMeta didn’t set a default, this could blank the request’s model. Please confirm info.UpstreamModelName is initialized to info.OriginModelName during InitChannelMeta.
Run:
🏁 Script executed:
#!/bin/bash
# Check that InitChannelMeta initializes UpstreamModelName
rg -n 'func\s+InitChannelMeta\(' -n
rg -n 'UpstreamModelName\s*=' -n relay/common/relay_info.go -C3Length of output: 103
I couldn’t find any evidence that InitChannelMeta sets UpstreamModelName to OriginModelName. Let’s search the whole repo to be sure:
🏁 Script executed:
#!/bin/bash
# Search for InitChannelMeta definition
rg -n 'func InitChannelMeta' -C3
# Look for any assignment to UpstreamModelName
rg -n 'UpstreamModelName\s*=' -C3
# Check for usage of OriginModelName as a fallback
rg -n 'OriginModelName' -C3Length of output: 20579
Initialize UpstreamModelName before mapping to avoid blank models
In relay/helper/model_mapped.go, the final block reads:
if request != nil {
request.SetModelName(info.UpstreamModelName)
}However, if info.IsModelMapped is false, info.UpstreamModelName was never set and this will clear out the request’s model. There is no InitChannelMeta function in this repo—by default only info.Request.SetModelName(info.OriginModelName) is called in relay/common/relay_info.go, and that does not initialize UpstreamModelName.
Please add an explicit default assignment at the top of the mapping function so UpstreamModelName always falls back to OriginModelName:
func applyModelMapping(info *relaycommon.RelayInfo, request *YourRequestType) {
+ // ensure we have a valid default before any remapping
+ info.UpstreamModelName = info.OriginModelName
currentModel := info.OriginModelName
visitedModels := map[string]bool{ currentModel: true }
// … existing mapping logic …
if request != nil {
request.SetModelName(info.UpstreamModelName)
}
}🤖 Prompt for AI Agents
In relay/helper/model_mapped.go around lines 54-55, ensure
info.UpstreamModelName is initialized before it's applied to the request: at the
top of the mapping function set a default so UpstreamModelName falls back to
OriginModelName (e.g. if info.UpstreamModelName is empty, assign
info.OriginModelName) so the subsequent
request.SetModelName(info.UpstreamModelName) cannot clear the request's model
when no mapping was provided.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
relay/compatible_handler.go (1)
49-67: Fix potential mismatch: ForceStreamOption can flip IncludeUsage but info.ShouldIncludeUsage stays staleIf ForceStreamOption sets IncludeUsage to true while the original StreamOptions had IncludeUsage=false, info.ShouldIncludeUsage remains false because it’s computed before the override. Recompute after the StreamOptions block.
- includeUsage := true - // 判断用户是否需要返回使用情况 - if request.StreamOptions != nil { - includeUsage = request.StreamOptions.IncludeUsage - } - // 如果不支持StreamOptions,将StreamOptions设置为nil if !info.SupportStreamOptions || !request.Stream { request.StreamOptions = nil } else { // 如果支持StreamOptions,且请求中没有设置StreamOptions,根据配置文件设置StreamOptions if constant.ForceStreamOption { request.StreamOptions = &dto.StreamOptions{ IncludeUsage: true, } } } - info.ShouldIncludeUsage = includeUsage + // 统一在覆盖后计算 IncludeUsage + includeUsage := true + if request.StreamOptions != nil { + includeUsage = request.StreamOptions.IncludeUsage + } + info.ShouldIncludeUsage = includeUsage
♻️ Duplicate comments (3)
relay/responses_handler.go (1)
23-26: Good fix: return 400 on bad type assertion instead of exiting the processThis aligns with earlier guidance to replace FatalLog/os.Exit with a typed API error. Message includes the actual type.
relay/compatible_handler.go (1)
30-33: Nice: bad type assertion now returns a 400 NewAPIErrorThis matches the pattern adopted across handlers and prevents process termination on user input errors.
relay/claude_handler.go (1)
24-28: Good: 400 on bad type assertion with SkipRetryThis removes the FatalLog crash path and returns a proper API error.
🧹 Nitpick comments (6)
relay/responses_handler.go (3)
28-31: Fix incorrect error message: copy target type is wrongThe message says “GeneralOpenAIRequest” while the code is copying an OpenAIResponsesRequest. Correct to avoid confusion during ops/debugging.
- return types.NewError(fmt.Errorf("failed to copy request to GeneralOpenAIRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry()) + return types.NewError(fmt.Errorf("failed to copy OpenAIResponsesRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry())
44-51: Honor channel-level pass-through flag for responses handlerTextHelper supports both global and channel-level pass-through; ResponsesHelper only checks the global setting. Align behavior to avoid surprises per-channel.
- if model_setting.GetGlobalSettings().PassThroughRequestEnabled { + if model_setting.GetGlobalSettings().PassThroughRequestEnabled || info.ChannelSetting.PassThroughBodyEnabled { body, err := common.GetRequestBody(c) if err != nil { - return types.NewError(err, types.ErrorCodeReadRequestBodyFailed, types.ErrOptionWithSkipRetry()) + return types.NewErrorWithStatusCode(err, types.ErrorCodeReadRequestBodyFailed, http.StatusBadRequest, types.ErrOptionWithSkipRetry()) } requestBody = bytes.NewBuffer(body)
67-71: Use structured debug logging instead of println and keep consistency with TextHelperPrefer logger.LogDebug for consistency and to integrate with log levels/fields.
import ( "bytes" "fmt" "io" "net/http" "one-api/common" "one-api/dto" + "one-api/logger" relaycommon "one-api/relay/common" "one-api/relay/helper" "one-api/service" "one-api/setting/model_setting" "one-api/types" "strings" "github.com/gin-gonic/gin" ) @@ - if common.DebugEnabled { - println("requestBody: ", string(jsonData)) - } + logger.LogDebug(c, fmt.Sprintf("responses request body: %s", string(jsonData)))Also applies to: 3-18
relay/compatible_handler.go (2)
76-85: Prefer structured debug logging over println for passthrough bodyUse logger.LogDebug as done later for the JSON body; removes ad-hoc prints and centralizes log control.
- if common.DebugEnabled { - println("requestBody: ", string(body)) - } + logger.LogDebug(c, fmt.Sprintf("text passthrough request body: %s", string(body)))
86-107: Avoid variable shadowing: “request := convertedRequest...” hides the outer requestThe inner “request” for convertedRequest shadows the outer pointer “request”, which is easy to misread and error-prone. Rename the inner variable.
- request := convertedRequest.(*dto.GeneralOpenAIRequest) + convReq := convertedRequest.(*dto.GeneralOpenAIRequest) containSystemPrompt := false - for _, message := range request.Messages { - if message.Role == request.GetSystemRoleName() { + for _, message := range convReq.Messages { + if message.Role == convReq.GetSystemRoleName() { containSystemPrompt = true break } } if !containSystemPrompt { // 如果没有系统提示,则添加系统提示 systemMessage := dto.Message{ - Role: request.GetSystemRoleName(), + Role: convReq.GetSystemRoleName(), Content: info.ChannelSetting.SystemPrompt, } - request.Messages = append([]dto.Message{systemMessage}, request.Messages...) + convReq.Messages = append([]dto.Message{systemMessage}, convReq.Messages...) } else if info.ChannelSetting.SystemPromptOverride { common.SetContextKey(c, constant.ContextKeySystemPromptOverride, true) // 如果有系统提示,且允许覆盖,则拼接到前面 - for i, message := range request.Messages { - if message.Role == request.GetSystemRoleName() { + for i, message := range convReq.Messages { + if message.Role == convReq.GetSystemRoleName() { if message.IsStringContent() { - request.Messages[i].SetStringContent(info.ChannelSetting.SystemPrompt + "\n" + message.StringContent()) + convReq.Messages[i].SetStringContent(info.ChannelSetting.SystemPrompt + "\n" + message.StringContent()) } else { contents := message.ParseContent() contents = append([]dto.MediaContent{ { Type: dto.ContentTypeText, Text: info.ChannelSetting.SystemPrompt, }, }, contents...) - request.Messages[i].Content = contents + convReq.Messages[i].Content = contents } break } } }relay/claude_handler.go (1)
97-101: Use structured debug logging for request body instead of printlnKeep logging consistent with other handlers and avoid ad-hoc prints.
import ( "bytes" "fmt" "io" "net/http" "one-api/common" + "one-api/logger" "one-api/dto" relaycommon "one-api/relay/common" "one-api/relay/helper" "one-api/service" "one-api/setting/model_setting" "one-api/types" "strings" @@ - if common.DebugEnabled { - println("requestBody: ", string(jsonData)) - } + logger.LogDebug(c, fmt.Sprintf("claude request body: %s", string(jsonData)))Also applies to: 3-18
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (8)
common/gin.go(1 hunks)relay/claude_handler.go(3 hunks)relay/compatible_handler.go(2 hunks)relay/embedding_handler.go(2 hunks)relay/gemini_handler.go(2 hunks)relay/image_handler.go(3 hunks)relay/rerank_handler.go(2 hunks)relay/responses_handler.go(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- common/gin.go
🚧 Files skipped from review as they are similar to previous changes (4)
- relay/rerank_handler.go
- relay/embedding_handler.go
- relay/image_handler.go
- relay/gemini_handler.go
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-05T17:14:17.246Z
Learnt from: neotf
PR: QuantumNous/new-api#1511
File: setting/ratio_setting/model_ratio.go:118-123
Timestamp: 2025-08-05T17:14:17.246Z
Learning: Claude models handle "-thinking" variants differently from Gemini models. For Claude models, only the base model (without "-thinking") gets an entry in defaultModelRatio map. The "-thinking" variants rely on the Claude relay handler stripping the suffix using strings.TrimSuffix(textRequest.Model, "-thinking") before looking up the ratio, so they automatically use the base model's ratio.
Applied to files:
relay/claude_handler.go
🧬 Code graph analysis (3)
relay/claude_handler.go (7)
dto/request_common.go (1)
Request(8-12)dto/claude.go (2)
ClaudeRequest(187-203)Thinking(399-402)types/error.go (3)
NewErrorWithStatusCode(215-231)ErrOptionWithSkipRetry(290-294)NewError(181-193)common/copy.go (1)
DeepCopy(8-21)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)setting/model_setting/claude.go (1)
GetClaudeSettings(41-47)common/utils.go (1)
GetPointer(218-220)
relay/responses_handler.go (5)
dto/request_common.go (1)
Request(8-12)dto/openai_request.go (1)
OpenAIResponsesRequest(762-784)types/error.go (4)
NewErrorWithStatusCode(215-231)ErrorCodeInvalidRequest(38-38)ErrOptionWithSkipRetry(290-294)NewError(181-193)common/copy.go (1)
DeepCopy(8-21)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)
relay/compatible_handler.go (6)
dto/request_common.go (1)
Request(8-12)dto/openai_request.go (3)
GeneralOpenAIRequest(25-72)WebSearchOptions(756-759)StreamOptions(223-225)types/error.go (5)
NewErrorWithStatusCode(215-231)ErrorCodeInvalidRequest(38-38)ErrOptionWithSkipRetry(290-294)NewError(181-193)ErrorCodeChannelModelMappedError(53-53)common/copy.go (1)
DeepCopy(8-21)relay/helper/model_mapped.go (1)
ModelMappedHelper(12-57)constant/env.go (1)
ForceStreamOption(6-6)
🔇 Additional comments (2)
relay/responses_handler.go (1)
33-34: UpstreamModelName is properly initialized in InitChannelMeta, no action neededInitChannelMeta (relay/common/relay_info.go:120–126) sets ChannelMeta.UpstreamModelName to the original model string from the context (constant.ContextKeyOriginalModel), which is then promoted to info.UpstreamModelName before any calls to ModelMappedHelper.
• ChannelMeta.UpstreamModelName assignment:UpstreamModelName: common.GetContextKeyString(c, constant.ContextKeyOriginalModel),• InitChannelMeta is always invoked (e.g. in ResponsesHelper at relay/responses_handler.go:21), ensuring info.UpstreamModelName is non-empty when ModelMappedHelper calls request.SetModelName(info.UpstreamModelName).
relay/claude_handler.go (1)
30-33: DeepCopy on input request is fineCopying before mutation prevents side effects on the original payload; error handling is appropriate.
| err = helper.ModelMappedHelper(c, info, request) | ||
| if err != nil { | ||
| return types.NewError(err, types.ErrorCodeChannelModelMappedError, types.ErrOptionWithSkipRetry()) | ||
| } |
There was a problem hiding this comment.
Normalize “-thinking” before model mapping; don’t override UpstreamModelName after mapping
Currently you map first, then trim “-thinking” and overwrite info.UpstreamModelName. This can defeat model mappings and desync request vs info. Normalize the model name before ModelMappedHelper, and let mapping set UpstreamModelName. Then apply thinking defaults after mapping without overwriting UpstreamModelName.
- err = helper.ModelMappedHelper(c, info, request)
- if err != nil {
- return types.NewError(err, types.ErrorCodeChannelModelMappedError, types.ErrOptionWithSkipRetry())
- }
+ // Normalize "-thinking" before mapping so mapping/ratios use the base model
+ thinkingVariant := false
+ if model_setting.GetClaudeSettings().ThinkingAdapterEnabled && strings.HasSuffix(request.Model, "-thinking") {
+ request.Model = strings.TrimSuffix(request.Model, "-thinking")
+ thinkingVariant = true
+ }
+ err = helper.ModelMappedHelper(c, info, request)
+ if err != nil {
+ return types.NewError(err, types.ErrorCodeChannelModelMappedError, types.ErrOptionWithSkipRetry())
+ }
@@
- if request.MaxTokens == 0 {
- request.MaxTokens = uint(model_setting.GetClaudeSettings().GetDefaultMaxTokens(request.Model))
- }
-
- if model_setting.GetClaudeSettings().ThinkingAdapterEnabled &&
- strings.HasSuffix(request.Model, "-thinking") {
+ if request.MaxTokens == 0 {
+ request.MaxTokens = uint(model_setting.GetClaudeSettings().GetDefaultMaxTokens(request.Model))
+ }
+ if model_setting.GetClaudeSettings().ThinkingAdapterEnabled && thinkingVariant {
if request.Thinking == nil {
// 因为BudgetTokens 必须大于1024
if request.MaxTokens < 1280 {
request.MaxTokens = 1280
}
// BudgetTokens 为 max_tokens 的 80%
request.Thinking = &dto.Thinking{
Type: "enabled",
BudgetTokens: common.GetPointer[int](int(float64(request.MaxTokens) * model_setting.GetClaudeSettings().ThinkingAdapterBudgetTokensPercentage)),
}
// TODO: 临时处理
// https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#important-considerations-when-using-extended-thinking
request.TopP = 0
request.Temperature = common.GetPointer[float64](1.0)
}
- request.Model = strings.TrimSuffix(request.Model, "-thinking")
- info.UpstreamModelName = request.Model
}Also applies to: 50-70
🤖 Prompt for AI Agents
In relay/claude_handler.go around lines 35-38 (and similarly lines ~50-70), the
code currently calls ModelMappedHelper and then trims "-thinking" and overwrites
info.UpstreamModelName which can break mappings and desync request vs info;
instead, normalize the incoming model name by trimming any "-thinking" suffix on
the request/model variable before calling ModelMappedHelper so mapping can set
info.UpstreamModelName correctly, do not overwrite info.UpstreamModelName after
ModelMappedHelper returns, and if you need to apply "thinking" defaults do so
afterwards by adjusting only the request/behavioral defaults (not
info.UpstreamModelName) so mapping remains authoritative.
…rank fix: retry requeset body incorrect and fix rerank
…rank fix: retry requeset body incorrect and fix rerank
…rank fix: retry requeset body incorrect and fix rerank
#1608
Summary by CodeRabbit
New Features
Refactor
Bug Fixes
Chores