Skip to content

fix: retry requeset body incorrect and fix rerank - #1642

Merged
Calcium-Ion merged 4 commits into
alphafrom
fix-retry-and-rerank
Aug 23, 2025
Merged

Calcium-Ion merged 4 commits into
alphafrom
fix-retry-and-rerank

Conversation

@Calcium-Ion

@Calcium-Ion Calcium-Ion commented Aug 23, 2025 •

Copy link
Copy Markdown
Member

#1608

Summary by CodeRabbit

  • New Features

    • Expanded image generation options: moderation, output format/compression, partial images, and watermark support.
    • New public SetModelName setters across request types for consistent model propagation.
    • Gemini embedding token metadata (single & batch); embedding endpoints non-streaming.
  • Refactor

    • Unified, more accurate model routing with cycle detection and centralized model propagation.
    • Introduced deep-copy usage so handlers operate on mutable copies of incoming requests.
  • Bug Fixes

    • Prevents unintended mutations of original requests during processing; improved error handling for copy failures.
  • Chores

    • Updated dependencies.

@coderabbitai

coderabbitai Bot commented Aug 23, 2025 •

Copy link
Copy Markdown
Contributor

Walkthrough

Adds a generic DeepCopy utility and adopts it across relay handlers to operate on copied request objects. Extends dto.Request with SetModelName and implements setters across DTOs. Refactors model mapping to a centralized helper using dto.Request. Expands ImageRequest fields and adds indirect dependencies.

Changes

Cohort / File(s) Summary
Deep-copy utility
common/copy.go
Adds generic DeepCopy[T any](src *T) (*T, error) using antlabs/pcopy with nil checks and error returns.
Request interface & DTO SetModelName
dto/request_common.go, dto/audio.go, dto/claude.go, dto/embedding.go, dto/openai_request.go, dto/rerank.go, dto/gemini.go, dto/openai_image.go
Adds SetModelName(modelName string) to Request and implements it on BaseRequest and many DTOs. Adds Gemini embedding helpers (IsStream, GetTokenCountMeta) and expands ImageRequest with new fields (Moderation, OutputFormat, OutputCompression, PartialImages, Watermark). Minor receiver/name tweaks.
Relay handlers — deep-copy adoption
relay/audio_handler.go, relay/claude_handler.go, relay/compatible_handler.go, relay/embedding_handler.go, relay/gemini_handler.go, relay/image_handler.go, relay/rerank_handler.go, relay/responses_handler.go
Each handler now asserts the concrete DTO, performs common.DeepCopy, handles copy errors (InvalidRequest / SkipRetry), and uses the copied request for ModelMappedHelper, conversion, and downstream logic; some handlers apply request-specific defaults/mutations on the copy.
Model mapping refactor
relay/helper/model_mapped.go
Signature changed to accept dto.Request; replaces per-format switch with JSON chain-based mapping, cycle detection, sets info.UpstreamModelName/IsModelMapped, and calls request.SetModelName(...) when provided.
Relay info propagation
relay/common/relay_info.go
After InitChannelMeta, calls info.Request.SetModelName(info.OriginModelName) when info.Request is non-nil.
Go module deps
go.mod
Adds indirect dependencies including antlabs/pcopy, mitchellh/copystructure, mitchellh/reflectwalk, imdario/mergo, Masterminds/*, huandu/xstrings, spf13/cast.
Minor import reorder
common/gin.go
Reordered imports; no functional change.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Client
  participant Handler as Relay Handler
  participant Copier as common.DeepCopy
  participant Mapper as ModelMappedHelper
  participant Req as dto.Request
  participant Upstream as Adaptor/Upstream

  Client->>Handler: Incoming request (concrete DTO)
  Handler->>Copier: DeepCopy(original DTO)
  alt copy ok
    Copier-->>Handler: copied DTO (request)
    Handler->>Mapper: ModelMappedHelper(c, info, request)
    Mapper->>Req: request.SetModelName(info.UpstreamModelName)
    Mapper-->>Handler: mapping result
    Handler->>Upstream: Convert & forward (*request)
    Upstream-->>Handler: Response
    Handler-->>Client: Response
  else copy error
    Copier-->>Handler: error
    Handler-->>Client: API error (InvalidRequest, SkipRetry)
  end
Loading
sequenceDiagram
  autonumber
  participant Mapper as ModelMappedHelper
  participant Config as model_mapping JSON
  participant Info as RelayInfo
  participant Req as dto.Request

  Mapper->>Config: Load mapping map
  Mapper->>Mapper: Resolve chain from OriginModelName (detect cycles)
  alt cycle detected
    Mapper-->>Info: return model_mapping_contains_cycle
  else resolved
    Mapper->>Info: Set UpstreamModelName, IsModelMapped
    opt request provided
      Mapper->>Req: SetModelName(UpstreamModelName)
    end
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested reviewers

  • seefs001

Poem

I hop through models, name in paw,
I copy, not mutate — that is the law.
I map the chain and dodge a loop,
Embed and image find their troupe.
With tiny paws I set the name — upstream we hop! 🐇✨

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix-retry-and-rerank

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (4)
go.mod (1)

3-4: Fix invalid Go version directives and align toolchain hint

The go directive must be major.minor only; go 1.23.4 is invalid and can break tooling. Also, the Heroku hint pins go1.18, which conflicts with module go 1.23. Align both.

Apply one of these diffs:

Option A — simplest, bump Heroku hint and fix go directive:

-// +heroku goVersion go1.18
+// +heroku goVersion go1.23
-go 1.23.4
+go 1.23

Option B — if you want to pin a patch toolchain (Go 1.22+ supports this):

-// +heroku goVersion go1.18
+// +heroku goVersion go1.23
-go 1.23.4
+go 1.23
+toolchain go1.23.4

Note: Keep the go directive at 1.23; use toolchain if patch pinning is required. If Heroku doesn’t consume the comment anymore, drop it instead.

relay/gemini_handler.go (2)

162-166: Possible panic: unsafe type assertion on resp without nil guard

resp.(*http.Response) will panic if an adaptor ever returns nil resp with nil error. Other handlers pass httpResp, which can be nil safely. Mirror that here.

Apply this diff:

-    usage, openaiErr := adaptor.DoResponse(c, resp.(*http.Response), info)
+    usage, openaiErr := adaptor.DoResponse(c, httpResp, info)

258-262: *Repeat the nil-guard fix for embedding: avoid resp.(http.Response)

Same potential panic exists here; pass the httpResp variable instead.

Apply this diff:

-    usage, openaiErr := adaptor.DoResponse(c, resp.(*http.Response), info)
+    usage, openaiErr := adaptor.DoResponse(c, httpResp, info)
relay/compatible_handler.go (1)

52-66: IncludeUsage may desync from ForceStreamOption; compute after stream-options normalization.

If the request supplied StreamOptions.IncludeUsage=false and ForceStreamOption=true, includeUsage remains false, diverging from the forced true.

Apply this diff to recalculate includeUsage after the StreamOptions block:

-	includeUsage := true
-	// 判断用户是否需要返回使用情况
-	if request.StreamOptions != nil {
-		includeUsage = request.StreamOptions.IncludeUsage
-	}
+	includeUsage := true
@@
-	if !info.SupportStreamOptions || !request.Stream {
-		request.StreamOptions = nil
-	} else {
-		// 如果支持StreamOptions,且请求中没有设置StreamOptions,根据配置文件设置StreamOptions
-		if constant.ForceStreamOption {
-			request.StreamOptions = &dto.StreamOptions{
-				IncludeUsage: true,
-			}
-		}
-	}
+	if !info.SupportStreamOptions || !request.Stream {
+		request.StreamOptions = nil
+	} else if constant.ForceStreamOption {
+		// 如果支持StreamOptions,且请求中没有设置StreamOptions,根据配置文件设置StreamOptions
+		request.StreamOptions = &dto.StreamOptions{IncludeUsage: true}
+	}
+	// 归一化后再计算 includeUsage
+	if request.StreamOptions != nil {
+		includeUsage = request.StreamOptions.IncludeUsage
+	} else {
+		includeUsage = true
+	}
🧹 Nitpick comments (16)
go.mod (1)

47-51: Remove manually pinned indirect dependencies; let Go derive them

  • The only package directly imported by your code is github.com/antlabs/pcopy (in common/copy.go). All other entries (copystructure, reflectwalk, sprig/v3, semver/v3, goutils, xstrings, mergo, cast) aren’t directly referenced and will be pulled in transitively by go mod tidy.
  • Change the pcopy line in go.mod to a regular (direct) requirement (remove the // indirect comment).
  • Delete the other eight manually listed indirect dependencies (lines 47–51, 76–78, 88–91, 95).
  • Run:
    go mod tidy -v
    to let Go recompute indirect modules automatically.
  • (Optional) After tidying, check if there are newer patch/minor releases for github.com/antlabs/pcopy and other direct deps, and bump them as appropriate.
dto/embedding.go (1)

51-55: EmbeddingRequest model setter is straightforward and safe

Matches the interface and respects the empty-model no-op convention. No functional issues.

Minor future tidy: elsewhere in this file, type switches re-assert the same types (e.g., r.Input.(string) inside a switch r.Input.(type)); you can bind to a variable in the switch header to avoid repeated assertions. Not required for this PR.

dto/openai_request.go (2)

186-190: SetModelName: trim whitespace; also note a likely field typo

  • Suggest trimming whitespace before checking emptiness to avoid setting a whitespace-only model name.
  • Minor nit: field name "ParallelTooCalls" (Line 51) looks misspelled; consider renaming to "ParallelToolCalls" in a follow-up that coordinates downstream references.

Apply within the selected range:

 func (r *GeneralOpenAIRequest) SetModelName(modelName string) {
-	if modelName != "" {
-		r.Model = modelName
+	if s := strings.TrimSpace(modelName); s != "" {
+		r.Model = s
 	}
 }

Also applies to: 51-52


850-854: SetModelName: trim whitespace to avoid setting invalid model names

Same rationale as above; keep behavior consistent across DTOs.

 func (r *OpenAIResponsesRequest) SetModelName(modelName string) {
-	if modelName != "" {
-		r.Model = modelName
+	if s := strings.TrimSpace(modelName); s != "" {
+		r.Model = s
 	}
 }
common/copy.go (1)

17-19: Remove impossible nil check on address of local variable

&dst can never be nil in Go; this branch is dead code and should be removed.

-	if &dst == nil {
-		return nil, fmt.Errorf("copy result cannot be nil")
-	}

If helpful, I can add a table-driven unit test to validate deep copy semantics (structs with maps/slices/pointers) and ensure mutations to source don’t affect the copy. Do you want me to draft it?

dto/openai_image.go (1)

66-70: SetModelName: trim whitespace

Prevent propagating accidental whitespace-only model names.

 func (i *ImageRequest) SetModelName(modelName string) {
-	if modelName != "" {
-		i.Model = modelName
+	if s := strings.TrimSpace(modelName); s != "" {
+		i.Model = s
 	}
 }
dto/gemini.go (2)

324-335: Token count meta: good coverage; optional enhancement

Current logic aggregates text parts. Optional: include Title (when present) to better reflect embedding inputs for search-oriented tasks.

 func (r *GeminiEmbeddingRequest) GetTokenCountMeta() *types.TokenCountMeta {
 	var inputTexts []string
 	for _, part := range r.Content.Parts {
 		if part.Text != "" {
 			inputTexts = append(inputTexts, part.Text)
 		}
 	}
+	if r.Title != "" {
+		inputTexts = append(inputTexts, r.Title)
+	}
 	inputText := strings.Join(inputTexts, "\n")
 	return &types.TokenCountMeta{
 		CombineText: inputText,
 	}
 }

Also applies to: 352-364


337-341: SetModelName: trim whitespace; batch propagation looks good

  • Trim whitespace before assignment for consistency.
  • Batch propagation over Requests is correct and aligns with the requirement that each item carries its own model.
 func (r *GeminiEmbeddingRequest) SetModelName(modelName string) {
-	if modelName != "" {
-		r.Model = modelName
+	if s := strings.TrimSpace(modelName); s != "" {
+		r.Model = s
 	}
 }

Also applies to: 366-372

dto/audio.go (1)

29-33: SetModelName: trim whitespace; add import

Trim whitespace before setting to avoid invalid model names. This file doesn’t currently import strings; add it.

 func (r *AudioRequest) SetModelName(modelName string) {
-	if modelName != "" {
-		r.Model = modelName
+	if s := strings.TrimSpace(modelName); s != "" {
+		r.Model = s
 	}
 }

Outside the selected range (add missing import):

import (
    "one-api/types"
    "github.com/gin-gonic/gin"
    "strings" // add
)
relay/responses_handler.go (1)

28-31: Fix error message: wrong type name in copy failure

Message says “GeneralOpenAIRequest” but we’re copying an OpenAIResponsesRequest. Keep error text accurate.

Apply this diff:

-    return types.NewError(fmt.Errorf("failed to copy request to GeneralOpenAIRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry())
+    return types.NewError(fmt.Errorf("failed to copy OpenAIResponsesRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry())
relay/audio_handler.go (1)

20-23: Make the error message more specific (optional)

Include the expected and actual types to aid diagnostics; optionally set status 400.

Apply this diff:

-    return types.NewError(errors.New("invalid request type"), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry())
+    return types.NewErrorWithStatusCode(
+        fmt.Errorf("invalid request type: expected *dto.AudioRequest, got %T", info.Request),
+        types.ErrorCodeInvalidRequest,
+        http.StatusBadRequest,
+        types.ErrOptionWithSkipRetry(),
+    )
relay/gemini_handler.go (1)

72-88: Optional: keep request’s model in sync after thinking-adapter retargeting

After adjusting info.OriginModelName/info.UpstreamModelName to a “-nothinking” variant, call request.SetModelName(info.UpstreamModelName) to keep the request object consistent for adaptors that may consult it.

Proposed snippet (outside the shown range where you switch the names):

// After updating info.(Origin|Upstream)ModelName
if request != nil {
    request.SetModelName(info.UpstreamModelName)
}
relay/rerank_handler.go (1)

27-30: Fix incorrect error message: refers to ImageRequest in rerank handler.

The returned error says “copy request to ImageRequest” but this is a rerank path, which will confuse operators and users.

Apply this diff:

-	request, err := common.DeepCopy(rerankReq)
-	if err != nil {
-		return types.NewError(fmt.Errorf("failed to copy request to ImageRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry())
-	}
+	request, err := common.DeepCopy(rerankReq)
+	if err != nil {
+		return types.NewError(fmt.Errorf("failed to copy request to RerankRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry())
+	}
relay/image_handler.go (1)

62-66: Use common.Marshal for consistency with other handlers.

Other handlers use common.Marshal (centralized JSON settings). Keep this consistent to avoid subtle encoding differences.

Apply this diff:

-			jsonData, err := json.Marshal(convertedRequest)
+			jsonData, err := common.Marshal(convertedRequest)

Additionally remove the now-unused import:

-import (
-	"bytes"
-	"encoding/json"
+import (
+	"bytes"
relay/claude_handler.go (1)

50-70: Thinking adapter: minor robustness and rounding nit.

  • Consider rounding BudgetTokens instead of truncation to avoid off-by-one under-allocation on percentages.
  • The suffix trim and assignment look good; they operate after model mapping and only when the effective upstream model still has “-thinking”.

Apply this diff:

-			request.Thinking = &dto.Thinking{
-				Type:         "enabled",
-				BudgetTokens: common.GetPointer[int](int(float64(request.MaxTokens) * model_setting.GetClaudeSettings().ThinkingAdapterBudgetTokensPercentage)),
-			}
+			request.Thinking = &dto.Thinking{
+				Type: "enabled",
+				BudgetTokens: func() *int {
+					bt := int(math.Round(float64(request.MaxTokens) *
+						model_setting.GetClaudeSettings().ThinkingAdapterBudgetTokensPercentage))
+					return &bt
+				}(),
+			}

Note: you’ll need to import math.

relay/helper/model_mapped.go (1)

18-20: Use errors.New instead of fmt.Errorf for constant strings; drop fmt import.

Minor cleanup to avoid unnecessary import and align with Go idioms.

Apply this diff:

-		err := json.Unmarshal([]byte(modelMapping), &modelMap)
-		if err != nil {
-			return fmt.Errorf("unmarshal_model_mapping_failed")
-		}
+		if err := json.Unmarshal([]byte(modelMapping), &modelMap); err != nil {
+			return errors.New("unmarshal_model_mapping_failed")
+		}

And adjust imports:

-import (
-	"encoding/json"
-	"errors"
-	"fmt"
-	"github.com/gin-gonic/gin"
-	"one-api/dto"
-	"one-api/relay/common"
-)
+import (
+	"encoding/json"
+	"errors"
+	"github.com/gin-gonic/gin"
+	"one-api/dto"
+	"one-api/relay/common"
+)
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 005e965 and 2c6a924.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (20)
  • common/copy.go (1 hunks)
  • dto/audio.go (1 hunks)
  • dto/claude.go (1 hunks)
  • dto/embedding.go (1 hunks)
  • dto/gemini.go (2 hunks)
  • dto/openai_image.go (2 hunks)
  • dto/openai_request.go (2 hunks)
  • dto/request_common.go (2 hunks)
  • dto/rerank.go (1 hunks)
  • go.mod (3 hunks)
  • relay/audio_handler.go (3 hunks)
  • relay/claude_handler.go (3 hunks)
  • relay/common/relay_info.go (1 hunks)
  • relay/compatible_handler.go (2 hunks)
  • relay/embedding_handler.go (2 hunks)
  • relay/gemini_handler.go (2 hunks)
  • relay/helper/model_mapped.go (2 hunks)
  • relay/image_handler.go (3 hunks)
  • relay/rerank_handler.go (2 hunks)
  • relay/responses_handler.go (1 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: 2025-08-05T17:14:17.246Z
Learnt from: neotf
PR: QuantumNous/new-api#1511
File: setting/ratio_setting/model_ratio.go:118-123
Timestamp: 2025-08-05T17:14:17.246Z
Learning: Claude models handle "-thinking" variants differently from Gemini models. For Claude models, only the base model (without "-thinking") gets an entry in defaultModelRatio map. The "-thinking" variants rely on the Claude relay handler stripping the suffix using strings.TrimSuffix(textRequest.Model, "-thinking") before looking up the ratio, so they automatically use the base model's ratio.

Applied to files:

  • relay/claude_handler.go
📚 Learning: 2025-08-08T17:12:43.157Z
Learnt from: RedwindA
PR: QuantumNous/new-api#1537
File: relay/gemini_handler.go:330-342
Timestamp: 2025-08-08T17:12:43.157Z
Learning: In the new-api repository, the `GeminiEmbeddingHandler` function in `relay/gemini_handler.go` is designed specifically for native Gemini embedding requests and therefore does not require the `ConvertGeminiRequest` step that is used in the chat handler. The embedding requests are already in the native Gemini format and don't need conversion.

Applied to files:

  • dto/gemini.go
  • relay/embedding_handler.go
  • relay/gemini_handler.go
📚 Learning: 2025-08-04T14:36:34.528Z
Learnt from: antecanis8
PR: QuantumNous/new-api#1500
File: relay/channel/gemini/adaptor.go:0-0
Timestamp: 2025-08-04T14:36:34.528Z
Learning: In the Gemini API batchEmbedContents endpoint, each request item in the "requests" array should include its own "model" field, as shown in the official documentation at https://ai.google.dev/api/embeddings?hl=zh-cn#method:-models.batchembedcontents. The model field is not a top-level parameter.

Applied to files:

  • dto/gemini.go
🧬 Code graph analysis (14)
dto/openai_request.go (1)
model/model_meta.go (1)
  • Model (22-42)
dto/rerank.go (1)
model/model_meta.go (1)
  • Model (22-42)
relay/rerank_handler.go (7)
dto/request_common.go (1)
  • Request (8-12)
dto/rerank.go (1)
  • RerankRequest (10-18)
common/sys_log.go (1)
  • FatalLog (20-24)
common/copy.go (1)
  • DeepCopy (8-21)
types/error.go (3)
  • NewError (181-193)
  • ErrorCodeInvalidRequest (38-38)
  • ErrOptionWithSkipRetry (290-294)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
relay/channel/ali/rerank.go (1)
  • ConvertRerankRequest (15-32)
relay/claude_handler.go (8)
dto/request_common.go (1)
  • Request (8-12)
dto/claude.go (2)
  • ClaudeRequest (187-203)
  • Thinking (399-402)
common/sys_log.go (1)
  • FatalLog (20-24)
common/copy.go (1)
  • DeepCopy (8-21)
types/error.go (3)
  • NewError (181-193)
  • ErrorCodeInvalidRequest (38-38)
  • ErrOptionWithSkipRetry (290-294)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
setting/model_setting/claude.go (1)
  • GetClaudeSettings (41-47)
common/utils.go (1)
  • GetPointer (218-220)
relay/common/relay_info.go (1)
dto/request_common.go (1)
  • Request (8-12)
dto/gemini.go (1)
types/request_meta.go (1)
  • TokenCountMeta (20-31)
relay/audio_handler.go (5)
dto/request_common.go (1)
  • Request (8-12)
dto/audio.go (1)
  • AudioRequest (9-15)
types/error.go (2)
  • NewError (181-193)
  • ErrOptionWithSkipRetry (290-294)
common/copy.go (1)
  • DeepCopy (8-21)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
relay/responses_handler.go (6)
dto/request_common.go (1)
  • Request (8-12)
dto/openai_request.go (1)
  • OpenAIResponsesRequest (762-784)
common/sys_log.go (1)
  • FatalLog (20-24)
common/copy.go (1)
  • DeepCopy (8-21)
types/error.go (3)
  • NewError (181-193)
  • ErrorCodeInvalidRequest (38-38)
  • ErrOptionWithSkipRetry (290-294)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
relay/embedding_handler.go (6)
dto/request_common.go (1)
  • Request (8-12)
dto/embedding.go (1)
  • EmbeddingRequest (21-32)
common/sys_log.go (1)
  • FatalLog (20-24)
common/copy.go (1)
  • DeepCopy (8-21)
types/error.go (3)
  • NewError (181-193)
  • ErrorCodeInvalidRequest (38-38)
  • ErrOptionWithSkipRetry (290-294)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
relay/image_handler.go (6)
dto/request_common.go (1)
  • Request (8-12)
dto/openai_image.go (1)
  • ImageRequest (11-28)
common/sys_log.go (1)
  • FatalLog (20-24)
common/copy.go (1)
  • DeepCopy (8-21)
types/error.go (3)
  • NewError (181-193)
  • ErrorCodeInvalidRequest (38-38)
  • ErrOptionWithSkipRetry (290-294)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
relay/helper/model_mapped.go (2)
relay/common/relay_info.go (1)
  • RelayInfo (73-118)
dto/request_common.go (1)
  • Request (8-12)
relay/gemini_handler.go (6)
dto/request_common.go (1)
  • Request (8-12)
dto/gemini.go (1)
  • GeminiChatRequest (12-18)
common/sys_log.go (1)
  • FatalLog (20-24)
common/copy.go (1)
  • DeepCopy (8-21)
types/error.go (3)
  • NewError (181-193)
  • ErrorCodeInvalidRequest (38-38)
  • ErrOptionWithSkipRetry (290-294)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
relay/compatible_handler.go (7)
dto/request_common.go (1)
  • Request (8-12)
dto/openai_request.go (3)
  • GeneralOpenAIRequest (25-72)
  • WebSearchOptions (756-759)
  • StreamOptions (223-225)
common/sys_log.go (1)
  • FatalLog (20-24)
common/copy.go (1)
  • DeepCopy (8-21)
types/error.go (4)
  • NewError (181-193)
  • ErrorCodeInvalidRequest (38-38)
  • ErrOptionWithSkipRetry (290-294)
  • ErrorCodeChannelModelMappedError (53-53)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
constant/env.go (1)
  • ForceStreamOption (6-6)
dto/openai_image.go (1)
dto/openai_request.go (1)
  • ResponseFormat (13-16)
🔇 Additional comments (25)
dto/request_common.go (1)

11-11: Manual Verification Required: Confirm SetModelName on All Request Types

The automated checks encountered parsing issues and did not reliably report which DTOs implement SetModelName or embed BaseRequest. Please manually verify that every *Request struct in your codebase either:

  • Defines an explicit
    func (r *XRequest) SetModelName(modelName string)
    method, or
  • Embeds BaseRequest (which provides the no-op default setter).

Key places to inspect:

  • All ...Request types under dto/ (or wherever your request DTOs live).
  • Ensure none of your request structs are left without a setter or embedded default.

Once confirmed, consider adding compile-time assertions to guard against future omissions, for example:

var _ Request = (*YourRequestType)(nil)
dto/rerank.go (1)

40-44: Setter aligns rerank model propagation with the rest of DTOs

The SetModelName implementation is consistent with other DTOs and guards on empty strings.

Add/extend tests to cover:

  • Incoming request with empty Model and non-empty propagated model.
  • Pre-set Model not overridden when modelName == "".
  • Retry path with DeepCopy uses the copied request, not the original (to avoid mutating the caller during retries).
dto/claude.go (2)

324-326: Param rename avoids receiver shadowing; returns the intended stream flag

LGTM. Returning c.Stream is the correct behavior here.


328-331: ClaudeRequest gains SetModelName; consistent with DTO interface

No issues. The empty-string guard mirrors other DTOs.

dto/gemini.go (2)

76-79: No-op SetModelName is correct for GeminiChatRequest

Gemini chat payload has no top-level model field; a no-op implementation is appropriate and satisfies the new interface.


319-323: IsStream correctly returns false for embeddings

Non-streaming semantics for both single and batch embeddings are correct.

Also applies to: 347-351

relay/common/relay_info.go (1)

166-168: Good: align request’s model to the origin early

Setting the request’s model to OriginModelName up front reduces surprises downstream and makes DeepCopy-based flows consistent.

relay/responses_handler.go (2)

33-34: LGTM: map model on the copied request

Switching ModelMappedHelper to operate on the copied request avoids mutating the original input.


51-51: LGTM: pass the copied request to the adaptor

Passing *request maintains the copy-only mutation contract established earlier.

relay/embedding_handler.go (2)

26-31: LGTM: copy-then-map flow is correct

Deep-copying first, then model-mapping via the dto.Request interface is consistent with the PR-wide pattern and keeps the original request immutable.


42-42: LGTM: use the copied request in conversion

Using *request here ensures downstream mutations don’t touch the original object.

relay/audio_handler.go (2)

20-23: Good change: return a typed error instead of crashing

Switching to types.NewError(..., invalid_request, SkipRetry) on type mismatch is the right behavior for a server.


25-31: LGTM: adopt DeepCopy + model mapping on the copy

This mirrors the pattern in other handlers and avoids mutating info.Request.

relay/gemini_handler.go (2)

61-68: LGTM: copy before mapping; operate on the copy

This aligns with the new DeepCopy-first contract and keeps the original immutable.


178-179: LGTM: tighten type to dto.Request for embedding handler

Using dto.Request instead of any improves type-safety and integrates with SetModelName.

relay/rerank_handler.go (2)

32-33: Typed ModelMappedHelper usage looks correct.

Passing the copied *dto.RerankRequest as dto.Request ensures model mapping updates propagate via SetModelName without mutating the original payload.


51-52: ConvertRerankRequest signature already correct

All ConvertRerankRequest methods in the Adaptor interface and every channel implementation take a dto.RerankRequest by value, and the handler correctly dereferences the pointer (*request) to pass that value. No changes are needed to avoid build breaks.

relay/image_handler.go (2)

30-36: DeepCopy + invalid_request on copy failure — good.

This addresses the “retry request body incorrect” class of bugs by preventing in-place mutations of the original DTO.


107-112: Token and log fields now reference the copied request — LGTM.

Using request.N, request.Quality, and request.Size maintains internal consistency post-copy.

Also applies to: 115-117, 121-123

relay/claude_handler.go (3)

30-36: DeepCopy path and error classification are correct.

Returning invalid_request with SkipRetry on copy failure matches the PR’s intent and avoids retry loops.


46-48: Default MaxTokens fallback is reasonable.

Deriving defaults from model settings when omitted is aligned with existing patterns.


80-81: *All ConvertClaudeRequest signatures match dto.ClaudeRequest – no action needed

I’ve checked every ConvertClaudeRequest implementation under relay/channel/*/adaptor.go, and they all use the exact signature:

func (a *Adaptor) ConvertClaudeRequest(
    *gin.Context,
    *relaycommon.RelayInfo,
    *dto.ClaudeRequest,
) (any, error)

so there is no risk of a signature mismatch when passing a *dto.ClaudeRequest. The call site in relay/claude_handler.go already passes request (which is declared as a *dto.ClaudeRequest), so everything lines up correctly.

relay/compatible_handler.go (2)

36-40: DeepCopy + typed ModelMappedHelper usage — LGTM.

The flow isolates mutations to the copied request and centralizes model mapping, which should address retry-body mutation issues.

Also applies to: 45-48


87-88: I’ve broadened the search to locate both the definition and all usages of ConvertOpenAIRequest and references to GeneralOpenAIRequest across the repo. Once we have those results, we can confirm if the signature changed to a pointer and whether all adaptors align with it.

relay/helper/model_mapped.go (1)

12-12: Typed dto.Request improves safety and simplifies call sites.

The new signature removes brittle type switches and standardizes model propagation via SetModelName.

Comment thread relay/claude_handler.go Resolved
Comment on lines +161 to +168

info.ChannelMeta = channelMeta

// reset some fields based on channel meta
// 重置某些字段,例如模型名称等
if info.Request != nil {
info.Request.SetModelName(info.OriginModelName)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Prevent blanking model name during mapping: initialize UpstreamModelName before SetModelName

If no model-mapping is provided, info.UpstreamModelName remains empty. Later, ModelMappedHelper unconditionally does request.SetModelName(info.UpstreamModelName), which will overwrite the request’s model with an empty string. Initialize info.UpstreamModelName to info.OriginModelName here to make the default safe.

Apply this diff:

 info.ChannelMeta = channelMeta

+ // initialize upstream model to origin by default to avoid empty overwrite later
+ info.UpstreamModelName = info.OriginModelName
+
 // reset some fields based on channel meta
 // 重置某些字段,例如模型名称等
 if info.Request != nil {
     info.Request.SetModelName(info.OriginModelName)
 }

Additionally, harden the mapper to avoid writing empty names when no mapping occurs (change in another file, shown for completeness):

--- a/relay/helper/model_mapped.go
+++ b/relay/helper/model_mapped.go
@@
-   if request != nil {
-       request.SetModelName(info.UpstreamModelName)
-   }
+   if request != nil {
+       target := info.UpstreamModelName
+       if target == "" {
+           target = info.OriginModelName
+       }
+       request.SetModelName(target)
+   }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
info.ChannelMeta = channelMeta
// reset some fields based on channel meta
// 重置某些字段,例如模型名称等
if info.Request != nil {
info.Request.SetModelName(info.OriginModelName)
}
info.ChannelMeta = channelMeta
// initialize upstream model to origin by default to avoid empty overwrite later
info.UpstreamModelName = info.OriginModelName
// reset some fields based on channel meta
// 重置某些字段,例如模型名称等
if info.Request != nil {
info.Request.SetModelName(info.OriginModelName)
}
🤖 Prompt for AI Agents
In relay/common/relay_info.go around lines 161 to 168, info.UpstreamModelName is
left empty when no mapping exists which allows later mapping logic to overwrite
the request's model with an empty string; initialize info.UpstreamModelName =
info.OriginModelName before calling request.SetModelName(info.OriginModelName)
so the default is safe, and also update the mapper to only call
request.SetModelName(...) when the mapped name is non-empty to prevent blanking
when no mapping occurs.

Comment thread relay/compatible_handler.go Resolved
Comment thread relay/embedding_handler.go Resolved
Comment thread relay/gemini_handler.go Resolved
Comment on lines +54 to 55
request.SetModelName(info.UpstreamModelName)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Ensure UpstreamModelName is set before calling SetModelName.

If InitChannelMeta didn’t set a default, this could blank the request’s model. Please confirm info.UpstreamModelName is initialized to info.OriginModelName during InitChannelMeta.

Run:


🏁 Script executed:

#!/bin/bash
# Check that InitChannelMeta initializes UpstreamModelName
rg -n 'func\s+InitChannelMeta\(' -n
rg -n 'UpstreamModelName\s*=' -n relay/common/relay_info.go -C3

Length of output: 103


I couldn’t find any evidence that InitChannelMeta sets UpstreamModelName to OriginModelName. Let’s search the whole repo to be sure:


🏁 Script executed:

#!/bin/bash
# Search for InitChannelMeta definition
rg -n 'func InitChannelMeta' -C3
# Look for any assignment to UpstreamModelName
rg -n 'UpstreamModelName\s*=' -C3
# Check for usage of OriginModelName as a fallback
rg -n 'OriginModelName' -C3

Length of output: 20579


Initialize UpstreamModelName before mapping to avoid blank models

In relay/helper/model_mapped.go, the final block reads:

if request != nil {
    request.SetModelName(info.UpstreamModelName)
}

However, if info.IsModelMapped is false, info.UpstreamModelName was never set and this will clear out the request’s model. There is no InitChannelMeta function in this repo—by default only info.Request.SetModelName(info.OriginModelName) is called in relay/common/relay_info.go, and that does not initialize UpstreamModelName.

Please add an explicit default assignment at the top of the mapping function so UpstreamModelName always falls back to OriginModelName:

 func applyModelMapping(info *relaycommon.RelayInfo, request *YourRequestType) {
+    // ensure we have a valid default before any remapping
+    info.UpstreamModelName = info.OriginModelName

     currentModel := info.OriginModelName
     visitedModels := map[string]bool{ currentModel: true }
     // … existing mapping logic …
 
     if request != nil {
         request.SetModelName(info.UpstreamModelName)
     }
 }
🤖 Prompt for AI Agents
In relay/helper/model_mapped.go around lines 54-55, ensure
info.UpstreamModelName is initialized before it's applied to the request: at the
top of the mapping function set a default so UpstreamModelName falls back to
OriginModelName (e.g. if info.UpstreamModelName is empty, assign
info.OriginModelName) so the subsequent
request.SetModelName(info.UpstreamModelName) cannot clear the request's model
when no mapping was provided.

Comment thread relay/image_handler.go Resolved
Comment thread relay/rerank_handler.go Resolved
Comment thread relay/responses_handler.go Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
relay/compatible_handler.go (1)

49-67: Fix potential mismatch: ForceStreamOption can flip IncludeUsage but info.ShouldIncludeUsage stays stale

If ForceStreamOption sets IncludeUsage to true while the original StreamOptions had IncludeUsage=false, info.ShouldIncludeUsage remains false because it’s computed before the override. Recompute after the StreamOptions block.

-    includeUsage := true
-    // 判断用户是否需要返回使用情况
-    if request.StreamOptions != nil {
-        includeUsage = request.StreamOptions.IncludeUsage
-    }
-
     // 如果不支持StreamOptions,将StreamOptions设置为nil
     if !info.SupportStreamOptions || !request.Stream {
         request.StreamOptions = nil
     } else {
         // 如果支持StreamOptions,且请求中没有设置StreamOptions,根据配置文件设置StreamOptions
         if constant.ForceStreamOption {
             request.StreamOptions = &dto.StreamOptions{
                 IncludeUsage: true,
             }
         }
     }
 
-    info.ShouldIncludeUsage = includeUsage
+    // 统一在覆盖后计算 IncludeUsage
+    includeUsage := true
+    if request.StreamOptions != nil {
+        includeUsage = request.StreamOptions.IncludeUsage
+    }
+    info.ShouldIncludeUsage = includeUsage
♻️ Duplicate comments (3)
relay/responses_handler.go (1)

23-26: Good fix: return 400 on bad type assertion instead of exiting the process

This aligns with earlier guidance to replace FatalLog/os.Exit with a typed API error. Message includes the actual type.

relay/compatible_handler.go (1)

30-33: Nice: bad type assertion now returns a 400 NewAPIError

This matches the pattern adopted across handlers and prevents process termination on user input errors.

relay/claude_handler.go (1)

24-28: Good: 400 on bad type assertion with SkipRetry

This removes the FatalLog crash path and returns a proper API error.

🧹 Nitpick comments (6)
relay/responses_handler.go (3)

28-31: Fix incorrect error message: copy target type is wrong

The message says “GeneralOpenAIRequest” while the code is copying an OpenAIResponsesRequest. Correct to avoid confusion during ops/debugging.

-    return types.NewError(fmt.Errorf("failed to copy request to GeneralOpenAIRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry())
+    return types.NewError(fmt.Errorf("failed to copy OpenAIResponsesRequest: %w", err), types.ErrorCodeInvalidRequest, types.ErrOptionWithSkipRetry())

44-51: Honor channel-level pass-through flag for responses handler

TextHelper supports both global and channel-level pass-through; ResponsesHelper only checks the global setting. Align behavior to avoid surprises per-channel.

-    if model_setting.GetGlobalSettings().PassThroughRequestEnabled {
+    if model_setting.GetGlobalSettings().PassThroughRequestEnabled || info.ChannelSetting.PassThroughBodyEnabled {
         body, err := common.GetRequestBody(c)
         if err != nil {
-            return types.NewError(err, types.ErrorCodeReadRequestBodyFailed, types.ErrOptionWithSkipRetry())
+            return types.NewErrorWithStatusCode(err, types.ErrorCodeReadRequestBodyFailed, http.StatusBadRequest, types.ErrOptionWithSkipRetry())
         }
         requestBody = bytes.NewBuffer(body)

67-71: Use structured debug logging instead of println and keep consistency with TextHelper

Prefer logger.LogDebug for consistency and to integrate with log levels/fields.

 import (
     "bytes"
     "fmt"
     "io"
     "net/http"
     "one-api/common"
     "one-api/dto"
+    "one-api/logger"
     relaycommon "one-api/relay/common"
     "one-api/relay/helper"
     "one-api/service"
     "one-api/setting/model_setting"
     "one-api/types"
     "strings"
 
     "github.com/gin-gonic/gin"
 )
@@
-        if common.DebugEnabled {
-            println("requestBody: ", string(jsonData))
-        }
+        logger.LogDebug(c, fmt.Sprintf("responses request body: %s", string(jsonData)))

Also applies to: 3-18

relay/compatible_handler.go (2)

76-85: Prefer structured debug logging over println for passthrough body

Use logger.LogDebug as done later for the JSON body; removes ad-hoc prints and centralizes log control.

-    if common.DebugEnabled {
-        println("requestBody: ", string(body))
-    }
+    logger.LogDebug(c, fmt.Sprintf("text passthrough request body: %s", string(body)))

86-107: Avoid variable shadowing: “request := convertedRequest...” hides the outer request

The inner “request” for convertedRequest shadows the outer pointer “request”, which is easy to misread and error-prone. Rename the inner variable.

-        request := convertedRequest.(*dto.GeneralOpenAIRequest)
+        convReq := convertedRequest.(*dto.GeneralOpenAIRequest)
         containSystemPrompt := false
-        for _, message := range request.Messages {
-            if message.Role == request.GetSystemRoleName() {
+        for _, message := range convReq.Messages {
+            if message.Role == convReq.GetSystemRoleName() {
                 containSystemPrompt = true
                 break
             }
         }
         if !containSystemPrompt {
             // 如果没有系统提示,则添加系统提示
             systemMessage := dto.Message{
-                Role:    request.GetSystemRoleName(),
+                Role:    convReq.GetSystemRoleName(),
                 Content: info.ChannelSetting.SystemPrompt,
             }
-            request.Messages = append([]dto.Message{systemMessage}, request.Messages...)
+            convReq.Messages = append([]dto.Message{systemMessage}, convReq.Messages...)
         } else if info.ChannelSetting.SystemPromptOverride {
             common.SetContextKey(c, constant.ContextKeySystemPromptOverride, true)
             // 如果有系统提示,且允许覆盖,则拼接到前面
-            for i, message := range request.Messages {
-                if message.Role == request.GetSystemRoleName() {
+            for i, message := range convReq.Messages {
+                if message.Role == convReq.GetSystemRoleName() {
                     if message.IsStringContent() {
-                        request.Messages[i].SetStringContent(info.ChannelSetting.SystemPrompt + "\n" + message.StringContent())
+                        convReq.Messages[i].SetStringContent(info.ChannelSetting.SystemPrompt + "\n" + message.StringContent())
                     } else {
                         contents := message.ParseContent()
                         contents = append([]dto.MediaContent{
                             {
                                 Type: dto.ContentTypeText,
                                 Text: info.ChannelSetting.SystemPrompt,
                             },
                         }, contents...)
-                        request.Messages[i].Content = contents
+                        convReq.Messages[i].Content = contents
                     }
                     break
                 }
             }
         }
relay/claude_handler.go (1)

97-101: Use structured debug logging for request body instead of println

Keep logging consistent with other handlers and avoid ad-hoc prints.

 import (
     "bytes"
     "fmt"
     "io"
     "net/http"
     "one-api/common"
+    "one-api/logger"
     "one-api/dto"
     relaycommon "one-api/relay/common"
     "one-api/relay/helper"
     "one-api/service"
     "one-api/setting/model_setting"
     "one-api/types"
     "strings"
@@
-        if common.DebugEnabled {
-            println("requestBody: ", string(jsonData))
-        }
+        logger.LogDebug(c, fmt.Sprintf("claude request body: %s", string(jsonData)))

Also applies to: 3-18

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 2c6a924 and 94536be.

📒 Files selected for processing (8)
  • common/gin.go (1 hunks)
  • relay/claude_handler.go (3 hunks)
  • relay/compatible_handler.go (2 hunks)
  • relay/embedding_handler.go (2 hunks)
  • relay/gemini_handler.go (2 hunks)
  • relay/image_handler.go (3 hunks)
  • relay/rerank_handler.go (2 hunks)
  • relay/responses_handler.go (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • common/gin.go
🚧 Files skipped from review as they are similar to previous changes (4)
  • relay/rerank_handler.go
  • relay/embedding_handler.go
  • relay/image_handler.go
  • relay/gemini_handler.go
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-05T17:14:17.246Z
Learnt from: neotf
PR: QuantumNous/new-api#1511
File: setting/ratio_setting/model_ratio.go:118-123
Timestamp: 2025-08-05T17:14:17.246Z
Learning: Claude models handle "-thinking" variants differently from Gemini models. For Claude models, only the base model (without "-thinking") gets an entry in defaultModelRatio map. The "-thinking" variants rely on the Claude relay handler stripping the suffix using strings.TrimSuffix(textRequest.Model, "-thinking") before looking up the ratio, so they automatically use the base model's ratio.

Applied to files:

  • relay/claude_handler.go
🧬 Code graph analysis (3)
relay/claude_handler.go (7)
dto/request_common.go (1)
  • Request (8-12)
dto/claude.go (2)
  • ClaudeRequest (187-203)
  • Thinking (399-402)
types/error.go (3)
  • NewErrorWithStatusCode (215-231)
  • ErrOptionWithSkipRetry (290-294)
  • NewError (181-193)
common/copy.go (1)
  • DeepCopy (8-21)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
setting/model_setting/claude.go (1)
  • GetClaudeSettings (41-47)
common/utils.go (1)
  • GetPointer (218-220)
relay/responses_handler.go (5)
dto/request_common.go (1)
  • Request (8-12)
dto/openai_request.go (1)
  • OpenAIResponsesRequest (762-784)
types/error.go (4)
  • NewErrorWithStatusCode (215-231)
  • ErrorCodeInvalidRequest (38-38)
  • ErrOptionWithSkipRetry (290-294)
  • NewError (181-193)
common/copy.go (1)
  • DeepCopy (8-21)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
relay/compatible_handler.go (6)
dto/request_common.go (1)
  • Request (8-12)
dto/openai_request.go (3)
  • GeneralOpenAIRequest (25-72)
  • WebSearchOptions (756-759)
  • StreamOptions (223-225)
types/error.go (5)
  • NewErrorWithStatusCode (215-231)
  • ErrorCodeInvalidRequest (38-38)
  • ErrOptionWithSkipRetry (290-294)
  • NewError (181-193)
  • ErrorCodeChannelModelMappedError (53-53)
common/copy.go (1)
  • DeepCopy (8-21)
relay/helper/model_mapped.go (1)
  • ModelMappedHelper (12-57)
constant/env.go (1)
  • ForceStreamOption (6-6)
🔇 Additional comments (2)
relay/responses_handler.go (1)

33-34: UpstreamModelName is properly initialized in InitChannelMeta, no action needed

InitChannelMeta (relay/common/relay_info.go:120–126) sets ChannelMeta.UpstreamModelName to the original model string from the context (constant.ContextKeyOriginalModel), which is then promoted to info.UpstreamModelName before any calls to ModelMappedHelper.
• ChannelMeta.UpstreamModelName assignment:

UpstreamModelName: common.GetContextKeyString(c, constant.ContextKeyOriginalModel),

• InitChannelMeta is always invoked (e.g. in ResponsesHelper at relay/responses_handler.go:21), ensuring info.UpstreamModelName is non-empty when ModelMappedHelper calls request.SetModelName(info.UpstreamModelName).

relay/claude_handler.go (1)

30-33: DeepCopy on input request is fine

Copying before mutation prevents side effects on the original payload; error handling is appropriate.

Comment thread relay/claude_handler.go
Comment on lines +35 to 38
err = helper.ModelMappedHelper(c, info, request)
if err != nil {
return types.NewError(err, types.ErrorCodeChannelModelMappedError, types.ErrOptionWithSkipRetry())
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Normalize “-thinking” before model mapping; don’t override UpstreamModelName after mapping

Currently you map first, then trim “-thinking” and overwrite info.UpstreamModelName. This can defeat model mappings and desync request vs info. Normalize the model name before ModelMappedHelper, and let mapping set UpstreamModelName. Then apply thinking defaults after mapping without overwriting UpstreamModelName.

-    err = helper.ModelMappedHelper(c, info, request)
-    if err != nil {
-        return types.NewError(err, types.ErrorCodeChannelModelMappedError, types.ErrOptionWithSkipRetry())
-    }
+    // Normalize "-thinking" before mapping so mapping/ratios use the base model
+    thinkingVariant := false
+    if model_setting.GetClaudeSettings().ThinkingAdapterEnabled && strings.HasSuffix(request.Model, "-thinking") {
+        request.Model = strings.TrimSuffix(request.Model, "-thinking")
+        thinkingVariant = true
+    }
+    err = helper.ModelMappedHelper(c, info, request)
+    if err != nil {
+        return types.NewError(err, types.ErrorCodeChannelModelMappedError, types.ErrOptionWithSkipRetry())
+    }
@@
-    if request.MaxTokens == 0 {
-        request.MaxTokens = uint(model_setting.GetClaudeSettings().GetDefaultMaxTokens(request.Model))
-    }
-
-    if model_setting.GetClaudeSettings().ThinkingAdapterEnabled &&
-        strings.HasSuffix(request.Model, "-thinking") {
+    if request.MaxTokens == 0 {
+        request.MaxTokens = uint(model_setting.GetClaudeSettings().GetDefaultMaxTokens(request.Model))
+    }
+    if model_setting.GetClaudeSettings().ThinkingAdapterEnabled && thinkingVariant {
         if request.Thinking == nil {
             // 因为BudgetTokens 必须大于1024
             if request.MaxTokens < 1280 {
                 request.MaxTokens = 1280
             }
 
             // BudgetTokens 为 max_tokens 的 80%
             request.Thinking = &dto.Thinking{
                 Type:         "enabled",
                 BudgetTokens: common.GetPointer[int](int(float64(request.MaxTokens) * model_setting.GetClaudeSettings().ThinkingAdapterBudgetTokensPercentage)),
             }
             // TODO: 临时处理
             // https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#important-considerations-when-using-extended-thinking
             request.TopP = 0
             request.Temperature = common.GetPointer[float64](1.0)
         }
-        request.Model = strings.TrimSuffix(request.Model, "-thinking")
-        info.UpstreamModelName = request.Model
     }

Also applies to: 50-70

🤖 Prompt for AI Agents
In relay/claude_handler.go around lines 35-38 (and similarly lines ~50-70), the
code currently calls ModelMappedHelper and then trims "-thinking" and overwrites
info.UpstreamModelName which can break mappings and desync request vs info;
instead, normalize the incoming model name by trimming any "-thinking" suffix on
the request/model variable before calling ModelMappedHelper so mapping can set
info.UpstreamModelName correctly, do not overwrite info.UpstreamModelName after
ModelMappedHelper returns, and if you need to apply "thinking" defaults do so
afterwards by adjusting only the request/behavioral defaults (not
info.UpstreamModelName) so mapping remains authoritative.

@Calcium-Ion
Calcium-Ion merged commit 4226746 into alpha Aug 23, 2025
@Calcium-Ion
Calcium-Ion deleted the fix-retry-and-rerank branch October 11, 2025 03:06
@coderabbitai coderabbitai Bot mentioned this pull request Nov 3, 2025
@coderabbitai coderabbitai Bot mentioned this pull request Dec 25, 2025
@coderabbitai coderabbitai Bot mentioned this pull request Jan 22, 2026
@coderabbitai coderabbitai Bot mentioned this pull request Mar 28, 2026
Closed
@coderabbitai coderabbitai Bot mentioned this pull request Apr 23, 2026
x22x22 pushed a commit to x22x22/new-api that referenced this pull request Apr 24, 2026
…rank

fix: retry requeset body incorrect and fix rerank
@coderabbitai coderabbitai Bot mentioned this pull request Jul 2, 2026
10 of 11 tasks
@coderabbitai coderabbitai Bot mentioned this pull request Jul 17, 2026
8 of 11 tasks
salem-2007 added a commit to salem-2007/new-api that referenced this pull request Sep 10, 2026
…rank

fix: retry requeset body incorrect and fix rerank
junjundesk pushed a commit to junjundesk/new-api that referenced this pull request Oct 3, 2026
…rank

fix: retry requeset body incorrect and fix rerank
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant