Skip to content

feat(relay): add Responses API → Chat Completions conversion for upst… - #3067

Closed
lsg1103275794 wants to merge 2 commits into
QuantumNous:mainfrom
lsg1103275794:feat/responses-api-via-chat-completions
Closed

lsg1103275794 wants to merge 2 commits into
QuantumNous:mainfrom
lsg1103275794:feat/responses-api-via-chat-completions

Conversation

@lsg1103275794

@lsg1103275794 lsg1103275794 commented Feb 28, 2026 •

Copy link
Copy Markdown

…ream compatibility

Allow channels that only support /v1/chat/completions to serve clients that speak the OpenAI Responses API (e.g. Codex CLI v0.106+).

Background

OpenAI Codex CLI v0.106+ exclusively uses the Responses API (/v1/responses). Many upstream providers (NVIDIA NIM, ZhipuAI, etc.) only implement Chat Completions (/v1/chat/completions). This change bridges the gap by transparently converting between the two formats inside the relay layer.

New files

  • relay/responses_via_chat_completions.go Main handler: converts Responses request → Chat Completions, forwards to upstream, converts response back (streaming + non-streaming). Streaming emits the full Responses API SSE event sequence Codex needs. Tool call chunks tracked by index for correct ID/name reconstruction.

  • service/openaicompat/responses_request_to_chat.go Converts Responses API request → Chat Completions request. Handles input array (message, function_call, function_call_output, input_text, input_image), merges consecutive function_calls into a single assistant message with tool_calls[].

  • service/openaicompat/chat_response_to_responses.go Converts Chat Completions response → Responses API response. Non-streaming + streaming chunk converters + completed event builder.

Modified files

  • relay/responses_handler.go: hook in ResponsesHelper
  • service/openaicompat/policy.go: ShouldResponsesUseChatCompletions*()
  • setting/model_setting/global.go: ResponsesToChatCompletionsPolicy struct
  • service/openai_chat_responses_compat.go: exported wrapper functions

Configuration (System Settings)

{
  "enabled": true,
  "all_channels": true,
  "model_patterns": [".*"]
}

Tested with

  • OpenAI Codex CLI v0.106.0 + NVIDIA NIM upstream (glm-5)
  • Multi-turn conversations, streaming, tool call round-trips ✓

Summary by CodeRabbit

  • New Features
    • Convert Responses requests to Chat Completions for compatible channels (streaming & non-streaming), including tool call and usage mapping.
    • Global routing policies and settings to control conversions with regex-based model matching (cached).
  • Behavior
    • New quota dispatch path that attributes consumption per model family and reuses reported usage.
  • Chores
    • Added compatibility wrappers to simplify request/response translations.

…ream compatibility

Allow channels that only support /v1/chat/completions to serve clients
that speak the OpenAI Responses API (e.g. Codex CLI v0.106+).

## Background

OpenAI Codex CLI v0.106+ exclusively uses the Responses API
(/v1/responses). Many upstream providers (NVIDIA NIM, ZhipuAI, etc.)
only implement Chat Completions (/v1/chat/completions). This change
bridges the gap by transparently converting between the two formats
inside the relay layer.

## New files

- relay/responses_via_chat_completions.go
  Main handler: converts Responses request → Chat Completions, forwards
  to upstream, converts response back (streaming + non-streaming).
  Streaming emits the full Responses API SSE event sequence Codex needs.
  Tool call chunks tracked by index for correct ID/name reconstruction.

- service/openaicompat/responses_request_to_chat.go
  Converts Responses API request → Chat Completions request.
  Handles input array (message, function_call, function_call_output,
  input_text, input_image), merges consecutive function_calls into
  a single assistant message with tool_calls[].

- service/openaicompat/chat_response_to_responses.go
  Converts Chat Completions response → Responses API response.
  Non-streaming + streaming chunk converters + completed event builder.

## Modified files

- relay/responses_handler.go: hook in ResponsesHelper
- service/openaicompat/policy.go: ShouldResponsesUseChatCompletions*()
- setting/model_setting/global.go: ResponsesToChatCompletionsPolicy struct
- service/openai_chat_responses_compat.go: exported wrapper functions

## Configuration (System Settings)

```json
{
  "enabled": true,
  "all_channels": true,
  "model_patterns": [".*"]
}
```

## Tested with

- OpenAI Codex CLI v0.106.0 + NVIDIA NIM upstream (glm-5)
- Multi-turn conversations, streaming, tool call round-trips ✓
@coderabbitai

coderabbitai Bot commented Feb 28, 2026 •

Copy link
Copy Markdown
Contributor

Walkthrough

Adds a Responses→Chat Completions routing path with request/response conversion (including streaming), policy-based routing settings, regex-based model matching, and centralized quota posting for model-specific consumption.

Changes

Cohort / File(s) Summary
Relay handler changes
relay/responses_handler.go
Adds an early branch to route qualifying Responses requests to chat completions, reuses usage objects, and adds postQuotaForModel to dispatch model-specific quota consumption.
Responses→Chat Completions bridge
relay/responses_via_chat_completions.go
New handler that converts Responses requests to Chat Completions, sends upstream, handles streaming and non-streaming responses, converts upstream responses/events back to Responses format, and returns usage/errors.
Compatibility wrappers
service/openai_chat_responses_compat.go
Adds public wrapper functions delegating conversions and policy checks between Chat Completions and Responses compat helpers.
Responses→Chat request conversion
service/openaicompat/responses_request_to_chat.go
New converter that maps Responses API inputs/tools to Chat Completions messages/fields with robust parsing and flushing semantics.
Chat→Responses response conversion
service/openaicompat/chat_response_to_responses.go
New converters translating chat responses and stream chunks into Responses API outputs and SSE events, plus SSE marshaling and completion event building.
Routing policy & regex
service/openaicompat/policy.go, service/openaicompat/regex.go, setting/model_setting/global.go
Introduces ChannelRoutingPolicy and policy helpers (global wrappers), a cached regex matcher matchAnyRegex, and adds ResponsesToChatCompletionsPolicy to global settings with accessors.

Sequence Diagram

sequenceDiagram
    participant Client
    participant Handler as responses_handler
    participant Adaptor
    participant Converter as RequestConverter
    participant ChatAPI as ChatCompletions
    participant StreamConv as StreamConverter
    participant Quota

    Client->>Handler: POST /v1/responses
    Handler->>Adaptor: adaptor.Init / checks
    Handler->>Handler: ShouldResponsesUseChatCompletionsGlobal?
    alt route to chat completions
        Handler->>Converter: ResponsesRequestToChatCompletionsRequest
        Converter-->>Handler: GeneralOpenAIRequest
        Handler->>ChatAPI: POST /v1/chat/completions (converted)
        alt streaming
            ChatAPI-->>StreamConv: SSE chunks
            StreamConv-->>Handler: Responses SSE events
            Handler-->>Client: SSE event stream
        else non-streaming
            ChatAPI-->>Handler: OpenAITextResponse
            Handler->>StreamConv: ChatCompletionsResponseToResponsesResponse
            StreamConv-->>Handler: OpenAIResponsesResponse + Usage
            Handler-->>Client: Responses API response
        end
        Handler->>Quota: postQuotaForModel(info, usage)
        Quota-->>Handler: quota ack
    else normal Responses flow
        Handler->>Handler: continue standard Responses handling
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

Suggested reviewers

  • creamlike1024
  • Calcium-Ion

Poem

🐰 Hops of code from bridge to stream,

Responses leap into chat with gleam,
Events unfurl, usage tallied true,
Tools and text hop back to you,
A rabbit cheers — new paths pursue!

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main feature: adding Responses API to Chat Completions conversion for upstream channel compatibility. It's concise and directly reflects the primary change.
Docstring Coverage ✅ Passed Docstring coverage is 92.59% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (5)
setting/model_setting/global.go (1)

35-61: Consider extracting a shared policy type to reduce duplication.

ResponsesToChatCompletionsPolicy is structurally identical to ChatCompletionsToResponsesPolicy (lines 10-33). Both have the same fields and the same IsChannelEnabled logic. You could define a generic ChannelRoutingPolicy type and alias or embed it for both use cases.

That said, keeping them separate is acceptable if you anticipate the policies diverging in the future.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@setting/model_setting/global.go` around lines 35 - 61,
ResponsesToChatCompletionsPolicy and ChatCompletionsToResponsesPolicy share
identical fields and IsChannelEnabled logic; refactor by introducing a single
reusable type (e.g., ChannelRoutingPolicy) that defines Enabled, AllChannels,
ChannelIDs, ChannelTypes, ModelPatterns and the IsChannelEnabled method, then
have ResponsesToChatCompletionsPolicy and ChatCompletionsToResponsesPolicy
either embed or alias ChannelRoutingPolicy so both retain their names while
eliminating duplication (update usages of IsChannelEnabled to work with the new
shared type).
relay/responses_handler.go (1)

86-90: Consider extracting quota posting logic to reduce duplication.

The gpt-4o-audio check and branching logic here duplicates lines 176-180. A helper function could centralize this:

func postQuotaForModel(c *gin.Context, info *relaycommon.RelayInfo, usage *dto.Usage) {
    if strings.HasPrefix(info.OriginModelName, "gpt-4o-audio") {
        service.PostAudioConsumeQuota(c, info, usage, "")
    } else {
        postConsumeQuota(c, info, usage)
    }
}

This is optional since the duplication is minor.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@relay/responses_handler.go` around lines 86 - 90, Duplicate branching that
checks info.OriginModelName and calls either service.PostAudioConsumeQuota or
postConsumeQuota should be extracted into a small helper to avoid repetition:
create a function (e.g., postQuotaForModel) that accepts (c *gin.Context, info
*relaycommon.RelayInfo, usage *dto.Usage) and inside it perform the
strings.HasPrefix(info.OriginModelName, "gpt-4o-audio") check and call
service.PostAudioConsumeQuota(c, info, usage, "") or postConsumeQuota(c, info,
usage) accordingly, then replace both duplicated blocks with a single call to
postQuotaForModel.
service/openaicompat/responses_request_to_chat.go (1)

158-162: Consider logging or returning error on malformed input items.

Silently continuing when an input item fails to unmarshal could hide data corruption or invalid client requests. At minimum, consider logging a warning.

Proposed fix
 	for _, raw := range items {
 		var item map[string]any
 		if err := common.Unmarshal(raw, &item); err != nil {
-			continue
+			// Log warning but continue processing other items
+			common.LogWarn(c, fmt.Sprintf("skipping malformed input item: %v", err))
+			continue
 		}

Note: Adjust the logging call to match your project's logging conventions.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@service/openaicompat/responses_request_to_chat.go` around lines 158 - 162,
The loop currently swallows unmarshalling errors for each element (for _, raw :=
range items { var item map[string]any; if err := common.Unmarshal(raw, &item);
err != nil { continue } }), which hides malformed input; change this to log a
warning (including the error and the raw payload) when common.Unmarshal fails
for a given raw item and then continue, or alternatively return an error if the
caller should be notified; locate the loop that references items/raw/item and
update the error branch to call the project logger (or the function's logger)
with context about the failure and the offending raw data instead of silently
continuing.
relay/responses_via_chat_completions.go (1)

164-478: Consider splitting chatToResponsesStreamHandler for maintainability.

At ~314 lines, this function handles multiple concerns: SSE header setup, event emission helpers, chunk parsing, text content streaming, tool call tracking, and completion events. Extracting helper functions (e.g., emitTextDelta, emitToolCallEvents, emitCompletionEvents) would improve readability.

This is optional given the function's cohesive purpose.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@relay/responses_via_chat_completions.go` around lines 164 - 478,
chatToResponsesStreamHandler is doing too many things in one function; extract
the streaming sub-tasks into focused helpers to improve readability and
maintainability: pull out the SSE header setup and sendEvent/sendCreatedIfNeeded
closures into a small helper, extract the content-delta logic into
emitTextDelta(caller state, msgID, fullText, sentOutputItem, sentContentPart,
contentStr) and call it from the StreamScannerHandler, extract the tool-call
handling into handleToolCallChunk(toolCallsByIndex, toolCallOrder,
delta.ToolCalls, sendEvent) that updates toolCallState and emits intermediate
function_call events, and extract finalization into emitCompletionEvents(respID,
model, fullText, usage, toolCalls, sentOutputItem, sentContentPart, sendEvent)
which emits the done events and the final completedEvent; keep the state types
(toolCallState, toolCallsByIndex, toolCallOrder, fullText, sent* flags) in the
outer scope and update chatToResponsesStreamHandler to orchestrate these helpers
(maintain existing function names: chatToResponsesStreamHandler, sendEvent,
sendCreatedIfNeeded) so callers/logic remain unchanged.
service/openaicompat/chat_response_to_responses.go (1)

158-161: Remove dead code block.

This empty if-block serves no purpose and should be removed for clarity.

Proposed fix
-	// Finish reason
-	if choice.FinishReason != nil && *choice.FinishReason != "" {
-		// Will be handled by the caller when building the completed event
-	}
-
 	return events
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@service/openaicompat/chat_response_to_responses.go` around lines 158 - 161,
Remove the dead empty conditional that checks choice.FinishReason in the
function that converts OpenAI chat choices to Responses (the block containing
"if choice.FinishReason != nil && *choice.FinishReason != \"\" { /* Will be
handled by the caller when building the completed event */ }"). Simply delete
that entire if-statement — leave no no-op branches; rely on the caller's
handling of the finish reason when building the completed event.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@relay/responses_via_chat_completions.go`:
- Around line 388-424: The sendEvent calls after the streaming loop (guarded by
sentContentPart and sentOutputItem) currently ignore their returned error, which
can silently drop failures (e.g., client disconnect); update the code around
sendEvent (used with msgID, sentContentPart, sentOutputItem, and
fullText.String()) to capture the returned error and handle it—either return it,
log it via the existing logger, or at minimum wrap each sendEvent call with an
if err := sendEvent(...); err != nil { logger.Warn/Error(...) }—or add a clear
comment stating that these "done" events are best-effort and failures are
intentionally ignored.
- Around line 88-91: The code unsafely asserts resp.(*http.Response) which can
panic; change to a safe type assertion using the resp variable (e.g., resp, ok
:= resp.(*http.Response)) and only proceed to set httpResp and update
info.IsStream when ok is true, otherwise handle the unexpected type by logging
or returning an error; update uses around statusCodeMappingStr, httpResp and
info.IsStream so callers downstream expect httpResp to be nil/unchanged when the
assertion fails.

---

Nitpick comments:
In `@relay/responses_handler.go`:
- Around line 86-90: Duplicate branching that checks info.OriginModelName and
calls either service.PostAudioConsumeQuota or postConsumeQuota should be
extracted into a small helper to avoid repetition: create a function (e.g.,
postQuotaForModel) that accepts (c *gin.Context, info *relaycommon.RelayInfo,
usage *dto.Usage) and inside it perform the
strings.HasPrefix(info.OriginModelName, "gpt-4o-audio") check and call
service.PostAudioConsumeQuota(c, info, usage, "") or postConsumeQuota(c, info,
usage) accordingly, then replace both duplicated blocks with a single call to
postQuotaForModel.

In `@relay/responses_via_chat_completions.go`:
- Around line 164-478: chatToResponsesStreamHandler is doing too many things in
one function; extract the streaming sub-tasks into focused helpers to improve
readability and maintainability: pull out the SSE header setup and
sendEvent/sendCreatedIfNeeded closures into a small helper, extract the
content-delta logic into emitTextDelta(caller state, msgID, fullText,
sentOutputItem, sentContentPart, contentStr) and call it from the
StreamScannerHandler, extract the tool-call handling into
handleToolCallChunk(toolCallsByIndex, toolCallOrder, delta.ToolCalls, sendEvent)
that updates toolCallState and emits intermediate function_call events, and
extract finalization into emitCompletionEvents(respID, model, fullText, usage,
toolCalls, sentOutputItem, sentContentPart, sendEvent) which emits the done
events and the final completedEvent; keep the state types (toolCallState,
toolCallsByIndex, toolCallOrder, fullText, sent* flags) in the outer scope and
update chatToResponsesStreamHandler to orchestrate these helpers (maintain
existing function names: chatToResponsesStreamHandler, sendEvent,
sendCreatedIfNeeded) so callers/logic remain unchanged.

In `@service/openaicompat/chat_response_to_responses.go`:
- Around line 158-161: Remove the dead empty conditional that checks
choice.FinishReason in the function that converts OpenAI chat choices to
Responses (the block containing "if choice.FinishReason != nil &&
*choice.FinishReason != \"\" { /* Will be handled by the caller when building
the completed event */ }"). Simply delete that entire if-statement — leave no
no-op branches; rely on the caller's handling of the finish reason when building
the completed event.

In `@service/openaicompat/responses_request_to_chat.go`:
- Around line 158-162: The loop currently swallows unmarshalling errors for each
element (for _, raw := range items { var item map[string]any; if err :=
common.Unmarshal(raw, &item); err != nil { continue } }), which hides malformed
input; change this to log a warning (including the error and the raw payload)
when common.Unmarshal fails for a given raw item and then continue, or
alternatively return an error if the caller should be notified; locate the loop
that references items/raw/item and update the error branch to call the project
logger (or the function's logger) with context about the failure and the
offending raw data instead of silently continuing.

In `@setting/model_setting/global.go`:
- Around line 35-61: ResponsesToChatCompletionsPolicy and
ChatCompletionsToResponsesPolicy share identical fields and IsChannelEnabled
logic; refactor by introducing a single reusable type (e.g.,
ChannelRoutingPolicy) that defines Enabled, AllChannels, ChannelIDs,
ChannelTypes, ModelPatterns and the IsChannelEnabled method, then have
ResponsesToChatCompletionsPolicy and ChatCompletionsToResponsesPolicy either
embed or alias ChannelRoutingPolicy so both retain their names while eliminating
duplication (update usages of IsChannelEnabled to work with the new shared
type).

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 21cfc1c and b38f40d.

📒 Files selected for processing (7)
  • relay/responses_handler.go
  • relay/responses_via_chat_completions.go
  • service/openai_chat_responses_compat.go
  • service/openaicompat/chat_response_to_responses.go
  • service/openaicompat/policy.go
  • service/openaicompat/responses_request_to_chat.go
  • setting/model_setting/global.go

Comment thread relay/responses_via_chat_completions.go Resolved
Comment thread relay/responses_via_chat_completions.go Resolved
Comment on lines +3 to +11
import (
"encoding/json"
"fmt"
"strings"
"time"

"github.com/QuantumNous/new-api/common"
"github.com/QuantumNous/new-api/dto"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Avoid direct import of encoding/json per coding guidelines.

The coding guidelines require using wrapper functions from common/json.go (common.Marshal(), common.Unmarshal(), etc.) instead of directly importing encoding/json. The json.Number type at line 33 can be handled by checking if the value is a string and parsing it.

Proposed fix
 import (
-	"encoding/json"
 	"fmt"
 	"strings"
 	"time"

And update the type switch at lines 26-37:

 		switch v := chatResp.Created.(type) {
 		case float64:
 			createdAt = int(v)
 		case int64:
 			createdAt = int(v)
 		case int:
 			createdAt = v
-		case json.Number:
-			if n, err := v.Int64(); err == nil {
-				createdAt = int(n)
-			}
+		case string:
+			// Handle JSON number encoded as string
+			var n int64
+			if _, err := fmt.Sscanf(v, "%d", &n); err == nil {
+				createdAt = int(n)
+			}
 		}

As per coding guidelines: "Do NOT directly import or call encoding/json in business code; use wrapper functions from common/json.go instead"

- Extract shared ChannelRoutingPolicy type to eliminate duplicate structs
- Safe type assertion for resp.(*http.Response) to prevent panics
- Check sendEvent return values in stream cleanup to catch disconnects
- Log malformed Responses input items instead of silent continue
- Remove dead FinishReason if-block in chat_response_to_responses.go
- Extract postQuotaForModel helper to deduplicate quota dispatch logic
- Add Go docstrings to all exported functions (80%+ coverage)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@service/openaicompat/chat_response_to_responses.go`:
- Around line 84-91: Add a nil-check for chatResp.Usage before reading its
fields in the conversion that builds respUsage; if chatResp.Usage is nil,
initialize respUsage as a dto.Usage with zero values (or only set
InputTokens/OutputTokens to 0), otherwise copy PromptTokens, CompletionTokens,
TotalTokens and set InputTokens/OutputTokens from PromptTokens/CompletionTokens
respectively. Update the code around the creation of respUsage (references:
chatResp.Usage and respUsage, dto.Usage) so it safely handles a nil Usage
without dereferencing it.

In `@service/openaicompat/responses_request_to_chat.go`:
- Around line 3-11: Remove the direct encoding/json import and stop using
json.RawMessage in this file: replace any parameter or type references to
json.RawMessage with either common.RawMessage (after re-exporting
json.RawMessage from common/json.go) or change the signature to use []byte and
update call sites accordingly; then remove "encoding/json" from the import block
and import only "github.com/QuantumNous/new-api/common" (or adjust call sites to
accept []byte) so all JSON types/operations flow through the common package.
Ensure you update references to json.RawMessage in functions and imports to the
chosen alternative consistently.

ℹ️ Review info

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b38f40d and bb8d47e.

📒 Files selected for processing (8)
  • relay/responses_handler.go
  • relay/responses_via_chat_completions.go
  • service/openai_chat_responses_compat.go
  • service/openaicompat/chat_response_to_responses.go
  • service/openaicompat/policy.go
  • service/openaicompat/regex.go
  • service/openaicompat/responses_request_to_chat.go
  • setting/model_setting/global.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • relay/responses_via_chat_completions.go

Comment on lines +84 to +91
usage := chatResp.Usage
respUsage := &dto.Usage{
PromptTokens: usage.PromptTokens,
CompletionTokens: usage.CompletionTokens,
TotalTokens: usage.TotalTokens,
InputTokens: usage.PromptTokens,
OutputTokens: usage.CompletionTokens,
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Potential nil pointer dereference on chatResp.Usage.

If the upstream Chat Completions response doesn't include a Usage field (which can happen in streaming or error scenarios), accessing chatResp.Usage.PromptTokens at line 86 will panic. Consider adding a nil check.

🛡️ Proposed fix
-	usage := chatResp.Usage
-	respUsage := &dto.Usage{
-		PromptTokens:     usage.PromptTokens,
-		CompletionTokens: usage.CompletionTokens,
-		TotalTokens:      usage.TotalTokens,
-		InputTokens:      usage.PromptTokens,
-		OutputTokens:     usage.CompletionTokens,
+	var respUsage *dto.Usage
+	if chatResp.Usage != nil {
+		respUsage = &dto.Usage{
+			PromptTokens:     chatResp.Usage.PromptTokens,
+			CompletionTokens: chatResp.Usage.CompletionTokens,
+			TotalTokens:      chatResp.Usage.TotalTokens,
+			InputTokens:      chatResp.Usage.PromptTokens,
+			OutputTokens:     chatResp.Usage.CompletionTokens,
+		}
 	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
usage := chatResp.Usage
respUsage := &dto.Usage{
PromptTokens: usage.PromptTokens,
CompletionTokens: usage.CompletionTokens,
TotalTokens: usage.TotalTokens,
InputTokens: usage.PromptTokens,
OutputTokens: usage.CompletionTokens,
}
var respUsage *dto.Usage
if chatResp.Usage != nil {
respUsage = &dto.Usage{
PromptTokens: chatResp.Usage.PromptTokens,
CompletionTokens: chatResp.Usage.CompletionTokens,
TotalTokens: chatResp.Usage.TotalTokens,
InputTokens: chatResp.Usage.PromptTokens,
OutputTokens: chatResp.Usage.CompletionTokens,
}
}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@service/openaicompat/chat_response_to_responses.go` around lines 84 - 91, Add
a nil-check for chatResp.Usage before reading its fields in the conversion that
builds respUsage; if chatResp.Usage is nil, initialize respUsage as a dto.Usage
with zero values (or only set InputTokens/OutputTokens to 0), otherwise copy
PromptTokens, CompletionTokens, TotalTokens and set InputTokens/OutputTokens
from PromptTokens/CompletionTokens respectively. Update the code around the
creation of respUsage (references: chatResp.Usage and respUsage, dto.Usage) so
it safely handles a nil Usage without dereferencing it.

Comment on lines +3 to +11
import (
"fmt"
"encoding/json"
"errors"
"strings"

"github.com/QuantumNous/new-api/common"
"github.com/QuantumNous/new-api/dto"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

# Check if common/json.go re-exports json.RawMessage
rg -n "RawMessage" common/json.go || echo "RawMessage not found in common/json.go"

# Check the first 50 lines of common/json.go to understand its structure
head -50 common/json.go

# Check how many Go files use json.RawMessage
echo "=== Files using json.RawMessage ==="
rg -l "json\.RawMessage" --type=go

# For files that use json.RawMessage, check if they import encoding/json
echo "=== Import patterns for json.RawMessage usage ==="
rg -B 20 "json\.RawMessage" --type=go | grep -E "(import|json\.RawMessage)" | head -30

Repository: QuantumNous/new-api

Length of output: 4565


Remove direct encoding/json import and use common package instead.

The import of encoding/json at line 5 violates the coding guideline: "Do NOT directly import or call encoding/json in business code." Although json.RawMessage is used as a parameter type rather than for marshal/unmarshal operations, the import itself is forbidden. The guideline requires all JSON-related code to go through the common package.

To resolve: Either re-export json.RawMessage from common/json.go and import it from there, or refactor the function signature to use []byte instead.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@service/openaicompat/responses_request_to_chat.go` around lines 3 - 11,
Remove the direct encoding/json import and stop using json.RawMessage in this
file: replace any parameter or type references to json.RawMessage with either
common.RawMessage (after re-exporting json.RawMessage from common/json.go) or
change the signature to use []byte and update call sites accordingly; then
remove "encoding/json" from the import block and import only
"github.com/QuantumNous/new-api/common" (or adjust call sites to accept []byte)
so all JSON types/operations flow through the common package. Ensure you update
references to json.RawMessage in functions and imports to the chosen alternative
consistently.

@seefs001

seefs001 commented Mar 1, 2026

Copy link
Copy Markdown
Collaborator

核心功能不接受Vibe Coding PR

@seefs001 seefs001 closed this Mar 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants