Skip to content

feat: Add Xiaomi channel support and enhance TTS functionality - #3479

Closed
HynoR wants to merge 1 commit into
QuantumNous:mainfrom
HynoR:feat/xm
Closed

HynoR wants to merge 1 commit into
QuantumNous:mainfrom
HynoR:feat/xm

Conversation

@HynoR

@HynoR HynoR commented Mar 28, 2026 •

Copy link
Copy Markdown
Contributor

添加小米渠道支持,并支持小米 tts 从 chat completion 接口到 openai tts 接口 的转换
fix #3353

测试结果 tts -> xiaomi tts chat
image

测试结构 chat -> xiaomi tts chat
image

计费
image

喜闻乐见
image
image
image

This pull request adds support for the Xiaomi MiMo channel throughout the codebase, enabling Xiaomi MiMo as a new AI provider. The changes include defining new constants and configuration for Xiaomi, implementing the channel adaptor and response handling, updating pricing and model ratios, and exposing the channel in the frontend options.

Xiaomi MiMo Channel Integration:

  • Added new channel and API type constants for Xiaomi MiMo, including ChannelTypeXiaomi and APITypeXiaomi, and updated mappings for names and base URLs. [1] [2] [3] [4] [5]
  • Implemented Xiaomi MiMo channel adaptor (adaptor.go) and TTS response handler (tts.go), supporting audio requests and response decoding. [1] [2]
  • Added Xiaomi MiMo model list and channel name constants.
  • Registered Xiaomi MiMo as a supported channel in the relay adaptor and enabled streaming support. [1] [2] [3]

Pricing and Model Configuration:

  • Added Xiaomi MiMo vendor and model pricing information to default vendor rules and model ratio settings. [1] [2]

Frontend Exposure:

  • Added Xiaomi MiMo as an option in the frontend channel selection list.

Summary by CodeRabbit

Release Notes

  • New Features

    • Added support for Xiaomi MiMo channel integration with eight available models
    • Added text-to-speech (TTS) functionality for Xiaomi
    • Added audio payload support for chat messages
    • Xiaomi channel now available in channel selection UI
  • Bug Fixes

    • Fixed Xiaomi channel type recognition in request routing

@coderabbitai

coderabbitai Bot commented Mar 28, 2026 •

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds support for the Xiaomi MiMo API channel by introducing new API type and channel type constants, implementing a channel adaptor with TTS audio request/response handling, registering model configurations and pricing rules, integrating with the relay system, and extending the web UI with a new Xiaomi channel option.

Changes

Cohort / File(s) Summary
API & Channel Constants
constant/api_type.go, constant/channel.go, relay/channel/xiaomi/constants.go
Introduces APITypeXiaomi, ChannelTypeXiaomi = 58, and exports Xiaomi model list (mimo-v2-pro, mimo-v2-flash, mimo-v2-omni, mimo-v2-tts, etc.) and channel identifier.
Xiaomi Channel Adaptor
relay/channel/xiaomi/adaptor.go, relay/channel/xiaomi/tts.go
Implements Xiaomi adaptor with Init, GetRequestURL, SetupRequestHeader, request conversion methods, TTS-specific audio handling (format normalization, voice defaults, base64 decoding), and response processing with error handling.
Relay Integration
relay/relay_adaptor.go, relay/common/relay_info.go, common/api_type.go
Updates GetAdaptor to instantiate Xiaomi adaptor for APITypeXiaomi, marks Xiaomi channel as streaming-compatible, and maps ChannelTypeXiaomi to APITypeXiaomi in ChannelType2APIType.
Model & Vendor Configuration
setting/ratio_setting/model_ratio.go, model/pricing_default.go
Adds default model ratios for mimo-v2-pro, mimo-v2-flash, mimo-v2-omni, and registers "mimo" keyword matching for vendor auto-mapping to "小米".
Data Transfer Object Updates
dto/openai_request.go, dto/openai_audio_response_test.go
Adds optional audio JSON field to Message struct and includes test validating audio field preservation through request/response round-trip serialization.
Web Frontend
web/classic/src/constants/channel.constants.js
Adds Xiaomi MiMo channel option (ID: 58, orange color label) to CHANNEL_OPTIONS in the UI.

Sequence Diagram

sequenceDiagram
    participant Client as OpenAI Client
    participant Relay as new-api Relay
    participant Adaptor as Xiaomi Adaptor
    participant XiaomiAPI as Xiaomi MiMo API
    
    Client->>Relay: POST /v1/audio/speech<br/>(OpenAI TTS format)
    activate Relay
    Relay->>Adaptor: ConvertAudioRequest()
    activate Adaptor
    Adaptor->>Adaptor: Normalize audio format<br/>(wav, pcm16)
    Adaptor->>Adaptor: Default voice if missing
    Adaptor->>Adaptor: Build xiaomiTTSRequest<br/>from Instructions+Input
    Adaptor->>Adaptor: Marshal to JSON
    Adaptor-->>Relay: Return io.Reader
    deactivate Adaptor
    
    Relay->>Adaptor: SetupRequestHeader()
    activate Adaptor
    Adaptor->>Adaptor: Add Authorization header<br/>(Bearer token + api-key)
    Adaptor-->>Relay: Headers ready
    deactivate Adaptor
    
    Relay->>XiaomiAPI: POST /v1/chat/completions<br/>(xiaomiTTSRequest)
    activate XiaomiAPI
    XiaomiAPI-->>Relay: 200 OK<br/>(mimoTTSResponse with<br/>base64 audio data)
    deactivate XiaomiAPI
    
    Relay->>Adaptor: DoResponse(resp)
    activate Adaptor
    Adaptor->>Adaptor: Unmarshal mimoTTSResponse
    Adaptor->>Adaptor: Decode base64 audio
    Adaptor->>Adaptor: Determine MIME type<br/>from audioFormat
    Adaptor->>Adaptor: Extract usage tokens
    Adaptor-->>Relay: (audio bytes, usage)
    deactivate Adaptor
    
    Relay->>Client: 200 OK<br/>(audio/wav or audio/pcm)
    deactivate Relay
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • PR #2081: Modifies the same functions (ChannelType2APIType, GetAdaptor) and adds a new API type with channel integration, following the same pattern of channel onboarding.
  • PR #2190: Adds a new API/channel pair with analogous code-level changes (constant registration, adaptor wiring, and relay integration).
  • PR #1915: Implements parallel channel support by adding API/channel type constants and wiring a new adaptor into the relay system through the same integration points.

Suggested reviewers

  • seefs001
  • creamlike1024

Poem

🐰 A Xiaomi door now opens wide,
With MiMo voices deep inside,
TTS audio, base64 decoded fast,
From chat completions, at last!
This fuzzy bundle hops with glee,
As new channels come to be! 🎵

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and specifically describes the primary changes: adding Xiaomi channel support and TTS functionality to the codebase.
Linked Issues check ✅ Passed The PR fully implements all coding requirements from issue #3353: Xiaomi channel constants, adaptor for bridging /v1/audio/speech to /v1/chat/completions, TTS request/response conversion, and model detection.
Out of Scope Changes check ✅ Passed All changes are directly aligned with adding Xiaomi channel support and TTS bridging. No unrelated modifications detected; frontend UI, pricing, and model ratio updates are all necessary for complete Xiaomi integration.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@relay/channel/xiaomi/adaptor.go`:
- Around line 56-57: The ConvertRerankRequest method currently returns (nil,
nil) which silently allows unsupported rerank traffic; update
Adaptor.ConvertRerankRequest to return an explicit error (e.g., using errors.New
or fmt.Errorf) indicating the provider does not support rerank so callers fail
fast; ensure you update imports to include "errors" or "fmt" and keep the
function signature unchanged so callers can handle the error.

In `@relay/channel/xiaomi/tts.go`:
- Around line 41-49: The resp.Body is only deferred closed after io.ReadAll, so
an early return on readErr leaks the body; move the defer resp.Body.Close() to
immediately after obtaining resp (i.e., just after the resp variable is set) or
right before calling io.ReadAll in the Xiaomi TTS handling code so that
resp.Body is closed on all paths including when readErr triggers the
types.NewErrorWithStatusCode return; update code around resp.Body, body, readErr
and the error-returning path accordingly.

In `@setting/ratio_setting/model_ratio.go`:
- Around line 246-248: The Xiaomi model ratio map is missing an explicit entry
for "mimo-v2-tts", causing it to fall back to the large default in
GetModelRatio; add an explicit key "mimo-v2-tts" to the same map that contains
"mimo-v2-pro"/"mimo-v2-flash"/"mimo-v2-omni" and set it to the intended ratio
(e.g., 0.05 or whatever agreed billing rate) so TTS requests are billed
correctly instead of using the 37.5 fallback.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 908406da-8855-4174-a987-6b95923119f2

📥 Commits

Reviewing files that changed from the base of the PR and between fbf235d and 49d2a99.

📒 Files selected for processing (11)
  • common/api_type.go
  • constant/api_type.go
  • constant/channel.go
  • model/pricing_default.go
  • relay/channel/xiaomi/adaptor.go
  • relay/channel/xiaomi/constants.go
  • relay/channel/xiaomi/tts.go
  • relay/common/relay_info.go
  • relay/relay_adaptor.go
  • setting/ratio_setting/model_ratio.go
  • web/src/constants/channel.constants.js

Comment thread relay/channel/xiaomi/adaptor.go Resolved
Comment thread relay/channel/xiaomi/tts.go Resolved
Comment thread setting/ratio_setting/model_ratio.go Resolved
@jianglizi

Copy link
Copy Markdown

何时合并?mimo-v2.5-pro都出来了

@Oniokey

Oniokey commented Apr 27, 2026

Copy link
Copy Markdown

关注,Xiaomi MiMo 有新模型了 V2.5 Pro

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

♻️ Duplicate comments (1)
relay/channel/xiaomi/adaptor.go (1)

61-62: ⚠️ Potential issue | 🟠 Major

Return an explicit error for unsupported rerank requests.

This still reports a successful conversion for a feature the adaptor does not implement, which pushes an empty payload further downstream instead of failing fast.

🔧 Proposed fix
 func (a *Adaptor) ConvertRerankRequest(c *gin.Context, relayMode int, request dto.RerankRequest) (any, error) {
-	return nil, nil
+	return nil, errors.New("xiaomi does not support rerank")
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@relay/channel/xiaomi/adaptor.go` around lines 61 - 62, The
ConvertRerankRequest method in Adaptor currently returns (nil, nil) for
unimplemented rerank behavior; change it to fail fast by returning a descriptive
error instead of nil error so callers know the feature is unsupported. Locate
function ConvertRerankRequest on type Adaptor and replace the nil,nil return
with a proper error (e.g., using fmt.Errorf or errors.New) like "rerank not
supported by Xiaomi adaptor" and ensure the package imports fmt or errors as
needed.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@dto/openai_request.go`:
- Line 280: GeneralOpenAIRequest.GetTokenCountMeta currently only iterates
Message.Content, so Message.Audio is ignored; update GetTokenCountMeta to detect
and include Message.Audio (json.RawMessage) in the TokenCountMeta.Files
accounting the same way as file-like content is counted: when a Message has a
non-nil/empty Audio field, add an entry to TokenCountMeta.Files (or increment
the files count/size) and ensure any filename/size extraction logic used for
other file fields is applied to Message.Audio so replayed assistant-audio
messages are included in token estimation and billing.

In `@relay/channel/xiaomi/adaptor.go`:
- Around line 69-86: ConvertAudioRequest currently converts any /v1/audio/speech
call into a MiMo TTS request; add a gate to reject non-TTS Xiaomi models. In
ConvertAudioRequest, before building mimoReq (and before normalizing/assigning
voice), validate request.Model against the Xiaomi TTS model whitelist (implement
or call a helper like isMimoTTSModel or add a supportedTTSModels set) and return
an explicit error (e.g., "unsupported Xiaomi TTS model") when not supported;
keep existing logic for shouldDefaultMimoVoice/isMimoVoiceDesignModel and
audioFormat assignment only after the model passes this check.

In `@relay/channel/xiaomi/tts.go`:
- Around line 86-90: The usage object returned from Xiaomi TTS must include the
audio-specific token details so relay/audio_handler.go routes to
PostAudioConsumeQuota; in relay/channel/xiaomi/tts.go where you build usage =
&dto.Usage{...}, also populate the audio token detail field (e.g.
usage.AudioTokens = &dto.AudioTokens{PromptTokens: mimoResp.Usage.PromptTokens,
CompletionTokens: mimoResp.Usage.CompletionTokens, TotalTokens:
mimoResp.Usage.TotalTokens} or set the corresponding
AudioPrompt/AudioCompletion/AudioTotal fields if your DTO uses different names)
so AudioTokens is non-nil and contains the token counts before returning.

---

Duplicate comments:
In `@relay/channel/xiaomi/adaptor.go`:
- Around line 61-62: The ConvertRerankRequest method in Adaptor currently
returns (nil, nil) for unimplemented rerank behavior; change it to fail fast by
returning a descriptive error instead of nil error so callers know the feature
is unsupported. Locate function ConvertRerankRequest on type Adaptor and replace
the nil,nil return with a proper error (e.g., using fmt.Errorf or errors.New)
like "rerank not supported by Xiaomi adaptor" and ensure the package imports fmt
or errors as needed.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 2b31f995-824d-4095-8e36-b19710900463

📥 Commits

Reviewing files that changed from the base of the PR and between b1e2988 and fafb5e5.

📒 Files selected for processing (7)
  • dto/openai_audio_response_test.go
  • dto/openai_request.go
  • relay/channel/xiaomi/adaptor.go
  • relay/channel/xiaomi/adaptor_test.go
  • relay/channel/xiaomi/constants.go
  • relay/channel/xiaomi/tts.go
  • relay/channel/xiaomi/tts_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • relay/channel/xiaomi/constants.go

Comment thread dto/openai_request.go
type Message struct {
Role string `json:"role"`
Content any `json:"content"`
Audio json.RawMessage `json:"audio,omitempty"`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Count message.audio in token metadata.

GeneralOpenAIRequest.GetTokenCountMeta() only walks Message.Content. With Message.Audio now accepted on the shared DTO, replayed assistant-audio messages bypass TokenCountMeta.Files and token estimation entirely, so this path is currently undercounted for quota/billing.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@dto/openai_request.go` at line 280, GeneralOpenAIRequest.GetTokenCountMeta
currently only iterates Message.Content, so Message.Audio is ignored; update
GetTokenCountMeta to detect and include Message.Audio (json.RawMessage) in the
TokenCountMeta.Files accounting the same way as file-like content is counted:
when a Message has a non-nil/empty Audio field, add an entry to
TokenCountMeta.Files (or increment the files count/size) and ensure any
filename/size extraction logic used for other file fields is applied to
Message.Audio so replayed assistant-audio messages are included in token
estimation and billing.

Comment on lines +69 to +86
func (a *Adaptor) ConvertAudioRequest(c *gin.Context, info *relaycommon.RelayInfo, request dto.AudioRequest) (io.Reader, error) {
if info.RelayMode != constant.RelayModeAudioSpeech {
return nil, errors.New("unsupported audio relay mode")
}
audioFormat := normalizeMimoAudioFormat(request.ResponseFormat)
voice := request.Voice
if shouldDefaultMimoVoice(request.Model) && voice == "" {
voice = "mimo_default"
}
mimoReq := xiaomiTTSRequest{
Model: request.Model,
Messages: buildMimoTTSMessages(request),
Audio: xiaomiTTSAudio{Voice: voice, Format: audioFormat},
}
if isMimoVoiceDesignModel(request.Model) {
mimoReq.Audio.Voice = ""
}
a.audioFormat = audioFormat

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Reject non-TTS Xiaomi models on the audio-speech path.

This conversion rewrites every Xiaomi /v1/audio/speech request into the MiMo chat+audio schema. If request.Model is not a supported TTS model, the adaptor sends an invalid upstream request instead of returning a clear local error. The bridge here should be gated to Xiaomi TTS models only.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@relay/channel/xiaomi/adaptor.go` around lines 69 - 86, ConvertAudioRequest
currently converts any /v1/audio/speech call into a MiMo TTS request; add a gate
to reject non-TTS Xiaomi models. In ConvertAudioRequest, before building mimoReq
(and before normalizing/assigning voice), validate request.Model against the
Xiaomi TTS model whitelist (implement or call a helper like isMimoTTSModel or
add a supportedTTSModels set) and return an explicit error (e.g., "unsupported
Xiaomi TTS model") when not supported; keep existing logic for
shouldDefaultMimoVoice/isMimoVoiceDesignModel and audioFormat assignment only
after the model passes this check.

Comment on lines +86 to +90
usage = &dto.Usage{
PromptTokens: mimoResp.Usage.PromptTokens,
CompletionTokens: mimoResp.Usage.CompletionTokens,
TotalTokens: mimoResp.Usage.TotalTokens,
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Populate audio token details before returning usage.

relay/audio_handler.go switches to PostAudioConsumeQuota only when the AudioTokens detail fields are set. This handler returns only the flat token counters, so Xiaomi /v1/audio/speech responses currently fall through the text-consumption path.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@relay/channel/xiaomi/tts.go` around lines 86 - 90, The usage object returned
from Xiaomi TTS must include the audio-specific token details so
relay/audio_handler.go routes to PostAudioConsumeQuota; in
relay/channel/xiaomi/tts.go where you build usage = &dto.Usage{...}, also
populate the audio token detail field (e.g. usage.AudioTokens =
&dto.AudioTokens{PromptTokens: mimoResp.Usage.PromptTokens, CompletionTokens:
mimoResp.Usage.CompletionTokens, TotalTokens: mimoResp.Usage.TotalTokens} or set
the corresponding AudioPrompt/AudioCompletion/AudioTotal fields if your DTO uses
different names) so AudioTokens is non-nil and contains the token counts before
returning.

@HynoR
HynoR marked this pull request as draft April 28, 2026 08:21
@HynoR

HynoR commented Apr 28, 2026

Copy link
Copy Markdown
Contributor Author

转 draft 测测 tts 没问题了再 open

@HynoR
HynoR marked this pull request as ready for review April 29, 2026 06:37
@HynoR HynoR closed this Apr 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Xiaomi MiMo mimo-v2-tts on OpenAI-compatible /v1/audio/speech

3 participants