Skip to content

perf(common): 批量复制 RawMessage,优化请求深拷贝 - #7221

Merged
Calcium-Ion merged 2 commits into
QuantumNous:mainfrom
mingisrookie:fix/responses-request-clone
Sep 6, 2026
Merged

Calcium-Ion merged 2 commits into
QuantumNous:mainfrom
mingisrookie:fix/responses-request-clone

Conversation

@mingisrookie

@mingisrookie mingisrookie commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

🔗 关联任务 / Related Issue

延续本 PR,无关联 issue。原作者 mingisrookie 在提交 3f4cca31 中定位并实现了 Responses 的 RawMessage 批量复制;本次由仓库 owner 确认范围后,以追加提交 2f3c5e0a 将优化统一到公共入口,保留原作者提交。代码与测试有 AI 辅助。

🚀 变更类型 / Type of change

  • 🐛 Bug 修复 (Bug fix)
  • ✨ 新功能 (New feature)
  • ⚡ 性能优化 / 重构 (Refactor)
  • 📝 文档更新 (Documentation)

📝 变更描述 / Description

当前 common.DeepCopy 使用 copier 对 RawMessage 逐字节反射复制,大请求会在转发上游前产生大量分配和 CPU 开销。现在为精确的 json.RawMessage → json.RawMessage 注册 converter,使用 bytes.Clone 复制到独立底层存储;保留 DeepCopy: true、IgnoreEmpty: true、原函数签名及错误处理。

Responses 恢复使用 common.DeepCopy,Chat、Claude、Gemini、Image、Audio、Embedding、Rerank 继续使用原有公共入口。最终相对基线只修改公共复制实现和一个集中维护的请求克隆测试文件。

复制包含调用前已有的修改;完成复制后,原请求与副本的原地修改、追加及字段替换互相隔离,后续渠道尝试仍从原请求复制。保留显式 false/0 指针、ThinkingBudget 及内部 ReasoningConversion,不执行 JSON 序列化往返,不改变克隆时机、模型映射或计费顺序。

兼容性边界:

  • 直接 RawMessage 字段、嵌套结构及类型明确的 RawMessage map/slice 元素可以命中优化;[]byte、*json.RawMessage 和装在 any 中的 RawMessage 继续走现有复制逻辑。
  • 保留源 RawMessage 的 nil/非 nil 空切片区别。这里有一处明确的行为修正:旧 copier 会把 map/slice 中的 nil RawMessage 转为空切片,新实现保留 nil,已有回归测试。
  • 不保证切片容量一致,不增加并发同步;复制时并发写入相同源内存仍由调用方负责同步。未修改 DTO、依赖或数据库行为。

📸 运行证明 / Proof of Work

本次验证基于追加提交的代码:Go 1.26.6、darwin/arm64、Apple M5 Pro、CGO_ENABLED=1。

通过的检查:

go test ./common ./relay/... -count=1
go test -race ./relay -run '^TestRequestDeepCopy' -count=1
go vet ./common ./relay/...
go test ./relay -run '^$' -bench '^BenchmarkRequestDeepCopy$' -benchmem -benchtime=1x -count=3
(cd relaykit && GOWORK=off go build ./...)
git diff --check
gofmt -l common/copy.go relay/responses_handler.go relay/request_clone_test.go

最后两项无输出。测试覆盖八种请求 DTO、嵌套 RawMessage 和动态内容、修改前后隔离、nil/空值、显式零值指针、内部状态、大输入以及连续渠道模型映射与重试。另用 Go overlay 将公共复制实现临时替换为旧版本,确认 map/slice 内 nil 保留用例在旧版本失败,在新版本通过;该对照不修改仓库文件。

以下为本次克隆微基准三次结果的范围;每次 benchtime=1x,小请求结果包含冷启动波动,不据此承诺稳定加速倍数:

输入 原反射复制 公共 RawMessage 优化 原版 allocs/op 优化后 allocs/op
256 B 48–204 µs 26–46 µs 276–284 22
4 KiB 246–260 µs 23–30 µs 4,116 22
1 MiB 54.1–55.4 ms 60.6–68.4 µs 1,048,596 22
10 MiB 538–544 ms 314–336 µs 10,485,780 22
40 MiB 2.103–2.124 s 1.077–1.119 ms 41,943,060–41,943,061 22

40 MiB 的累计分配量从约 83,888,400 B/op 降到 41,945,912 B/op;优化后分配次数不再随 RawMessage 字节数线性增长。256 B 小请求的分配字节数可能略增,未宣称所有尺寸的内存开销都下降。

原作者在初始 Responses 专用版本上报告过 Linux 隔离环境的完整请求实验:40 MB 非透传首字节约 19.4–19.6 s → 5.4–5.7 s,且两版上游请求体哈希一致。这是原作者对旧提交的历史结果;本次没有重跑完整 HTTP、真实供应商或真实模型 TTFT 实验,不能用该结果代替公共化后代码的端到端验证。

本次未运行全仓测试、真实前端或数据库矩阵;未修改数据库相关行为。远端 CI 状态以本 PR 当前提交的 Checks 为准。

✅ 提交前检查项 / Checklist

  • 人工确认: 等待 owner 对最终代码与描述进行人工审阅。
  • 非重复提交: 直接续改原 PR perf(common): 批量复制 RawMessage,优化请求深拷贝 #7221,保留原作者提交,未另建重复 PR。
  • 新功能关联 Issue: 不适用,本项为既有复制路径的性能优化。
  • 事前沟通: 仓库 owner 已确认公共 RawMessage 优化范围及实施计划。
  • 功能范围: 公共请求深拷贝优化,无 Coding Plan、逆向渠道或第三方封装接口改动。
  • 范围聚焦: 公共复制实现与集中回归测试,无依赖升级。
  • 本地验证: 上述测试、race、vet、独立构建和基准通过。
  • 安全合规: 未添加敏感凭据。

Summary by CodeRabbit

  • Bug Fixes

    • Improved request cloning to keep raw JSON data independent during mutations and retries.
    • Prevented changes to cloned request data from affecting the original request, across supported request types.
  • Tests

    • Added coverage for cloning behavior across chat, Claude, Gemini, image, audio, embedding, rerank, and response requests.
    • Added validation for optional, large, mapped, and mutable request data.

@coderabbitai

coderabbitai Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: ad008d38-c918-4cbd-9709-6b066dd8f7ce

📥 Commits

Reviewing files that changed from the base of the PR and between 3f4cca3 and 2f3c5e0.

📒 Files selected for processing (2)
  • common/copy.go
  • relay/request_clone_test.go

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


Walkthrough

DeepCopy now clones json.RawMessage values with bytes.Clone. New relay tests verify clone isolation across request protocols, retries, mutable fields, and input sizes. Benchmarks compare reflective copying with raw-byte deep copying.

Changes

Request cloning

Layer / File(s) Summary
Raw message clone implementation
common/copy.go
DeepCopy uses a dedicated json.RawMessage converter backed by bytes.Clone.
Clone isolation coverage
relay/request_clone_test.go
Tests cover Responses requests, retries, raw mutations, nested protocol DTOs, large inputs, and outbound JSON equality. Benchmarks compare reflective and raw-byte copying.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 2f3c5

该变更通过直接复制 RawMessage 字节存储来降低大请求的复制开销,同时保留请求重试和模型映射期间的隔离语义;当前没有已证实的合并阻塞风险。

Poem

A rabbit checks each byte in line
Raw messages clone and shine
Retries hop with copies true
Nested fields stay separate too
Across each protocol path
The burrow tests the clone craft

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 4 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: using batch RawMessage copying to improve request deep-copy performance. It is concise and directly related to the changeset.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Calcium-Ion Calcium-Ion self-assigned this Sep 5, 2026
@Calcium-Ion Calcium-Ion changed the title perf(relay): 批量复制 Responses 原始 JSON,减少大请求首响应延迟 perf(common): 批量复制 RawMessage,优化请求深拷贝 Sep 6, 2026
@Calcium-Ion
Calcium-Ion merged commit 2cf177a into QuantumNous:main Sep 6, 2026
1 check was pending
mrdjango added a commit to mrdjango/models-gateway that referenced this pull request Sep 8, 2026
* fix: temp disable /messages/count_tokens

* feat(relay): explicit @ model modifiers and canonical billing identity

Model-name post-processing is rebuilt around an explicit trailing
@key:value modifier syntax (thinking/effort/temperature/topp) that
overrides request fields, survives model mapping, and records
conversion diagnostics on the consume log.

- Legacy naked aliases (-thinking, -nothinking, -thinking-<budget>,
  effort tails) now parse only for positively matched families
  (gpt-*/o-series, claude-*, gemini-*, incl. vendor/ namespaces);
  names like qwen-max stay opaque. EffortTailModelIDs remains the
  escape hatch for real in-family IDs such as gpt-5.1-codex-max.
- Billing identity resolves once in ModelPriceHelper via a ladder:
  configured request name first (legacy wildcard entries intact), then
  canonical billing names rebuilt from parsed intent
  (base@effort:E@thinking:S, then base@thinking:S; order, duplicates,
  and budget values are irrelevant; temperature/topp never priced),
  then base. Routing and token limits fall back through
  RoutingMatchModelName; pricing lookups stay wildcard-only.
- Pass-through stays byte-identical: modifiers and aliases are neither
  parsed nor validated there and forward verbatim for the upstream
  (or a chained gateway) to interpret.
- Unknown modifier keys and invalid known-key values are rejected with
  400; models whose real names contain @tag:value are exempted via the
  thinking-suffix blacklist, which now supports re:-prefixed Go regex
  entries.
- Claude reasoning render coerces unsupported combinations (disable,
  adaptive, budgets) with warning diagnostics instead of erroring;
  native-protocol requests without host syntax pass through untouched.

BREAKING(openrouter): drop the host-invented "-thinking" model-name
alias (added in 4f6d16e) that trimmed any *-thinking model on
OpenRouter channels and injected reasoning.enabled. It matched too
broadly and mangled real model IDs such as kimi-k2-thinking.
Migration: use some-model@thinking:on, or keep the old public name via
a channel model mapping {"some-model-thinking": "some-model@thinking:on"}.
Claude/Gemini family aliases (incl. anthropic/claude-*-thinking) keep
working via the family whitelist.

* fix(relay): preserve reasoning effort without implicit remapping

* docs: require expression pricing and consolidated tests

* feat(billing): add built-in expression pricing for gpt-6-astra

* perf(common): 批量复制 RawMessage,优化请求深拷贝 (QuantumNous#7221)

* perf(relay): bulk-copy Responses raw JSON fields

* perf(common): share RawMessage deep-copy optimization

---------

Co-authored-by: CaIon <i@caion.me>

* fix(relay): apply model-specific OpenAI chat capabilities (QuantumNous#7211)

* fix(relay): treat gpt-5 and later generations alike for max_completion_tokens

IsOpenAIGPT5Model matched on the literal prefix "gpt-5", so gpt-6-astra
(and every generation after it) fell through the gpt-5 request rules:
max_tokens was forwarded as-is and the provider rejected it with
"Unsupported parameter: 'max_tokens' is not supported with this model.
Use 'max_completion_tokens' instead." The same gap left temperature,
top_p and logprobs untouched, each of which the provider also rejects,
and made the channel test button report a 400 for a healthy deployment.

Match on the major version instead (gpt-<n>... with n >= 5). Callers are
unchanged: ConvertOpenAIRequest, GetSystemRoleName, buildTestRequest and
the health check all go through this one helper. buildTestRequest now
sends max_completion_tokens for these models directly instead of relying
on the later conversion. gpt-4.1, gpt-4o, gpt-oss, gpt-image and
gpt-realtime names still do not match.

Verified against Azure OpenAI gpt-6-astra (2026-09-03): with the old
prefix max_tokens / temperature / top_p / logprobs each returned 400,
while gpt-5.6-luna with the same payload returned 200.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bwz8o5UeoRtrtDusKaayp

* fix(relay): separate OpenAI chat model compatibility rules

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: CaIon <i@caion.me>

* feat(security): add access token management and audit logs

Move account security settings into a dedicated page and add token status, rotation, revocation, and access history.

Store audit events with role snapshots and JSON metadata, add audit.read authorization and an independent audit page, and upgrade the ClickHouse driver to v2.46.0.

* fix(db): avoid redundant schema migrations on restart

Upgrade the SQLite GORM driver to correctly distinguish unique indexes from constraints. Normalize equivalent MySQL decimal defaults and PostgreSQL CHAR metadata during migration comparison while preserving real schema changes.

Validation: 27 fresh-start and rc.26 upgrade scenarios using SQLite 3.50.4, MySQL 5.7.44, PostgreSQL 9.6.24, and ClickHouse 25.8.33.6; 15 upgraded databases passed uniqueness checks. Relational databases issue no DDL on unchanged restarts. Existing ClickHouse TTL synchronization remains unchanged.

* fix(security): bind verification proofs to sessions and actions

Require single-use operation proofs for passkey enrollment, two-factor setup, and channel key access. Add password and OAuth verification flows, enforce session-bound enrollment, and redact OAuth callback secrets from logs.

Validation: affected Go packages pass; frontend typecheck, changed-file lint, and 111 tests pass. Security enrollment regressions pass on SQLite 3.50.4, MySQL 8.0.46, and PostgreSQL 16.15. Full frontend lint has pre-existing errors outside the changed files.

* feat(auth): migrate Telegram to unified OAuth

Use authorization code flow with PKCE and verified ID tokens for Telegram login, binding, and security verification. Preserve existing bindings and require administrator OAuth configuration.

Keep the restricted WeChat first-enrollment session proof, fix missing-target authentication errors, and preserve callback requests after OAuth popups close.

* feat(security): require verification for access token management

* feat(security): harden account binding and password changes

Require scoped, single-use verification for account bindings and password
operations. Bind OAuth authorization and email confirmations to the initiating
session; preserve the last usable login method and audit operation outcomes.

Apply Unicode-aware password length limits, Argon2id writes with bcrypt
compatibility, and long-password encryption.
Return has_password with the existing profile SELECT without extra queries.
Reuse the existing security dialogs and add all seven locale translations.

Validation:
- Go: go test ./common ./model ./service ./middleware ./controller ./router -count=1
- DB: SQLite 3.50.4, MySQL 8.4.11, PostgreSQL 16.15; separate main/log databases
- MySQL/PostgreSQL: TEST_SECURITY_DIALECT=<dialect> with TEST_<DIALECT>_DSN,
  go test ./controller -run '^(TestSecurityAccount|TestSecurityEnrollment|TestGenerateOAuthCode|TestOAuthBind|TestTelegramOAuth)' -count=1 -v
- Web: relevant Vitest suites, bun run typecheck, targeted oxlint/format,
  bun run i18n:sync, and bun run build

Roll out dual-format readers to every instance with
ACCOUNT_PASSWORD_HASH_ALGORITHM=bcrypt before enabling Argon2id writes
and the new UI. Rollbacks must retain Argon2id and v2 envelope readers.

Relevant controls: ASVS 5.0.0 6.2.1-6.2.3, 6.2.5-6.2.9, 6.3.7, 7.4.3, 7.5.1;
this change does not assert application-wide ASVS certification.

* feat(audit): complete token and quota operation records

Record successful and failed API token operations with safe target metadata. Capture quota adjustments in a transaction, synchronize committed cache differences, and correlate audit and top-up records.

Show operation targets, changes, quota balances, and failure details consistently across audit and usage logs, with translations for all seven locales.

Validated controller, middleware, and model tests; 78 frontend tests; typecheck and lint; real SQLite 3.50.4, MySQL 8.4.11, and PostgreSQL 16.15 with shared and separate log databases.

* fix(dashboard): simplify completed setup guide

* feat(auth): unify login verification and secure account deletion

Treat TOTP and Passkey as alternative enrolled factors across login and
sensitive account operations. Gate every primary login transport before
issuing a session, require WebAuthn user verification, and consume login
challenges atomically with session creation.

Reuse the shared verification UI for login, 2FA management, and account
deletion. Require scoped, single-use deletion proof; recheck the session
inside the deletion transaction and revoke all sessions afterward.

Validation: controller/service/model/middleware tests; real SQLite 3.50.4,
MySQL 8.4.11, and PostgreSQL 16.15 security regressions; frontend tests,
TypeScript, targeted lint, formatting, and production build.

Deploy the frontend and all backend nodes together. No schema changes.

* feat(models): rework model/vendor management and pricing

Introduce a unified model management experience: catalog metadata
validation, vendor management, batch delete with channel/pricing
cleanup, model pricing snapshot editing with optimistic concurrency,
and an upstream ratio-sync flow with price cells. Move configuration
into dedicated pricing config/metadata-sync/vendor-management backend
services and add audit records for model/vendor/pricing mutations.

Rework the models page around vendors and model connections, add
model-pricing and vendor-management dialogs, and replace the shared
Select usages with the Combobox component across subscriptions,
plugins, OAuth presets, audit filters, and settings. Add the model
pricing panel and verify behavior with focused tests.

* feat(ali): support wan3.0 all-in-one video models

* feat(ali): support wan3.0 all-in-one video models

Extend the Alibaba task plugin for wan3.0-video / wan3.0-video-prime
using DashScope media, resolution/ratio, duration rules, and billing ratios.

* fix(ali): make wan3.0 smart duration billable and alias-safe

Review fixes on top of QuantumNous#7240 (qiuliw):

- duration -1 never reached the plugin: the host rejects negative
  canonical duration/seconds facts before any hook runs, on every
  entry point. Decoders now normalize -1 into an auto_duration marker;
  convert emits -1 upstream and bills 30s up front; non-wan3.0 models
  reject the marker instead of silently defaulting to 5s.
- extractUsageOnComplete read output.duration/output.resolution, but
  wan3.0 reports usage.output_video_duration and numeric usage.SR, so
  smart-duration and resolution settlement never reconciled. Read the
  usage block first, keep the legacy output fields as fallback.
- convert keyed default resolution on the client model name, so a
  channel-mapped alias fell to 720P while the direct request got 1080P.
  Every model-shaped decision in convert now uses ctx.upstreamModel.
- Unknown wan3.0 size values were silently coerced to 1080P; reject them.
- Image-only openai_responses input was accepted for every model,
  regressing t2v models into pre-consume then upstream rejection.
  Restore the guard and allow image-only for i2v and wan3.0 only.
- Native passthrough dropped wan3.0 parameters (ratio, audio) unless
  media was present; forward them via metadata for wan3.0.
- Hoist the triplicated size-to-resolution map; add wan3.0 contract tests.

---------

Co-authored-by: qiuliw <a1807191473@qgmail.com>

* fix(relay): preserve Kimi K3 dynamic tool loading messages

Kimi K3 injects tools mid-conversation via a system message that carries
a `tools` array. `dto.Message` had no such field, so the tools were
silently dropped during the parse/re-marshal round trip and the upstream
rejected the request with `'tool_choice'='required' requires a 'tools'
field`.

- add `Message.Tools` (json.RawMessage passthrough)
- omit the `content` key only for tool-loading messages with nil content,
  as Kimi rejects `tools` next to `content`; all other messages keep
  emitting `"content": null`
- count message-level tools in token estimation
- skip tool-loading messages in channel system prompt injection and make
  the compatible handler reuse applySystemPromptIfNeeded
- add kimi-k3 to the moonshot model list

Fixes QuantumNous#7235

* refactor(json): route JSON helpers through a host-injectable codec

`common/json.go` and `relaykit/relayconvert/kitutil/json.go` were two
hard-wired copies of the same encoding/json wrapper, so swapping the JSON
engine required editing both modules.

- kitutil defines a `Codec` interface with a standard-library default and
  a `SetCodec` hook, mirroring the existing SetLogging host hook; every
  kitutil JSON helper and relaykit DTO (un)marshal method goes through it
- `common/json.go` forwards to kitutil and injects `hostJSONCodec` from
  init() so tests run on the same engine as production; swapping the
  engine now touches only this type in the root module
- route the remaining direct encoding/json calls inside relaykit
  (dto/values.go, responses stream validation) through kitutil
- add a codec routing test and a host codec conformance test locking the
  encoding semantics the DTOs depend on

Direct encoding/json call sites in the root module are left for a
separate cleanup.

* fix(web): keep drawer popups interactive and shim storage in tests

- Portal Combobox and Select popups into the vaul DrawerContent via a
  portal-container context so they stay inside the Radix modal layer
  instead of inheriting body pointer-events: none
- Provide an in-memory localStorage/sessionStorage in test-setup when the
  Node 25+ global accessor resolves to undefined and shadows jsdom
- Add regression tests for popups rendered inside and outside the drawer

* fix(perf): return hourly success-rate series for model health bar

The perf summary API only returned the last three non-empty buckets as
bare success rates, so the 24-slot status strip on model cards never lit
more than three bars and could not show hours without traffic.

Replace recent_success_rates with recent_success_series: one timestamped
point per hour that had requests, aligned to hour start regardless of
the configured bucket size. The badge now anchors its 24 slots to the
client's current hour and places each point by timestamp, leaving hours
without data gray.

* fix(plugin): suppress factory layer when disabling an overridden task plugin

Disabling a task plugin that has both a factory built-in and an override
row only flipped the override flag, so the built-in kept routing the same
models and same-name uploads (e.g. minimax-h3 vs MiniMax-H3) still hit a
routing conflict. Now the key also enters the disabled-factory set and the
list reports "disabled" instead of "disabled_fallback" when nothing serves.

* fix(plugins): make sunoapi alias-safe and lock alias echo across built-ins

Decode on ctx.upstreamModel || ctx.model and echo ctx.model; fix the
lyrics/music render branch that read a nonexistent ctx.requestBody.model;
stop sending empty Accept/Content-Type. Bump sunoapi to 1.0.2. Add an
alias-echo table test covering every built-in.

* refactor(task): remove the custom-plugin layer switch

TaskPluginOverrideEnabled had no UI since the master switch landed, yet
when left off it marked every third-party plugin "disabled; platform
unavailable" regardless of its own toggle. Drop the option, env var,
registry flag, and the dead branch in ListTaskPlugins; the master switch
and per-plugin toggles are the only two levels now.

* fix(web): switch the pricing card grid to three columns at xl

The model card grid and its loading skeleton only reached three columns at
2xl, leaving a two-column layout on common desktop widths. Use the xl
breakpoint for both and lock it with a test.

* feat(pricing): support site currency in pricing editors

* feat(plugins): extend plugin metadata and icon support

* feat(web): improve plugin management and marketplace

* feat(channels): improve plugin channel setup and icons

* feat(pricing): improve pricing editors and log display

* feat(models): improve model listing, pricing and visibility filters

Include configured channel models without creating metadata, derive square
visibility from live routes and metadata policy, and filter before pagination.

Share pricing display with the model square, show expression tiers and task
unit prices, preserve zero rates, and expose full pricing and visibility
reasons from compact responsive rows. Complete all seven locale translations.

Validated frontend tests, typecheck, lint and production build, plus the
model database matrix on SQLite 3.50.4, MySQL 5.7.44 and PostgreSQL 9.6.24.

* feat(usage-logs): refine mobile layout and keep quick actions visible

* feat(web): refine API key and user quota displays

Show remaining and used API key quota with a progress bar, and use consistent mobile cards, group multiplier badges, and activity timestamps. Present available user balance with used quota underneath and translate the new labels.

Resolve full API keys only for explicit copy or chat actions. Reviewed OWASP Authentication and Session Management guidance and ASVS 5.0.0 V14.2.6 and V8.3.1; backend authorization is unchanged, and regression tests cover refused and denied key resolution. This frontend change does not assert application-wide ASVS compliance.

Validation: 55 related component tests passed; the latest mobile group and quota changes passed 35 focused tests. TypeScript, scoped lint, formatting, production build, and git diff checks passed. Responsive previews verified narrow screens and finite, unlimited, exhausted, and inactive quota states.

* feat(usage-logs): add searchable group filter

Reuse the shared Combobox to suggest groups for the active log view while allowing historical group names to be entered manually. Exclude the automatic routing pseudo-group from suggestions and preserve masking, reset, URL navigation, and mobile drawer behavior.

Extend the existing Combobox with keyboard event forwarding and accessible labels, preserve the selected custom value on focus, and close suggestions on blur. Enter confirms a selection before submitting the filter.

Validation: 25 tests passed across the shared Combobox and log group, type, and mobile filter suites. TypeScript, scoped lint, formatting, and git diff checks passed.

* fix(keys): show desktop quota amounts side by side

Place remaining quota on the left and used quota on the right above the progress bar, without visible labels in desktop rows. Preserve accessible descriptions and the existing mobile labels and stacked layout.

Validation: updated regressions failed before the change and all 26 API key listing tests passed afterward. TypeScript, scoped lint, formatting, production build, and git diff checks passed.

* style(keys): widen the desktop quota column

Increase the quota column width and minimum width from 220px to 260px to give the side-by-side amounts more room.

Validation: all 26 API key listing tests, TypeScript, scoped lint, formatting, and git diff checks passed.

* fix(keys): preserve spacing after desktop quota content

Bound desktop quota content to 180px so column width provides whitespace before the group column. Remove the quota-only padding override to use the same cell padding as other columns. Preserve the mobile quota layout.

Validation: layout regressions failed before the fix and all 26 API key listing tests passed afterward. TypeScript, scoped lint, formatting, production build, and git diff checks passed.

* feat(redemptions): add batch deletion and optional file exports

Add confirmed multi-select deletion through one batch API request and one
soft-delete statement. Record the affected count and requested IDs in a
separate batch audit event, and identify legacy events with missing counts.

After creation, offer an unchecked Save as a file option with TXT/Markdown
formats and optional name/quota fields. Keep Done as the default completion
action. Include translations for all seven frontend locales.

Validation:
- Redemption and audit frontend regression tests, typecheck, and scoped lint.
- go build ./...
- go test ./controller -run '^TestDeleteRedemptionBatch$' -count=1 -v
  with TEST_MYSQL_DSN, TEST_MYSQL_LOG_DSN, TEST_POSTGRES_DSN,
  and TEST_POSTGRES_LOG_DSN set to isolated primary and log databases.
- Real SQLite 3.50.4, MySQL 8.4.11, and PostgreSQL 16.15 passed, including
  deletion of 15 records, duplicate/missing IDs, zero-row retries, audit
  deduplication, invalid input, and exclusion of credentials from logs.

* docs: update project architecture and Go conventions

* refactor: modernize Go code conventions

* refactor(web): unify table cells and quota details

* test(dashboard): re-query the setup guide toggle while the overview settles

"does not show a completed setup entry when the key lookup fails" failed
on both CI runs of this PR, and intermittently in local full-suite runs,
while passing whenever the file ran alone.

The assertion held the node returned by findByRole across an async
boundary. The guide subtree is remounted while the remaining overview
queries settle, which detaches the captured element, so toBeVisible saw
a detached node — the reported element had aria-expanded="true" but no
children, which is what an unmounted subtree leaves behind.

Re-query inside waitFor so a transient remount cannot fail the
assertion. A guide that never expands still fails, so the test keeps its
meaning, and this follows web/AGENTS.md: async tests wait for an
explicit UI state rather than a captured node.

Verified: the file passes alone and the full suite passes 995/995.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: CaIon <i@caion.me>
Co-authored-by: mingisrookie <99650899+mingisrookie@users.noreply.github.com>
Co-authored-by: PDMaker <86941825+pdmaker@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: qiuliw <a1807191473@qgmail.com>
mrdjango added a commit to mrdjango/models-gateway that referenced this pull request Sep 9, 2026
* fix: temp disable /messages/count_tokens

* feat(relay): explicit @ model modifiers and canonical billing identity

Model-name post-processing is rebuilt around an explicit trailing
@key:value modifier syntax (thinking/effort/temperature/topp) that
overrides request fields, survives model mapping, and records
conversion diagnostics on the consume log.

- Legacy naked aliases (-thinking, -nothinking, -thinking-<budget>,
  effort tails) now parse only for positively matched families
  (gpt-*/o-series, claude-*, gemini-*, incl. vendor/ namespaces);
  names like qwen-max stay opaque. EffortTailModelIDs remains the
  escape hatch for real in-family IDs such as gpt-5.1-codex-max.
- Billing identity resolves once in ModelPriceHelper via a ladder:
  configured request name first (legacy wildcard entries intact), then
  canonical billing names rebuilt from parsed intent
  (base@effort:E@thinking:S, then base@thinking:S; order, duplicates,
  and budget values are irrelevant; temperature/topp never priced),
  then base. Routing and token limits fall back through
  RoutingMatchModelName; pricing lookups stay wildcard-only.
- Pass-through stays byte-identical: modifiers and aliases are neither
  parsed nor validated there and forward verbatim for the upstream
  (or a chained gateway) to interpret.
- Unknown modifier keys and invalid known-key values are rejected with
  400; models whose real names contain @tag:value are exempted via the
  thinking-suffix blacklist, which now supports re:-prefixed Go regex
  entries.
- Claude reasoning render coerces unsupported combinations (disable,
  adaptive, budgets) with warning diagnostics instead of erroring;
  native-protocol requests without host syntax pass through untouched.

BREAKING(openrouter): drop the host-invented "-thinking" model-name
alias (added in 4f6d16e) that trimmed any *-thinking model on
OpenRouter channels and injected reasoning.enabled. It matched too
broadly and mangled real model IDs such as kimi-k2-thinking.
Migration: use some-model@thinking:on, or keep the old public name via
a channel model mapping {"some-model-thinking": "some-model@thinking:on"}.
Claude/Gemini family aliases (incl. anthropic/claude-*-thinking) keep
working via the family whitelist.

* fix(relay): preserve reasoning effort without implicit remapping

* docs: require expression pricing and consolidated tests

* feat(billing): add built-in expression pricing for gpt-6-astra

* perf(common): 批量复制 RawMessage,优化请求深拷贝 (QuantumNous#7221)

* perf(relay): bulk-copy Responses raw JSON fields

* perf(common): share RawMessage deep-copy optimization

---------

Co-authored-by: CaIon <i@caion.me>

* fix(relay): apply model-specific OpenAI chat capabilities (QuantumNous#7211)

* fix(relay): treat gpt-5 and later generations alike for max_completion_tokens

IsOpenAIGPT5Model matched on the literal prefix "gpt-5", so gpt-6-astra
(and every generation after it) fell through the gpt-5 request rules:
max_tokens was forwarded as-is and the provider rejected it with
"Unsupported parameter: 'max_tokens' is not supported with this model.
Use 'max_completion_tokens' instead." The same gap left temperature,
top_p and logprobs untouched, each of which the provider also rejects,
and made the channel test button report a 400 for a healthy deployment.

Match on the major version instead (gpt-<n>... with n >= 5). Callers are
unchanged: ConvertOpenAIRequest, GetSystemRoleName, buildTestRequest and
the health check all go through this one helper. buildTestRequest now
sends max_completion_tokens for these models directly instead of relying
on the later conversion. gpt-4.1, gpt-4o, gpt-oss, gpt-image and
gpt-realtime names still do not match.

Verified against Azure OpenAI gpt-6-astra (2026-09-03): with the old
prefix max_tokens / temperature / top_p / logprobs each returned 400,
while gpt-5.6-luna with the same payload returned 200.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bwz8o5UeoRtrtDusKaayp

* fix(relay): separate OpenAI chat model compatibility rules

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: CaIon <i@caion.me>

* feat(security): add access token management and audit logs

Move account security settings into a dedicated page and add token status, rotation, revocation, and access history.

Store audit events with role snapshots and JSON metadata, add audit.read authorization and an independent audit page, and upgrade the ClickHouse driver to v2.46.0.

* fix(db): avoid redundant schema migrations on restart

Upgrade the SQLite GORM driver to correctly distinguish unique indexes from constraints. Normalize equivalent MySQL decimal defaults and PostgreSQL CHAR metadata during migration comparison while preserving real schema changes.

Validation: 27 fresh-start and rc.26 upgrade scenarios using SQLite 3.50.4, MySQL 5.7.44, PostgreSQL 9.6.24, and ClickHouse 25.8.33.6; 15 upgraded databases passed uniqueness checks. Relational databases issue no DDL on unchanged restarts. Existing ClickHouse TTL synchronization remains unchanged.

* fix(security): bind verification proofs to sessions and actions

Require single-use operation proofs for passkey enrollment, two-factor setup, and channel key access. Add password and OAuth verification flows, enforce session-bound enrollment, and redact OAuth callback secrets from logs.

Validation: affected Go packages pass; frontend typecheck, changed-file lint, and 111 tests pass. Security enrollment regressions pass on SQLite 3.50.4, MySQL 8.0.46, and PostgreSQL 16.15. Full frontend lint has pre-existing errors outside the changed files.

* feat(auth): migrate Telegram to unified OAuth

Use authorization code flow with PKCE and verified ID tokens for Telegram login, binding, and security verification. Preserve existing bindings and require administrator OAuth configuration.

Keep the restricted WeChat first-enrollment session proof, fix missing-target authentication errors, and preserve callback requests after OAuth popups close.

* feat(security): require verification for access token management

* feat(security): harden account binding and password changes

Require scoped, single-use verification for account bindings and password
operations. Bind OAuth authorization and email confirmations to the initiating
session; preserve the last usable login method and audit operation outcomes.

Apply Unicode-aware password length limits, Argon2id writes with bcrypt
compatibility, and long-password encryption.
Return has_password with the existing profile SELECT without extra queries.
Reuse the existing security dialogs and add all seven locale translations.

Validation:
- Go: go test ./common ./model ./service ./middleware ./controller ./router -count=1
- DB: SQLite 3.50.4, MySQL 8.4.11, PostgreSQL 16.15; separate main/log databases
- MySQL/PostgreSQL: TEST_SECURITY_DIALECT=<dialect> with TEST_<DIALECT>_DSN,
  go test ./controller -run '^(TestSecurityAccount|TestSecurityEnrollment|TestGenerateOAuthCode|TestOAuthBind|TestTelegramOAuth)' -count=1 -v
- Web: relevant Vitest suites, bun run typecheck, targeted oxlint/format,
  bun run i18n:sync, and bun run build

Roll out dual-format readers to every instance with
ACCOUNT_PASSWORD_HASH_ALGORITHM=bcrypt before enabling Argon2id writes
and the new UI. Rollbacks must retain Argon2id and v2 envelope readers.

Relevant controls: ASVS 5.0.0 6.2.1-6.2.3, 6.2.5-6.2.9, 6.3.7, 7.4.3, 7.5.1;
this change does not assert application-wide ASVS certification.

* feat(audit): complete token and quota operation records

Record successful and failed API token operations with safe target metadata. Capture quota adjustments in a transaction, synchronize committed cache differences, and correlate audit and top-up records.

Show operation targets, changes, quota balances, and failure details consistently across audit and usage logs, with translations for all seven locales.

Validated controller, middleware, and model tests; 78 frontend tests; typecheck and lint; real SQLite 3.50.4, MySQL 8.4.11, and PostgreSQL 16.15 with shared and separate log databases.

* fix(dashboard): simplify completed setup guide

* feat(auth): unify login verification and secure account deletion

Treat TOTP and Passkey as alternative enrolled factors across login and
sensitive account operations. Gate every primary login transport before
issuing a session, require WebAuthn user verification, and consume login
challenges atomically with session creation.

Reuse the shared verification UI for login, 2FA management, and account
deletion. Require scoped, single-use deletion proof; recheck the session
inside the deletion transaction and revoke all sessions afterward.

Validation: controller/service/model/middleware tests; real SQLite 3.50.4,
MySQL 8.4.11, and PostgreSQL 16.15 security regressions; frontend tests,
TypeScript, targeted lint, formatting, and production build.

Deploy the frontend and all backend nodes together. No schema changes.

* feat(models): rework model/vendor management and pricing

Introduce a unified model management experience: catalog metadata
validation, vendor management, batch delete with channel/pricing
cleanup, model pricing snapshot editing with optimistic concurrency,
and an upstream ratio-sync flow with price cells. Move configuration
into dedicated pricing config/metadata-sync/vendor-management backend
services and add audit records for model/vendor/pricing mutations.

Rework the models page around vendors and model connections, add
model-pricing and vendor-management dialogs, and replace the shared
Select usages with the Combobox component across subscriptions,
plugins, OAuth presets, audit filters, and settings. Add the model
pricing panel and verify behavior with focused tests.

* feat(ali): support wan3.0 all-in-one video models

* feat(ali): support wan3.0 all-in-one video models

Extend the Alibaba task plugin for wan3.0-video / wan3.0-video-prime
using DashScope media, resolution/ratio, duration rules, and billing ratios.

* fix(ali): make wan3.0 smart duration billable and alias-safe

Review fixes on top of QuantumNous#7240 (qiuliw):

- duration -1 never reached the plugin: the host rejects negative
  canonical duration/seconds facts before any hook runs, on every
  entry point. Decoders now normalize -1 into an auto_duration marker;
  convert emits -1 upstream and bills 30s up front; non-wan3.0 models
  reject the marker instead of silently defaulting to 5s.
- extractUsageOnComplete read output.duration/output.resolution, but
  wan3.0 reports usage.output_video_duration and numeric usage.SR, so
  smart-duration and resolution settlement never reconciled. Read the
  usage block first, keep the legacy output fields as fallback.
- convert keyed default resolution on the client model name, so a
  channel-mapped alias fell to 720P while the direct request got 1080P.
  Every model-shaped decision in convert now uses ctx.upstreamModel.
- Unknown wan3.0 size values were silently coerced to 1080P; reject them.
- Image-only openai_responses input was accepted for every model,
  regressing t2v models into pre-consume then upstream rejection.
  Restore the guard and allow image-only for i2v and wan3.0 only.
- Native passthrough dropped wan3.0 parameters (ratio, audio) unless
  media was present; forward them via metadata for wan3.0.
- Hoist the triplicated size-to-resolution map; add wan3.0 contract tests.

---------

Co-authored-by: qiuliw <a1807191473@qgmail.com>

* fix(relay): preserve Kimi K3 dynamic tool loading messages

Kimi K3 injects tools mid-conversation via a system message that carries
a `tools` array. `dto.Message` had no such field, so the tools were
silently dropped during the parse/re-marshal round trip and the upstream
rejected the request with `'tool_choice'='required' requires a 'tools'
field`.

- add `Message.Tools` (json.RawMessage passthrough)
- omit the `content` key only for tool-loading messages with nil content,
  as Kimi rejects `tools` next to `content`; all other messages keep
  emitting `"content": null`
- count message-level tools in token estimation
- skip tool-loading messages in channel system prompt injection and make
  the compatible handler reuse applySystemPromptIfNeeded
- add kimi-k3 to the moonshot model list

Fixes QuantumNous#7235

* refactor(json): route JSON helpers through a host-injectable codec

`common/json.go` and `relaykit/relayconvert/kitutil/json.go` were two
hard-wired copies of the same encoding/json wrapper, so swapping the JSON
engine required editing both modules.

- kitutil defines a `Codec` interface with a standard-library default and
  a `SetCodec` hook, mirroring the existing SetLogging host hook; every
  kitutil JSON helper and relaykit DTO (un)marshal method goes through it
- `common/json.go` forwards to kitutil and injects `hostJSONCodec` from
  init() so tests run on the same engine as production; swapping the
  engine now touches only this type in the root module
- route the remaining direct encoding/json calls inside relaykit
  (dto/values.go, responses stream validation) through kitutil
- add a codec routing test and a host codec conformance test locking the
  encoding semantics the DTOs depend on

Direct encoding/json call sites in the root module are left for a
separate cleanup.

* fix(web): keep drawer popups interactive and shim storage in tests

- Portal Combobox and Select popups into the vaul DrawerContent via a
  portal-container context so they stay inside the Radix modal layer
  instead of inheriting body pointer-events: none
- Provide an in-memory localStorage/sessionStorage in test-setup when the
  Node 25+ global accessor resolves to undefined and shadows jsdom
- Add regression tests for popups rendered inside and outside the drawer

* fix(perf): return hourly success-rate series for model health bar

The perf summary API only returned the last three non-empty buckets as
bare success rates, so the 24-slot status strip on model cards never lit
more than three bars and could not show hours without traffic.

Replace recent_success_rates with recent_success_series: one timestamped
point per hour that had requests, aligned to hour start regardless of
the configured bucket size. The badge now anchors its 24 slots to the
client's current hour and places each point by timestamp, leaving hours
without data gray.

* fix(plugin): suppress factory layer when disabling an overridden task plugin

Disabling a task plugin that has both a factory built-in and an override
row only flipped the override flag, so the built-in kept routing the same
models and same-name uploads (e.g. minimax-h3 vs MiniMax-H3) still hit a
routing conflict. Now the key also enters the disabled-factory set and the
list reports "disabled" instead of "disabled_fallback" when nothing serves.

* fix(plugins): make sunoapi alias-safe and lock alias echo across built-ins

Decode on ctx.upstreamModel || ctx.model and echo ctx.model; fix the
lyrics/music render branch that read a nonexistent ctx.requestBody.model;
stop sending empty Accept/Content-Type. Bump sunoapi to 1.0.2. Add an
alias-echo table test covering every built-in.

* refactor(task): remove the custom-plugin layer switch

TaskPluginOverrideEnabled had no UI since the master switch landed, yet
when left off it marked every third-party plugin "disabled; platform
unavailable" regardless of its own toggle. Drop the option, env var,
registry flag, and the dead branch in ListTaskPlugins; the master switch
and per-plugin toggles are the only two levels now.

* fix(web): switch the pricing card grid to three columns at xl

The model card grid and its loading skeleton only reached three columns at
2xl, leaving a two-column layout on common desktop widths. Use the xl
breakpoint for both and lock it with a test.

* feat(pricing): support site currency in pricing editors

* feat(plugins): extend plugin metadata and icon support

* feat(web): improve plugin management and marketplace

* feat(channels): improve plugin channel setup and icons

* feat(pricing): improve pricing editors and log display

* feat(models): improve model listing, pricing and visibility filters

Include configured channel models without creating metadata, derive square
visibility from live routes and metadata policy, and filter before pagination.

Share pricing display with the model square, show expression tiers and task
unit prices, preserve zero rates, and expose full pricing and visibility
reasons from compact responsive rows. Complete all seven locale translations.

Validated frontend tests, typecheck, lint and production build, plus the
model database matrix on SQLite 3.50.4, MySQL 5.7.44 and PostgreSQL 9.6.24.

* feat(usage-logs): refine mobile layout and keep quick actions visible

* feat(web): refine API key and user quota displays

Show remaining and used API key quota with a progress bar, and use consistent mobile cards, group multiplier badges, and activity timestamps. Present available user balance with used quota underneath and translate the new labels.

Resolve full API keys only for explicit copy or chat actions. Reviewed OWASP Authentication and Session Management guidance and ASVS 5.0.0 V14.2.6 and V8.3.1; backend authorization is unchanged, and regression tests cover refused and denied key resolution. This frontend change does not assert application-wide ASVS compliance.

Validation: 55 related component tests passed; the latest mobile group and quota changes passed 35 focused tests. TypeScript, scoped lint, formatting, production build, and git diff checks passed. Responsive previews verified narrow screens and finite, unlimited, exhausted, and inactive quota states.

* feat(usage-logs): add searchable group filter

Reuse the shared Combobox to suggest groups for the active log view while allowing historical group names to be entered manually. Exclude the automatic routing pseudo-group from suggestions and preserve masking, reset, URL navigation, and mobile drawer behavior.

Extend the existing Combobox with keyboard event forwarding and accessible labels, preserve the selected custom value on focus, and close suggestions on blur. Enter confirms a selection before submitting the filter.

Validation: 25 tests passed across the shared Combobox and log group, type, and mobile filter suites. TypeScript, scoped lint, formatting, and git diff checks passed.

* fix(keys): show desktop quota amounts side by side

Place remaining quota on the left and used quota on the right above the progress bar, without visible labels in desktop rows. Preserve accessible descriptions and the existing mobile labels and stacked layout.

Validation: updated regressions failed before the change and all 26 API key listing tests passed afterward. TypeScript, scoped lint, formatting, production build, and git diff checks passed.

* style(keys): widen the desktop quota column

Increase the quota column width and minimum width from 220px to 260px to give the side-by-side amounts more room.

Validation: all 26 API key listing tests, TypeScript, scoped lint, formatting, and git diff checks passed.

* fix(keys): preserve spacing after desktop quota content

Bound desktop quota content to 180px so column width provides whitespace before the group column. Remove the quota-only padding override to use the same cell padding as other columns. Preserve the mobile quota layout.

Validation: layout regressions failed before the fix and all 26 API key listing tests passed afterward. TypeScript, scoped lint, formatting, production build, and git diff checks passed.

* feat(redemptions): add batch deletion and optional file exports

Add confirmed multi-select deletion through one batch API request and one
soft-delete statement. Record the affected count and requested IDs in a
separate batch audit event, and identify legacy events with missing counts.

After creation, offer an unchecked Save as a file option with TXT/Markdown
formats and optional name/quota fields. Keep Done as the default completion
action. Include translations for all seven frontend locales.

Validation:
- Redemption and audit frontend regression tests, typecheck, and scoped lint.
- go build ./...
- go test ./controller -run '^TestDeleteRedemptionBatch$' -count=1 -v
  with TEST_MYSQL_DSN, TEST_MYSQL_LOG_DSN, TEST_POSTGRES_DSN,
  and TEST_POSTGRES_LOG_DSN set to isolated primary and log databases.
- Real SQLite 3.50.4, MySQL 8.4.11, and PostgreSQL 16.15 passed, including
  deletion of 15 records, duplicate/missing IDs, zero-row retries, audit
  deduplication, invalid input, and exclusion of credentials from logs.

* docs: update project architecture and Go conventions

* refactor: modernize Go code conventions

* refactor(web): unify table cells and quota details

* test(dashboard): re-query the setup guide toggle while the overview settles

"does not show a completed setup entry when the key lookup fails" failed
on both CI runs of this PR, and intermittently in local full-suite runs,
while passing whenever the file ran alone.

The assertion held the node returned by findByRole across an async
boundary. The guide subtree is remounted while the remaining overview
queries settle, which detaches the captured element, so toBeVisible saw
a detached node — the reported element had aria-expanded="true" but no
children, which is what an unmounted subtree leaves behind.

Re-query inside waitFor so a transient remount cannot fail the
assertion. A guide that never expands still fails, so the test keeps its
meaning, and this follows web/AGENTS.md: async tests wait for an
explicit UI state rather than a captured node.

Verified: the file passes alone and the full suite passes 995/995.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(rate-limit): enhance rate-limited responses with error envelope and exempt internal service routes

---------

Co-authored-by: CaIon <i@caion.me>
Co-authored-by: mingisrookie <99650899+mingisrookie@users.noreply.github.com>
Co-authored-by: PDMaker <86941825+pdmaker@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: qiuliw <a1807191473@qgmail.com>
salem-2007 added a commit to salem-2007/new-api that referenced this pull request Sep 10, 2026
* perf(relay): bulk-copy Responses raw JSON fields

* perf(common): share RawMessage deep-copy optimization

---------

Co-authored-by: CaIon <i@caion.me>
MAakber pushed a commit to MAakber/new-api that referenced this pull request Sep 11, 2026
* perf(relay): bulk-copy Responses raw JSON fields

* perf(common): share RawMessage deep-copy optimization

---------

Co-authored-by: CaIon <i@caion.me>
(cherry picked from commit 2cf177a)
lazzman pushed a commit to lazzman/new-api that referenced this pull request Sep 14, 2026
* perf(relay): bulk-copy Responses raw JSON fields

* perf(common): share RawMessage deep-copy optimization

---------

Co-authored-by: CaIon <i@caion.me>
Yorick-Ryu pushed a commit to Yorick-Ryu/new-api that referenced this pull request Sep 18, 2026
* perf(relay): bulk-copy Responses raw JSON fields

* perf(common): share RawMessage deep-copy optimization

---------

Co-authored-by: CaIon <i@caion.me>
(cherry picked from commit 2cf177a)
piexian added a commit to piexian/new-api that referenced this pull request Oct 4, 2026
…s 77-80 (#16)

* fix: purge authentication data on hard user deletion (QuantumNous#6168)

* fix: purge authentication data on hard user deletion

* fix: fail closed when 2FA status lookup fails

* fix: reject stale Telegram login callbacks

* fix(twofa): prevent concurrent backup code and lockout bypasses

* fix(auth): harden user deletion and Telegram verification

(cherry picked from commit b6b97a6)

* fix: infer MiniMax vendor for MiniMax models (QuantumNous#6164)

* fix: infer MiniMax vendor for MiniMax models

* Delete model/pricing_default_test.go

---------

Co-authored-by: duanxufu <duanxufu@rededa.com>
(cherry picked from commit a63364d)

* fix(data-table): update table body immediately when toggling columns (QuantumNous#6253)

DataTableRow's React.memo did not depend on columnVisibility.
TanStack row references stay stable when visible columns change, so the
table body did not refresh after View -> Toggle columns until another
action rebuilt the column definitions.

Capture a visible column id signature (visibleColumnIds) outside the
memo and compare it, matching the existing isSelected snapshot pattern.
All tables that use DataTableRow pick up the fix.

(cherry picked from commit dc9aeab)

* fix: prevent duplicate tool calls in Responses-to-Chat streaming (QuantumNous#6225)

When a function call was already registered under its output_index key
via response.output_item.added, the synthetic events built from the
terminal response.completed output carry no output_index and resolve to
a different item-based key. ensureToolForEvent then created a second
tool index and resent the full arguments, so Chat Completions clients
received the same tool call twice.

Reuse the tool registered under itemIDToKey/callIDToKey before creating
a new one, and alias the new key to the existing tool.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 1086038)

* fix: drop realtime beta header for ga models and register new ones (QuantumNous#6032)

(cherry picked from commit 16bfae1)

* fix: prevent duplicate suno task refunds via cas status update (QuantumNous#6074)

* fix: prevent duplicate suno task refunds via cas status update

* fix: reconcile failed task refunds

---------

Co-authored-by: CaIon <i@caion.me>
(cherry picked from commit e0d5156)

* fix: log response body when parsed upstream error message is empty

When an upstream error response parses as valid JSON but yields no usable
error message (e.g. an aggregator gateway returning {"error":{"message":""}}),
RelayErrorHandler previously produced a bare "bad response status code N"
error with no trace of the original body, making the failure undiagnosable.
Log the body preview in that case, mirroring the existing behavior for
unparseable bodies.

(cherry picked from commit 84a79b6)

* feat: add gemini-3-pro-image and gemini-3.1-flash-image GA models (QuantumNous#6371)

(cherry picked from commit 8b41def)

* fix: 慢查询/错误 SQL 日志参数化 (QuantumNous#6493)

* fix: parameterize slow/error SQL logs to avoid leaking credentials

* fix: validate SQL_SLOW_THRESHOLD_MS range

* fix: sanitize database driver error messages in SQL logs

* refactor: sanitize at gorm log writer seam to keep caller attribution

(cherry picked from commit c3db414)

* feat: support zstd request decompression (QuantumNous#6545)

(cherry picked from commit 0f9f668)

* fix(aws): cancel Bedrock requests on client disconnect (QuantumNous#6589)

* fix(aws): cancel Bedrock requests on client disconnect

* fix(billing): log effective usage billing path

(cherry picked from commit bd585d7)

* feat(rate-limit): add user critical rate limit middleware for access token and aff transfer routes

(cherry picked from commit 1da23d6)

* fix(ali): stop injecting top_p into requests that omit it (QuantumNous#6674)

(cherry picked from commit 2399de9)

* fix(ollama): preserve reasoning and tool-call context (QuantumNous#6605)

(cherry picked from commit 8ad159a)

* fix: backend length validation (QuantumNous#5548)

(cherry picked from commit d49160f)

* fix: 修复 Gemini 风格 /v1/models 列表请求 (QuantumNous#6199)

* fix: support Gemini model listing on v1 route

* test: cover Gemini query key model listing

(cherry picked from commit 3d5dc36)

* fix(topup): settle recharge orders atomically

(cherry picked from commit 50e5377)

* fix(billing): 异步任务退款时同步减少 used_quota (QuantumNous#6795)

* fix(billing): 异步任务退款时同步减少 used_quota
退款时仅恢复了 quota(剩余额度),但未同步减少 used_quota(已用额度),
导致"总额度"(quota + used_quota)随退款次数持续虚增,超出用户实际充值金额。

修复三处退款路径:
- RefundTaskQuota:任务失败完整退款
- RecalculateTaskQuota:差额结算退款分支
- controller/midjourney.go:Midjourney 任务失败退款

新增 model.UpdateUserUsedQuota 公开函数,仅调整 used_quota 不影响 request_count。

* fix(billing): 任务退款时同步扣减渠道 used_quota

* fix(billing): complete async task refund accounting

* style(model): group internal Midjourney fields

---------

Co-authored-by: CaIon <i@caion.me>
(cherry picked from commit 58d4e9b)

* fix(ali): 修复阿里图片模型映射后仍使用原始模型名判断协议的问题 (QuantumNous#6772)

* feat(ali): support Qwen Image 3 requests

* refactor(ali): narrow Qwen Image 3 support

* fix(ali): focus image model mapping fix

(cherry picked from commit 93d2df8)

* fix(topup): reject uncreditable orders before payment (QuantumNous#6845)

* fix(topup): reject uncreditable orders before payment

* fix(topup): align zero-ratio Stripe validation

* fix(topup): mirror settlement conversions in validation

(cherry picked from commit 2a0ce34)

* fix: settle Responses cached token usage (QuantumNous#6892)

(cherry picked from commit f116414)

* fix(billing): 修复时间规则恒真表达式导致倍率全天生效 (QuantumNous#6934)

(cherry picked from commit ac381ac)

* fix(relay): 请求参数校验错误返回 HTTP 400 (QuantumNous#6774)

* fix(relay): return 400 for invalid request parameters

(cherry picked from commit 0f2a207)

* fix(ali): honor image response format (QuantumNous#5513) (QuantumNous#7048)

(cherry picked from commit 0bee5d4)

* fix(sqlite): enable WAL + working busy timeout + _txlock=immediate to stop concurrent write lockouts (QuantumNous#7030)

* fix(sqlite): enable WAL + working busy timeout + _txlock=immediate to stop concurrent write lockouts

(cherry picked from commit 1751f43)

* fix(relay): bound the wait for upstream response headers (fixes unbounded heap growth → OOM) (QuantumNous#6949)

* fix(relay): bound the wait for upstream response headers (fixes unbounded heap growth)

The relay transport sets a dial timeout, a TLS handshake timeout and an expect-continue
timeout, but nothing bounds how long it waits for the upstream *response headers* after
the request has been written. An upstream that accepts the connection and then never
answers -- without sending FIN/RST, which is what happens when a NAT/firewall silently
drops the flow or the provider hangs -- parks the goroutine in
net/http.(*persistConn).roundTrip forever.

That goroutine keeps the whole request alive, which in practice means three copies of the
request body stay reachable for the lifetime of the process: the raw bytes from
io.ReadAll in CreateBodyStorageFromReader, the decoded messages held as json.RawMessage,
and the re-marshalled upstream body from common.Marshal. BodyStorageCleanup cannot help
here: it runs after c.Next() returns, and for these requests c.Next() never returns.

Measured on v1.0.0-rc.23 in production (see QuantumNous#6947 for the full evidence):

  - 23 goroutines stuck in persistConn.roundTrip on a single 40h-old instance,
    blocked between 353 and 1894 minutes (5.9h to 31.5h)
  - 96.9% of the live heap, sampled after a forced GC, attributable to those three
    body copies (HeapAlloc 892 MiB surviving three GC cycles; HeapObjects dropping
    30x while bytes dropped only 25%)
  - the live floor grows with uptime: 33.7 MiB at 0.1h, 89.2 at 13.8h, 510.0 at 40.1h,
    955.2 at 146.8h, OOMKilled at 172.9h -- same image, same config, same load

Doubling the memory limit and adding GOMEMLIMIT only moved the OOM from 132h to 172.9h.

RELAY_TIMEOUT (http.Client.Timeout) cannot be used for this: it covers the whole response
read and would cut legitimate long streaming calls, which is why it defaults to 0.
ResponseHeaderTimeout only bounds the wait for the headers; streaming after they arrive is
unaffected.

The default is deliberately generous. Non-streaming upstreams usually send the response
headers only once generation has finished, so the value has to leave room for a long
completion. 1800s is 12x shorter than the shortest hang observed here while leaving
several times the headroom a normal non-streaming request needs; 0 restores the previous
unbounded behaviour.

The assignment goes next to the other transport.* lines rather than inside the else
branch: newRelayHTTPTransport() normally takes the http.DefaultTransport.Clone() path,
and DefaultTransport does not set ResponseHeaderTimeout either.

This repo already sets ResponseHeaderTimeout on its other outbound transports
(controller/model_sync.go, controller/ratio_sync.go); the relay path appears to have
been missed.

Refs QuantumNous#6947. Likely also the root cause of QuantumNous#6731, which reported the same symptom
(production OOM on /v1/responses after ~64h) but was closed for template reasons.

* review: clamp overflowing timeout values and switch the test to testify

Addresses the two CodeRabbit findings on this PR.

Overflow (common/init.go:113): a RELAY_RESPONSE_HEADER_TIMEOUT beyond ~9.2e9 seconds
overflows time.Duration and can wrap into a *tiny positive* timeout, which would cut
every relay request instead of only the stuck ones. The value is now clamped before the
conversion, with regression tests for both the negative and the overflowing input.

I did not add fail-on-startup validation for negative values, for two reasons: the
existing `if seconds > 0` guard already treats them as "disabled", and the neighbouring
env-driven timeouts in this file are less strict still -- RelayIdleConnTimeout is
converted with no guard at all. Failing startup on a bad value would be a behaviour
change out of step with the rest of the file; happy to add it if you'd prefer that
direction repo-wide.

Test style: switched to testify (require.Equal / require.Zero / require.Positive), which
is what every other test under service/ uses.

go build, go vet and go test ./common/... ./service/... pass.
(`go build ./...` fails on the `web/dist` embed both with and without this change -- the
frontend bundle is not checked in.)

(cherry picked from commit b518d00)

* fix: adapt relaykit hosttypes to types and add missing errors import

* fix(log): preserve quota in usage statistics (QuantumNous#7108)

* fix(log): preserve quota in usage statistics

(cherry picked from commit 8c8c415)

* fix(model): do not treat no-op system task state writes as lock loss (QuantumNous#7135)

(cherry picked from commit b7017c2)

* perf(common): 批量复制 RawMessage,优化请求深拷贝 (QuantumNous#7221)

* perf(relay): bulk-copy Responses raw JSON fields

* perf(common): share RawMessage deep-copy optimization

---------

Co-authored-by: CaIon <i@caion.me>
(cherry picked from commit 2cf177a)

* fix(audio): normalize file extension case in GetAudioDuration (QuantumNous#7321)

Uppercase or mixed-case extensions (e.g. .MP3, .WAV) fell through to the
default branch and returned "unsupported audio format", so audio token
usage could not be measured. Lowercase the extension before the switch.

Closes QuantumNous#7319

(cherry picked from commit 76f7daf)

* fix(ollama): preserve tool calls from final stream frame | 保留流式末尾帧的 tool calls (QuantumNous#7376)

(cherry picked from commit 8e5e091)

* fix(gemini): reject :countTokens as an unknown route instead of relaying it as generateContent (QuantumNous#7388)

(cherry picked from commit 81336fc)

* fix(relay): request stream usage on every cross-protocol conversion to OpenAI chat (QuantumNous#7389)

(cherry picked from commit 610334d)

* fix(billing): count upstream images by payload instead of data length

OpenaiImageHandler and the JSON-to-SSE fallback billed the gjson data.#
element count. An object-shaped data yielded 0 and kept the requested n
while the SSE fallback dropped the image and sent only [DONE]; one image
split into a url entry and a b64_json entry billed twice; entries carrying
only revised_prompt billed as images.

Count images by payload: an object with url or b64_json is one image and
arrays bill max(url entries, b64_json entries). The SSE fallback forwards
one completed event per image payload for both shapes. xAI inherits the
fix through OpenaiImageHandler.

(cherry picked from commit 07578fa)

* fix(relay): map Images output_tokens_details into img_o billing (QuantumNous#7410)

(cherry picked from commit e2bb05f)

* fix(task): accept any 2xx upstream status on task submission (QuantumNous#7478)

(cherry picked from commit d61d6be)

* fix(auth): reject non-standard roles when creating a user (QuantumNous#7531)

(cherry picked from commit 2506e1b)

* fix: isolate TLS configs across HTTP transports (QuantumNous#7627)

(cherry picked from commit feefe09)

* fix(model): migrate database constraints, options primary key, and avoid redundant migrations

* feat(channel): add Sub2API, NewAPI, vLLM, and SGLang channels (77-80) and Tencent TokenHub

---------

Co-authored-by: Seefs <40468931+seefs001@users.noreply.github.com>
Co-authored-by: fuxdev <15846422264@163.com>
Co-authored-by: duanxufu <duanxufu@rededa.com>
Co-authored-by: DawnMoon1542 <160485532+DawnMoon1542@users.noreply.github.com>
Co-authored-by: Niku <nikuliu@foxmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: feitianbubu <feitianbubu@qq.com>
Co-authored-by: CaIon <i@caion.me>
Co-authored-by: bigsong <35025755+bigsongeth@users.noreply.github.com>
Co-authored-by: ENCHIGO <38551565+ENCHIGO@users.noreply.github.com>
Co-authored-by: zhibisora <73344387+zhibisora@users.noreply.github.com>
Co-authored-by: wans10 <51012039+wans10@users.noreply.github.com>
Co-authored-by: Scott <908181134@qq.com>
Co-authored-by: Shawn Wang <32839114+enwaiax@users.noreply.github.com>
Co-authored-by: Qi <108174052+LiaoQi98@users.noreply.github.com>
Co-authored-by: zcxads666 <128150298+zcxads666@users.noreply.github.com>
Co-authored-by: Alex Xiang <ax2@zicode.com>
Co-authored-by: PuppetKL <154485567+PuppetKL@users.noreply.github.com>
Co-authored-by: Xayinn <129403670+LinineTy@users.noreply.github.com>
Co-authored-by: txgo <tianxi.liu@gmail.com>
Co-authored-by: jimmyleocn <42566991+jimmyleocn@users.noreply.github.com>
Co-authored-by: mingisrookie <99650899+mingisrookie@users.noreply.github.com>
Co-authored-by: Vladimir Babin <vovababin@gmail.com>
Co-authored-by: sxhstc <406739419@qq.com>
leicanxi pushed a commit to leicanxi/flowbee-api that referenced this pull request Oct 5, 2026
* perf(relay): bulk-copy Responses raw JSON fields

* perf(common): share RawMessage deep-copy optimization

---------

Co-authored-by: CaIon <i@caion.me>
(cherry picked from commit 2cf177a)
timwenx pushed a commit to timwenx/new-api that referenced this pull request Nov 18, 2026
* perf(relay): bulk-copy Responses raw JSON fields

* perf(common): share RawMessage deep-copy optimization

---------

Co-authored-by: CaIon <i@caion.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants