Skip to content

fix(catalog): resolve gateway model metadata from other publishers - #1036

Merged
vastsa merged 1 commit into
vastsa:mainfrom
lenmei233:fix/catalog-cross-provider-model-metadata
Sep 24, 2026
Merged

vastsa merged 1 commit into
vastsa:mainfrom
lenmei233:fix/catalog-cross-provider-model-metadata

Conversation

@lenmei233

Copy link
Copy Markdown
Contributor

Problem

Configuring ModelScope (an OpenAI-compatible gateway at https://api-inference.modelscope.cn/v1) in PI-Desktop left the context window and tool support unidentified for most of the models its endpoint serves (#938).

The issue proposed widening the model-id matching ("weak matching"), since metadata lookup keys off the exact model id and ids carry a vendor prefix and date suffixes such as 0731. That is not the defect. Exact matching already accepts those ids:

pair catalogModelIdsMatch
Qwen/Qwen3-235B-A22B-Instruct-2507 vs itself match
deepseek-ai/DeepSeek-V4-Flash-0731 vs itself match

A global weak match would make things worse, not better: Qwen/Qwen3.5-27B and Qwen/Qwen3-235B-A22B-...-2507 normalize to the same qwen/qwen3-... family, so a fuzzy lookup would hand the 235B's window to the 27B.

Root cause

models.dev indexes a gateway's copy of a model under the vendor that owns the weights, not under the gateway:

  • modelscope publishes only 7 records of its own (older Qwen3-2507 / GLM ids)
  • but deepseek-ai/DeepSeek-V4-Flash-0731 is published verbatim by 11 other providers (deepinfra, nebius, huggingface, nvidia, …)

ModelsDevCatalog.findModel scopes the lookup to the row's own catalog provider once that provider is known (models-dev-catalog.ts:1079), so every one of those ids missed and fell back to the generic 128k text-only shape.

Measured against the real endpoint and the bundled snapshot: 0 of 31 ModelScope text models resolved before the change; 14 resolve after.

Change

Adds a cross-provider exact-id fallback, consulted only when the row resolved to a known catalog provider that published nothing for the id. Four deliberate constraints:

  1. Only a case-insensitive identical id transfers. A record reached through an alias (proxy/openai/gpt-4o) is a different id and keeps its own limits.
  2. A provider sharing the row's own endpoint is skipped — it is an alias for the row, so its silence is an answer about this deployment. This preserves the existing invariant that a shared catalog API prefers the explicitly selected vendor.
  3. Tool support must be unanimous across publishers, because a wrong true puts tool declarations on the wire that the endpoint may reject.
  4. Everything else is the intersection and limits are medians, so a borrow can only under-claim; a user who knows the endpoint does more can still enable it in Advanced. An unknown endpoint keeps its existing behaviour exactly.

Metadata only: configured wire ids, provider identity, IPC and binding precedence are untouched.

Files

  • apps/desktop/electron/main/models-dev-catalog.ts — borrowedModel() + ModelsDevCatalog.borrowedAcrossProviders()
  • apps/desktop/test/models-dev-catalog.test.mjs — 7 new tests (2 confirmed failing on the pre-fix code)
  • docs/spec/03-runtime/13-model-catalog-and-selection.md — new §11.3.1

Verification

command result
pnpm --filter @pi-desktop/shared test 1026 passed
node --test test/models-dev-catalog.test.mjs 38 passed (31 existing + 7 new)
8 related model/catalog suites all passed
pnpm --filter @pi-desktop/desktop typecheck passed
pnpm build:js passed
pnpm check:architecture, pnpm check:agent-policy, pnpm check:pr-base passed

Also self-checked: aliases are never borrowed, memoization cannot leak a borrow into a provider-scoped row, reasoning: false exposes no thinking levels, and a borrow does not mutate shared catalog records.

Known limits

  • 11 of the 31 ModelScope ids exist in no models.dev record (Shanghai_AI_Laboratory/Intern-S1, ZhipuAI/GLM-5.2, PaddlePaddle/ERNIE-*, …) — an upstream data gap a client cannot infer.
  • A few are refused by design where publishers disagree on tool support (e.g. MiniMax/MiniMax-M1-80k). Both classes still allow a manual Advanced override.

pnpm lint fails on apps/desktop/src/styles/composer-menus.css (2 pre-existing style-token violations in a file this PR does not touch).

fixes #938

models.dev indexes a gateway's copy of a model under the vendor that owns the
weights, not under the gateway. An endpoint serving `Vendor/Model` ids such as
`deepseek-ai/DeepSeek-V4-Flash-0731` therefore has no record of its own while
several other publishers state the identical id. `findModel` scoped the lookup
to the row's own catalog provider once that provider was known, so every such
model missed and fell back to the generic 128k text-only shape, leaving its
context window and tool support invisible. On ModelScope none of the 31 models
its endpoint serves resolved; 14 do now.

Borrowing is deliberately narrow. It runs only when the row resolved to a
catalog provider that published nothing for the id, and only for a
case-insensitive identical id: a record reached through an alias describes a
different id and keeps its own limits. A provider sharing the row's endpoint is
skipped, because it is an alias for the row and its silence answers for this
deployment. Tool support must be unanimous across publishers, since a wrong
`true` puts tool declarations on the wire that the endpoint may reject, while
reasoning, vision and attachment are intersections and limits are medians, so a
borrow can only under-claim. An unknown endpoint keeps its existing behaviour.

The issue suggested widening suffix matching for ids such as `-0731`; that is
not the defect. Exact matching already accepts those ids, and a global weak
match would conflate distinct models (`Qwen/Qwen3.5-27B` and
`Qwen/Qwen3-235B-A22B-...-2507` normalize alike). This changes metadata only:
configured ids, provider identity and binding precedence are untouched.

fixes vastsa#938
@vastsa
vastsa force-pushed the fix/catalog-cross-provider-model-metadata branch from 7e43e3e to b293f53 Compare September 24, 2026 18:43
@vastsa
vastsa merged commit 41d15d5 into vastsa:main Sep 24, 2026
3 of 4 checks passed
@lenmei233
lenmei233 deleted the fix/catalog-cross-provider-model-metadata branch September 24, 2026 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] models数据拉取好像有问题,希望优化一下

2 participants