Skip to content

fix(ai-gateway): resolve permission error when allow all models is configured (#3629) - #3732

Open
Adityakk9031 wants to merge 32 commits into
fosrl:devfrom
Adityakk9031:fix/allow-all-models-permission-3629
Open

Adityakk9031 wants to merge 32 commits into
fosrl:devfrom
Adityakk9031:fix/allow-all-models-permission-3629

Conversation

@Adityakk9031

@Adityakk9031 Adityakk9031 commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

close: #3629

Description & Root Cause

When configuring an AI Provider (such as Custom, OpenRouter, or Vercel AI Gateway) with Allow List set to "Allow all models" (*), requests from AI clients like Open Code (e.g., testing with models such as unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL) failed with a proxy/permissions error (404 Not Found returned as a forbidden/proxy failure).

Investigation revealed two root causes:

  1. Model Discovery on GET /v1/models/:model:
    • Because * is a wildcard pattern, expandProviderModels does not expand it into explicit model IDs. For aggregator/custom providers, provider.catalog was empty because they lack a fixed vendor prefix catalog.
    • When clients query GET /v1/models/:model for a model allowed by wildcard, handleV1Models was only searching the pre-enumerated models array (models.find), returning 404 Not Found instead of checking if any attached provider allows the requested model via wildcard patterns (isAllowedByLists).
  2. Provider Selection Tie-Breaking for Custom / Unknown Models:
    • In catalogOwnershipScore (server/lib/aiProviderSelection.ts), unknown/custom models not in the static catalog scored 0 for all providers.
    • When tie-breaking in compareProviderCandidates, typed providers (e.g. openai, class rank 2) were preferred over custom providers (class rank 0), mistakenly routing non-catalog / self-hosted models to native vendor endpoints that cannot serve them.

Changes Made

  • server/routers/aiGateway/v1Models.ts:
    • In catalogMetadataForType: Populated catalog entries for custom, openRouter, and vercelAiGateway using aiModelCatalog.getAll() so wildcard matching has full catalog metadata.
    • In handleV1Models: For GET /v1/models/:model, added fallback wildcard validation with isAllowedByLists(requestedModel, p.allows, p.blocks) across discovery providers before returning 404.
  • server/lib/aiModelDiscovery.ts:
    • Included non-pattern configured model keys in expandProviderModels candidates.
  • server/lib/aiProviderSelection.ts & docs/ai-gateway-provider-selection.md:
    • Updated catalogOwnershipScore so aggregator/custom providers score 1 for unknown/custom models while typed providers score 0 (typed catalog miss), properly routing custom models to custom/aggregator providers.
    • Updated documentation table and Example H accordingly.

Verification

  • npm run dev:check: All TypeScript type checks and Prettier formatting passed (0 errors).
  • Server build (node esbuild.mjs -e server/index.ts -o dist/server.mjs) succeeded.

@oschwartz10612
oschwartz10612 force-pushed the dev branch 2 times, most recently from a9ac442 to c6c12f1 Compare September 15, 2026 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants