[Defilan]

What

Mount POST /v1/completions on the ModelRouter router-proxy, routed through the same shared handleCompletion path as chat, embeddings, and rerank (internal/router/proxy.go, Mount). One route registration plus a doc-comment refresh; no behavior on existing endpoints changes. The docs concept page now lists every route the proxy serves.

Why

Fixes #2017.

IDE autocomplete and fill-in-the-middle (FIM) clients send raw prompts to /v1/completions, and benchmark tools such as vllm bench serve --backend openai default to it. A request to it through a ModelRouter got a 404 from the proxy, so teams fronting developer autocomplete or benchmarking a model through its router could not use a ModelRouter at all. vLLM, SGLang, and llama.cpp's server already serve the endpoint, so the backends support it.

How

Mount registers POST /v1/completions with p.handleCompletion("/v1/completions"), mirroring how #1819 added embeddings and rerank: upstreamPath is forwarded verbatim, so the backend receives the request at /v1/completions, not the chat path.

The issue asked to confirm, not assume, that the shared handler's machinery holds for a completions body (prompt instead of messages). Each point was verified against the code and pinned by a test:

• Model matching and rule selection: extractFeatures (internal/router/proxy.go) parses only model and stream from the body and never touches messages/prompt, and matchModels (internal/router/match.go) operates on the extracted field, so a completions body routes identically. Pinned by TestProxyCompletionsModelRuleSelectsBackend, which also pins the ordering hazard (a chat-* rule declared before the code-* rule must not capture a code-* model).
• Budget charging from usage, non-streamed and streamed: parseUsage dispatches to UsageTokens (JSON) or UsageTokensFromSSE (SSE) in internal/router/usage.go, both of which read only usage.{prompt_tokens,completion_tokens,total_tokens}, the shape legacy completions return. InjectStreamUsage rewrites only stream/stream_options, which legacy completions also accept; vLLM's serving_completion.py confirms a final SSE usage chunk (empty choices) before data: [DONE] when include_usage is set. Pinned by TestProxyCompletionsBudgetCharged and TestProxyCompletionsStreamingBudgetCharged (the latter replays a faithful legacy-completions SSE stream with choices[].text chunks).
• Per-endpoint dispatch and health path from feat(router): opt-in endpoint-level backends for per-request balancing #1820: the completions route flows through the same dispatchWithFallback -> Dispatch -> pickEndpoint/joinURL and per-endpoint markHealthy/markUnhealthy chain (internal/router/backend.go). Nothing in that chain reads the body shape, and applyModelOverride only rewrites the model field. Confirmed by code reading; the dispatcher-level endpoint tests in internal/router/backend_test.go take the path as an argument and are path-agnostic.

Security posture (per AGENTS.md, "A ClusterIP is not a network boundary"): this adds an endpoint, not a surface. The same data-plane port and Service serve it, so it is reachable by exactly the pods that could already reach /v1/chat/completions, and no new scoping object ships with it. It sits on the identical request path: rule matching, enforceFailClosed, budget admission, and audit all run before dispatch for every mounted route, so fail-closed rules and budgets govern completions exactly as they govern chat. The budget tests below pin the charging side, and TestProxyFailClosedOnCompletions pins the refusal side.

New tests in internal/router/proxy_test.go: TestProxyMountsCompletions (route mounts, body relayed byte-for-byte, response relayed), TestProxyCompletionsModelRuleSelectsBackend, TestProxyCompletionsBudgetCharged, TestProxyCompletionsStreamingBudgetCharged, and TestProxyFailClosedOnCompletions (a sensitive completions request against a fail-closed rule whose only backend is down is refused with 503 and dispatches nowhere). Test helpers added: an sseScript override on the fake backend (endpoint-specific SSE replay, usage tail shared) and path-parameterized budgetPostPath/budgetStreamPostPath that the existing helpers delegate to.

Every test's falsification was applied to the working tree, the named test run, the failure pasted below, and the tree restored:

• Removed the POST /v1/completions line from Mount -> TestProxyMountsCompletions: proxy_test.go:317: /v1/completions = 404, want 200 (FAIL).
• Made UsageTokensFromSSE match a prefix that never occurs -> TestProxyCompletionsStreamingBudgetCharged: proxy_test.go:1638: snapshot = [{Name:router-cap ScopeKey:router UsedTokens:0 ...}], want one entry with UsedTokens 100 (a streamed completions response must be charged from the final usage chunk) (FAIL).
• Made UsageTokens report no usage -> TestProxyCompletionsBudgetCharged: proxy_test.go:1608: snapshot = [{Name:router-cap ScopeKey:router UsedTokens:0 ...}], want one entry with UsedTokens 100 (a completions response must be charged from its usage object) (FAIL).
• Removed the matchModels guard from Matcher.ruleMatches -> TestProxyCompletionsModelRuleSelectsBackend: proxy_test.go:492: chat-model calls = 1, want 0 (a chat-* rule must not capture a code-* model) (FAIL).
• Neutered the runtime fail-closed branch in handleCompletion (if decision.FailClosed made dead) -> TestProxyFailClosedOnCompletions: proxy_test.go:547: status = 502, want 503 fail-closed runtime (FAIL).

Gates, run locally on this branch (including the added test commit):

• make test: PASS (full suite; internal/router at 93.7% statement coverage).
• make lint: PASS (0 issues.).
• make lint-deadcode: PASS (0 issues.).
• make fmt / make vet: PASS, no diff from make fmt.
• make lint-all: FAIL on this branch, but identically on a pristine upstream/main checkout: golangci-lint v2.12.2 panics inside staticcheck v0.7.0 (buildir: unexpected expr: *ast.KeyValueExpr and fact_purity: package "poll" ... interface conversion: interface {} is nil, not *buildir.IR). Pre-existing tooling incompatibility under the local Go 1.27.1 toolchain, unrelated to this change.

Not run: make test-e2e-rbac. It is required after controller changes that add API calls (+kubebuilder:rbac markers); this change touches only the router-proxy's HTTP mux and adds no controller API surface.

Assisted-by: GLM-5.3-Flash (local GLM coding agent; drafted the route mount, tests, and docs from issue #2017; every gate and falsification listed above was executed in-session and its raw output is quoted here)

Checklist

• Tests added/updated
• make test passes locally
• make lint passes locally
• Commit messages follow conventional commits
• All commits are signed off (git commit -s) per DCO
• AI assistance (if any) is disclosed above, per CONTRIBUTING.md
• Documentation updated (if user-facing change)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!