[joryirving]

What

Adds spec.rules[].route.poolActivation: Wait | IfIdle to ModelRouter. Under IfIdle, a rule whose preferred backend is a non-resident ModelPool member skips that backend immediately when the incumbent is busy and falls through to the next backend in the rule, instead of holding the request until the incumbent drains. Wait is the default and preserves today's behaviour exactly.

Why

A pool holds a cross-model request until the incumbent is idle, bounded only by the pool-wide swapBudget. That is the right contract for a caller that accepts one model and the wrong one for a caller that merely prefers one: on a single 24 GB card shared by a small assistant model and a large coder, the assistant waits minutes behind a batch job while the warm coder could serve it now. Client-side fallback cannot solve it because a timeout cannot distinguish "incumbent busy" from "swap already in flight", so it abandons real swaps and lands on a member that is mid-unload.

Fixes #1786

How

• RuleRoute.PoolActivation on the CRD (enum Wait;IfIdle, default Wait), mirrored on the wire RuleRoute, validated in Config.Validate, threaded through MatchResult and copied by translateRule.
• Activator.AcquireWithMode: in the !swapping && resident != member branch, a busy incumbent under IfIdle returns a new ErrIncumbentBusy instead of blocking; Acquire delegates with Wait. The existing deferred cleanup handles the waiter count. A swap already in flight is waited on in both modes, since the incumbent is unloading and cannot serve either way.
• dispatchWithFallback passes the matched rule's mode; the existing continue on activation error does the fallthrough.
• New counter llmkube_modelpool_busy_skips_total{router,pool,member}.
• Docs: new subsection in docs/operations/gpu-sharing.md, including the caveat that "busy" means proxy-tracked traffic (direct-to-Service traffic is invisible to the check; the controller's /slots drain still protects it at swap time).

Tests: activator unit tests for the skip and for the in-flight-swap exception, a proxy end-to-end test (rule [coder, judge], judge resident and busy, request served by judge with coder never activated), the translateRule passthrough, and closed-enum validation. Both behavioural tests fail with the IfIdle branch disabled and pass with it.

Assisted-by: Claude Code (Fable 5.1) generated the implementation, tests and docs from my design; I reviewed the diff, and ran make test, make lint and make lint-all locally (all green, router coverage 93.7%).

Checklist

• Tests added/updated
• make test passes locally
• make lint passes locally
• Commit messages follow conventional commits
• All commits are signed off (git commit -s) per DCO
• AI assistance (if any) is disclosed above, per CONTRIBUTING.md
• Documentation updated (if user-facing change)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[Defilan]

Approving, I traced the contract end to end and tried to break it and could not. The IfIdle skip sits inside the no-swap-running branch, so an in-flight swap is still waited on in both modes, and the skip path unlocks cleanly with the deferred waiter cleanup doing its thing. Both behavioural tests bite: I reverted the IfIdle branch and they failed exactly as your body claims (hold budget exceeded instead of ErrIncumbentBusy, and a 5s hold instead of instant fallback), and reverting the translateRule copy failed the passthrough test too. A probe with an unknown mode string held like Wait, so the mode check is exact and fail-safe. make manifests and make chart-crds produced no drift on your head, and the full router/controller/metrics suite is green here.

Two things worth a look, neither blocking. When every backend in a rule skips, the client gets 502 "all backends failed: model pool incumbent busy", while the sibling ErrHoldBudgetExceeded gets 503 with Retry-After and a pool_activation_timeout audit reason. It is the same transient condition and a retry after the incumbent drains does serve, so I would give it the same treatment or at least its own audit reason. Also, issue #1786's body describes a different ask entirely, resources.memory emitted as limits and a suggested resources.memoryLimit field, under this title, filed that way with no edits. Merging this closes the issue with the memory ask unaddressed, so worth fixing the issue body or refiling the memory part first.

Metric is registered, the docs caveat about proxy-tracked traffic is right, and the disclosure checks out. Thanks, nice work.