Skip to content

fix(foreman-agent): make the repo package's git env hermetic and forward GIT_CONFIG_* - #2014

Merged
Defilan merged 6 commits into
defilantech:mainfrom
Defilan:fix/foreman-agent-git-config-env
Oct 12, 2026
Merged

Defilan merged 6 commits into
defilantech:mainfrom
Defilan:fix/foreman-agent-git-config-env

Conversation

@Defilan

@Defilan Defilan commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

What

TestRemoteURL in pkg/foreman/agent/repo failed for contributors whose git configuration rewrites GitHub HTTPS URLs to SSH (a global url."git@github.com:".insteadOf rule), and the standard override GIT_CONFIG_GLOBAL=/dev/null did not help because runGit's minimal environment dropped it. Three changes:

  • Clone now builds its env through baseEnv() instead of a hand-built HOME-only slice; it was the one git call site the forwarding did not reach.
  • baseEnv() now forwards GIT_CONFIG_GLOBAL, GIT_CONFIG_SYSTEM and GIT_CONFIG_NOSYSTEM to the git subprocess when they are set (and non-empty). They hold file paths and flags, not credentials, so the no-secrets intent is unchanged. Empty values are skipped: git has no defined behavior for an empty filename, and /dev/null is the documented way to skip a level.
  • The package's tests are now hermetic: TestMain pins HOME, XDG_CONFIG_HOME, GIT_CONFIG_GLOBAL, GIT_CONFIG_SYSTEM and GIT_CONFIG_NOSYSTEM to empty temp files, so a developer's ~/.gitconfig (or a shell-exported GIT_CONFIG_GLOBAL, which baseEnv now forwards) cannot change test results. GIT_CONFIG_NOSYSTEM=1 also covers a rewrite in /etc/gitconfig, which HOME cannot.

Why

Fixes #2013.

Defect class: a test reads ambient machine state (the developer's git configuration reaches git remote get-url through HOME), and the caller-controlled standard override for exactly that state is dropped by the minimal subprocess environment. Production is unaffected: the only caller, pushedRemoteMismatch (pkg/foreman/agent/executor_native.go), compares the repository name and gitRemoteOwnerRepo parses both the HTTPS and the scp-style SSH form.

How

  • baseEnv() is the single composition point: every git path in the package (branch, clone, push, auth, RemoteURL) builds env = baseEnv() and appends Auth.Env(), so one forwarding point covers all call sites.
  • An audit of every runGit call site (branch, commit, diff, push, clone) found no other hand-built env: git.go owns PATH, auth.go defines Auth.Env(), and every remaining caller derives from baseEnv(), so the composition-point claim above now covers every git path including Clone. The runGit doc comment's contract ("callers append HOME, GIT_*, and the Auth env via extraEnv") is preserved.
  • Hermeticity is package-wide via TestMain, not per-test, so future tests inherit it. HOME and XDG_CONFIG_HOME alone are not enough once GIT_CONFIG_GLOBAL is forwarded: a shell-exported GIT_CONFIG_GLOBAL would re-leak, so all three location variables are pinned to empty files created in a temp dir.
  • Verified against git's documented semantics (git(1), ENVIRONMENT): GIT_CONFIG_GLOBAL/GIT_CONFIG_SYSTEM replace the whole configuration level (setting one disables both standard files of that level; /dev/null skips it), and GIT_CONFIG_NOSYSTEM is documented for exactly this "predictable environment" use together with HOME and XDG_CONFIG_HOME. GIT_CONFIG_GLOBAL/SYSTEM date to git 2.32; older gits ignore the forwarded variables, which is harmless.

Tests

  • TestRemoteURLGitConfigGlobalForwarding: sets an insteadOf rule in a temp global config, points GIT_CONFIG_GLOBAL at it, and expects RemoteURL to return the rewritten SSH URL. Fails on code that drops the variable (verified: RemoteURL(fork) = "https://github.com/Defilan/LLMKube.git", want the insteadOf-rewritten URL).
  • TestBaseEnv: pins the subprocess contract. HOME only by default; the three variables forwarded verbatim when set; set-but-empty skipped; a sentinel secret and PATH never forwarded. Falsified by appending os.Environ() (secret, PATH and empty-value assertions all failed) and by dropping the empty-value guard (empty GIT_CONFIG_SYSTEM forwarded).
  • TestGitEnvIsHermetic: pins TestMain itself (HOME under the OS temp dir, GIT_CONFIG_NOSYSTEM=1). Falsified by removing the Setenvs (HOME = "/Users/defilan", want a directory under .../T/).
  • TestCloneGitConfigGlobalForwarding: clones through an insteadOf rule that rewrites the URL to a local bare repo and asserts the clone lands on the seeded commit. Falsified by reverting Clone to the hand-built env (git then fetches example.invalid literally and exits 128).
  • TestRemoteURL unchanged; it is the detector for the reported bug. Reproduced before the fix with a hostile HOME containing the insteadOf rule (RemoteURL(fork) = "git@github.com:Defilan/LLMKube.git", want the fork URL; GIT_CONFIG_GLOBAL=/dev/null did not help) and the same hostile environment passes after the fix, along with a hostile GIT_CONFIG_GLOBAL.

Out of scope

Other git-exec sites outside this package (pkg/foreman/slicer/integrate.go, pkg/foreman/agent/tools/run_integrate.go, run_reconcile.go) do not use runGit/baseEnv and were left alone; #2013 scopes this fix to pkg/foreman/agent/repo.

AI assistance

Written by GLM-5.3-Flash NVFP4 (GLM-5.3-Flash, local inference on 3x NVIDIA GB10s running LLMKube) and reviewed and signed off by Christopher Maher.

Checklist

  • Tests added/updated
  • make test passes locally
  • make lint passes locally
  • Commit messages follow conventional commits
  • All commits are signed off (git commit -s) per DCO
  • AI assistance (if any) is disclosed above, per CONTRIBUTING.md
  • Documentation updated (if user-facing change)

runGit starts git from a minimal environment so secrets never reach
the subprocess, but that also dropped GIT_CONFIG_GLOBAL,
GIT_CONFIG_SYSTEM and GIT_CONFIG_NOSYSTEM, so the standard way to
override git configuration had no effect and TestRemoteURL failed for
anyone whose global config rewrites GitHub HTTPS URLs to SSH.

The three variables hold file paths and flags, not credentials, so
baseEnv now forwards them when set. Empty values are skipped: git has
no defined behavior for an empty filename, and /dev/null is the
documented way to skip a level.

Signed-off-by: Christopher Maher <chris@mahercode.io>
TestMain pins git's configuration sources to empty temp files: HOME
and XDG_CONFIG_HOME alone do not defeat a shell-exported
GIT_CONFIG_GLOBAL once baseEnv forwards it, so all three location
variables are pinned and GIT_CONFIG_NOSYSTEM=1 covers /etc/gitconfig.
A developer's insteadOf rewrite can no longer change test results.

New specs pin each piece: the forwarding test sets an insteadOf rule
in a temp global config and expects the rewritten URL (fails on code
that drops the variable), TestBaseEnv pins the subprocess contract
(HOME only by default, the three variables forwarded, set-but-empty
skipped, no secrets, no PATH), and TestGitEnvIsHermetic pins TestMain
itself.

Signed-off-by: Christopher Maher <chris@mahercode.io>
runGit's minimal environment never forwards GIT_CONFIG_COUNT, but the
direct-exec test helpers (mustGit, seedOrigin) inherit the full process
environment, so a developer with config entries injected through the
environment could still steer helper-side git calls. TestMain now
unsets it and TestGitEnvIsHermetic pins the absence.

Signed-off-by: Christopher Maher <chris@mahercode.io>
…re output

When the no-secrets invariant is broken, the failure messages printed
the whole baseEnv slice, which would dump the process environment,
secrets included, into CI logs. The messages now name unexpected
variables only; the sentinel and PATH entries carry no sensitive
values, so they print in full.

Signed-off-by: Christopher Maher <chris@mahercode.io>
@codecov

codecov Bot commented Oct 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Clone built its subprocess env by hand (HOME only), so GIT_CONFIG_GLOBAL
and the other config-location overrides never reached git clone. The
insteadOf rewrite a contributor relies on to clone through SSH was
ignored, the same failure defilantech#2013 fixed for the other call sites, and the
package could not clone from an alias URL at all.

baseEnv is the composition point every other git path already uses;
Clone joins it, still appending Auth.Env() for the GIT_ASKPASS path.

Signed-off-by: Christopher Maher <chris@mahercode.io>
An insteadOf rule in a temp global config rewrites the clone URL to a
local bare repo, so the clone succeeds only when git read the forwarded
variable and lands on the seeded commit either way. With the hand-built
env reverted, git fetches example.invalid literally and exits 128.

Signed-off-by: Christopher Maher <chris@mahercode.io>
@Defilan
Defilan marked this pull request as ready for review October 12, 2026 18:17
@Defilan
Defilan merged commit 62837d9 into defilantech:main Oct 12, 2026
27 checks passed
@github-actions github-actions Bot mentioned this pull request Oct 12, 2026
hbjydev pushed a commit to hbjydev/phoebe that referenced this pull request Oct 20, 2026
… 0.10.4) (#885)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [ghcr.io/defilantech/charts/llmkube](https://github.com/defilantech/LLMKube) | patch | `0.10.2` → `0.10.4` |

---

### Release Notes

<details>
<summary>defilantech/LLMKube (ghcr.io/defilantech/charts/llmkube)</summary>

### [`v0.10.4`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0104-2026-10-16)

[Compare Source](defilantech/LLMKube@v0.10.3...v0.10.4)

##### Features

- **agent:** cap the bytes a single Model download may write ([#&#8203;2088](defilantech/LLMKube#2088)) ([3852411](defilantech/LLMKube@3852411))
- **api:** endpoint API key and externalTrafficPolicy for InferenceService ([#&#8203;2043](defilantech/LLMKube#2043)) ([27a9214](defilantech/LLMKube@27a9214))
- **controller:** detect a stalled watch path with a lease canary, bounded reconciles and stall alerts ([#&#8203;2072](defilantech/LLMKube#2072)) ([d00e0cf](defilantech/LLMKube@d00e0cf))
- **controller:** detect keyed pool siblings and vLLM env-var keys in ModelRouter ([#&#8203;2068](defilantech/LLMKube#2068)) ([78889f0](defilantech/LLMKube@78889f0))
- **controller:** detect ModelRouter backends keyed through --api-key on the command line ([#&#8203;2063](defilantech/LLMKube#2063)) ([3674acc](defilantech/LLMKube@3674acc))
- **controller:** stage HF mirror repos with spec.files and expand globs from the repo listing ([#&#8203;2071](defilantech/LLMKube#2071)) ([9e5d7d3](defilantech/LLMKube@9e5d7d3))

##### Bug Fixes

- **agent:** health restarts stop the unhealthy engine first, wait for repeated failures, and back off ([#&#8203;2090](defilantech/LLMKube#2090)) ([407e02e](defilantech/LLMKube@407e02e))
- **agent:** keep a Metal service running when its Model lookup fails transiently ([#&#8203;2089](defilantech/LLMKube#2089)) ([e22eeab](defilantech/LLMKube@e22eeab))
- **controller:** merge DRA pod scheduling fields and reject multiNode with resourceClaims ([#&#8203;2062](defilantech/LLMKube#2062)) ([9319745](defilantech/LLMKube@9319745))
- no device-plugin GPU limit when a Model uses DRA resourceClaims ([#&#8203;2060](defilantech/LLMKube#2060)) ([8c3e434](defilantech/LLMKube@8c3e434))

##### Documentation

- APISIX in front of LLMKube ([#&#8203;2053](defilantech/LLMKube#2053)) ([16286c6](defilantech/LLMKube@16286c6))
- **metal:** fix install commands and update the quickstart for relay mode ([#&#8203;2087](defilantech/LLMKube#2087)) ([8bc7c6f](defilantech/LLMKube@8bc7c6f))
- use an extension-less link in the GPU fault guide ([#&#8203;2056](defilantech/LLMKube#2056)) ([780e710](defilantech/LLMKube@780e710))

### [`v0.10.3`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0103-2026-10-13)

[Compare Source](defilantech/LLMKube@v0.10.2...v0.10.3)

##### Features

- **api:** accept SGLang's Blackwell attention backends; flag flash_attn ([#&#8203;2033](defilantech/LLMKube#2033)) ([2385fd4](defilantech/LLMKube@2385fd4))
- **chart:** alert on GPU faults the device plugin ignores ([#&#8203;2031](defilantech/LLMKube#2031)) ([e4697ef](defilantech/LLMKube@e4697ef))
- **controller:** add opt-in vLLM engine-stall liveness probe ([#&#8203;2022](defilantech/LLMKube#2022)) ([429434a](defilantech/LLMKube@429434a))
- **controller:** apply the Model's HF_ENDPOINT mirror to runtime downloads ([#&#8203;2039](defilantech/LLMKube#2039)) ([cbececa](defilantech/LLMKube@cbececa))
- **controller:** stall-aware liveness for SGLang pods; share the detector ([#&#8203;2029](defilantech/LLMKube#2029)) ([212b014](defilantech/LLMKube@212b014))
- **router:** serve POST /v1/completions on the router-proxy ([#&#8203;2019](defilantech/LLMKube#2019)) ([1724a02](defilantech/LLMKube@1724a02))

##### Bug Fixes

- **controller:** bind SGLang dual-stack so it can become Ready on IPv4 clusters ([#&#8203;2028](defilantech/LLMKube#2028)) ([7d87c7e](defilantech/LLMKube@7d87c7e))
- **controller:** bound multiNode recreates and keep the failure evidence ([#&#8203;1996](defilantech/LLMKube#1996)) ([3331610](defilantech/LLMKube@3331610))
- **controller:** refuse to overwrite or delete a same-name HPA ([#&#8203;2010](defilantech/LLMKube#2010)) ([09affe5](defilantech/LLMKube@09affe5))
- **controller:** report a missing CA ConfigMap instead of wedging pods ([#&#8203;1995](defilantech/LLMKube#1995)) ([c624e02](defilantech/LLMKube@c624e02))
- **controller:** report AutoscalingReady=True when the HPA is managed ([#&#8203;2012](defilantech/LLMKube#2012)) ([b877ee5](defilantech/LLMKube@b877ee5))
- **controller:** split reconcileMultiNodeGroup under the gocyclo limit ([#&#8203;2000](defilantech/LLMKube#2000)) ([023a5c4](defilantech/LLMKube@023a5c4))
- **controller:** stop the prefetch status loop while the CA ConfigMap is missing ([#&#8203;2037](defilantech/LLMKube#2037)) ([b0b3cc6](defilantech/LLMKube@b0b3cc6))
- **foreman-agent:** make the repo package's git env hermetic and forward GIT_CONFIG\_\* ([#&#8203;2014](defilantech/LLMKube#2014)) ([62837d9](defilantech/LLMKube@62837d9))
- **router:** compile AgentGateway mode against Gateway API Inference Extension v1 ([#&#8203;1990](defilantech/LLMKube#1990)) ([ce817ba](defilantech/LLMKube@ce817ba))
- **router:** refuse to adopt same-name router-proxy objects ([#&#8203;2008](defilantech/LLMKube#2008)) ([e8f190a](defilantech/LLMKube@e8f190a))
- **router:** remove router-proxy objects for routers not in Proxy mode ([#&#8203;2006](defilantech/LLMKube#2006)) ([c04f824](defilantech/LLMKube@c04f824))
- **router:** stop truncating streams at the header timeout; quarantine stalled streams ([#&#8203;2026](defilantech/LLMKube#2026)) ([c4335c9](defilantech/LLMKube@c4335c9))
- **router:** time only upstream silence in the stream idle guard ([#&#8203;2035](defilantech/LLMKube#2035)) ([933426c](defilantech/LLMKube@933426c))

##### Documentation

- **air-gapped:** cover vLLM and SGLang safetensors checkpoints ([#&#8203;1993](defilantech/LLMKube#1993)) ([4d3fa70](defilantech/LLMKube@4d3fa70))
- replace deprecated Strix Halo GTT sizing with TTM parameters ([#&#8203;2003](defilantech/LLMKube#2003)) ([09d8d73](defilantech/LLMKube@09d8d73))

</details>

---

### Configuration

📅 **Schedule**: (in timezone Europe/London)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xNjcuNSIsInVwZGF0ZWRJblZlciI6IjQ0LjE2Ny41IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL3BhdGNoIl19-->

Reviewed-on: https://git.hayden.moe/hayden/phoebe/pulls/885
eleboucher pushed a commit to eleboucher/homelab that referenced this pull request Oct 31, 2026
…0.10.15) (#2331)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [ghcr.io/defilantech/charts/llmkube](https://github.com/defilantech/LLMKube) | HelmChart | patch | `0.10.2` → `0.10.15` |

---

> ⚠️ **Warning**
>
> Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/567) for more information.

---

### Release Notes

<details>
<summary>defilantech/LLMKube (ghcr.io/defilantech/charts/llmkube)</summary>

### [`v0.10.15`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01015-2026-10-31)

[Compare Source](defilantech/LLMKube@v0.10.14...v0.10.15)

##### Features

- **agent:** add --engine-env-passthrough to pass named variables to Metal engines ([6d4fba0](defilantech/LLMKube@6d4fba0))
- **agent:** give Metal engines an allowlisted environment (fixes [#&#8203;2314](defilantech/LLMKube#2314)) ([6d4fba0](defilantech/LLMKube@6d4fba0))
- **foreman:** select the forge provider by name, remote host, or default ([#&#8203;2292](defilantech/LLMKube#2292)) ([239411a](defilantech/LLMKube@239411a))
- **foreman:** stream in-turn generation progress ([#&#8203;2313](defilantech/LLMKube#2313)) ([3d57743](defilantech/LLMKube@3d57743))
- **metal-agent:** read only marked Secrets through sourceSecretRef (fixes [#&#8203;1956](defilantech/LLMKube#1956)) ([eaecad3](defilantech/LLMKube@eaecad3))
- **metal:** limit the metal-agent's Service and EndpointSlice writes by admission policy ([#&#8203;2319](defilantech/LLMKube#2319)) ([a8ea5bf](defilantech/LLMKube@a8ea5bf))
- **release:** publish a Homebrew cask for the metal-agent ([#&#8203;2234](defilantech/LLMKube#2234)) ([1b29497](defilantech/LLMKube@1b29497))
- **router:** mirror requests to the other backends of a shadow rule (fixes [#&#8203;2327](defilantech/LLMKube#2327)) ([40266ab](defilantech/LLMKube@40266ab))

##### Bug Fixes

- **agent:** pin the exact powermetrics argument vector in sudoers ([d28c55f](defilantech/LLMKube@d28c55f))
- **controller:** treat a ModelPool resident removed from spec.members as gone ([#&#8203;2310](defilantech/LLMKube#2310)) ([87464ad](defilantech/LLMKube@87464ad))
- **foreman:** bound the Metal engine port read from the agent's EndpointSlice ([#&#8203;2321](defilantech/LLMKube#2321)) ([75837e3](defilantech/LLMKube@75837e3))
- **metal-agent:** re-check a refused InferenceService on a backoff (fixes [#&#8203;2329](defilantech/LLMKube#2329)) ([6f65b30](defilantech/LLMKube@6f65b30))
- **metal-agent:** re-issue an ingress certificate the Mac's clock no longer accepts ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- **metal-agent:** read sourceSecretRef from the Model's namespace ([eaecad3](defilantech/LLMKube@eaecad3))
- **metal-agent:** retry a start that hit a transient source Secret read error ([5e13d5c](defilantech/LLMKube@5e13d5c))
- **metal-agent:** retry a transient remote size probe failure without refusing (fixes [#&#8203;2326](defilantech/LLMKube#2326)) ([5e13d5c](defilantech/LLMKube@5e13d5c))
- **metal:** bind MetalNode Reachable to the Mac with a UID-named certificate (refs [#&#8203;2265](defilantech/LLMKube#2265)) ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- **metal:** expire a MetalNode Reachable result no check has renewed (refs [#&#8203;2265](defilantech/LLMKube#2265)) ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- **metal:** read MetalNode verifier finish times on the controller's clock (refs [#&#8203;2265](defilantech/LLMKube#2265)) ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- **metal:** trust only the reported certificate in the MetalNode dial-back ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- preserve Foreman audit and transcript records per attempt ([#&#8203;2311](defilantech/LLMKube#2311)) ([3e838f9](defilantech/LLMKube@3e838f9))
- **router:** dispatch only to backends with a rule's required capabilities (fixes [#&#8203;2328](defilantech/LLMKube#2328)) ([e533192](defilantech/LLMKube@e533192))
- **router:** honor the weighted rule strategy on the Proxy data plane (fixes [#&#8203;2325](defilantech/LLMKube#2325)) ([7b34a43](defilantech/LLMKube@7b34a43))

##### Documentation

- **macos:** explain why Login Items lists the agent as Defilan Technologies LLC ([#&#8203;2307](defilantech/LLMKube#2307)) ([5b5e17e](defilantech/LLMKube@5b5e17e))
- move Foreman stream guide before references ([#&#8203;2322](defilantech/LLMKube#2322)) ([d856cb1](defilantech/LLMKube@d856cb1))
- **upgrade:** a MetalNode is Reachable only when the Mac's metal-agent reports an ingress certificate naming its MetalNode. After upgrading the controller, every Mac shows Reachable=False with reason AgentUpgradeRequired, and its hostIP is cleared, until its metal-agent is upgraded too; upgrade the agents promptly. Nothing routes to hostIP yet, so serving is unaffected. The dial-back verifier image's curl must be 7.57 or later with a TLS library that accepts a self-signed certificate as its own trust anchor; the default curlimages/curl image does. ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- **upgrade:** Metal engines now inherit only PATH, HOME, USER, LOGNAME, TMPDIR, LANG, TZ, LC\_\*, HF_HOME and HF_HUB_CACHE from the agent's environment, plus the variables the agent sets for each runtime. If an engine relies on another variable from the agent's launchd environment (for example a GGML or MLX tuning variable, or proxy settings for llama-server's own `-hf` download), list it in `--engine-env-passthrough`. Engines adopted from a previous agent process keep their old environment until they restart; run `llmkube-metal-agent stop-engines` after the bootout to apply it at once. ([6d4fba0](defilantech/LLMKube@6d4fba0))
- **upgrade:** on the Proxy data plane, a ModelRouter rule with match.requiredCapabilities now sends requests only to the backends that advertise all of them, under every strategy. A primary-fallback rule no longer sends to, or falls back to, a backend that lacks them; when every capable backend fails, the request fails instead of landing on an incapable one. ([e533192](defilantech/LLMKube@e533192))
- **upgrade:** on the Proxy data plane, ModelRouter rules with strategy: shadow now send a copy of each request to the rule's other backends, including external ones, which bill those requests. A shadow rule with one backend now fails validation, and one that mirrors to a ModelPool member fails the compile and keeps the previous config. ([40266ab](defilantech/LLMKube@40266ab))
- **upgrade:** on the Proxy data plane, ModelRouter rules with strategy: weighted now split requests by backend weight instead of sending all of them to the first backend. weight: 0 drains a backend, matching the Gateway data plane. A weighted rule whose backends cannot take traffic now fails validation, and a weighted rule across two members of one ModelPool fails the compile and keeps the previous config. ([7b34a43](defilantech/LLMKube@7b34a43))
- **upgrade:** the Metal agent now reads a Model's spec.sourceSecretRef only when the Secret is labeled inference.llmkube.dev/model-source-credentials=true; otherwise the InferenceService is refused with SourceSecretNotAllowed. Before upgrading the agent, label each credentials Secret your Metal Models use: `kubectl label secret <name> -n <namespace> inference.llmkube.dev/model-source-credentials=true`. For one release, `--allow-unlabeled-source-secrets` keeps the old behavior with a warning; it is deprecated. A Hugging Face source whose Secret cannot be read is now refused instead of downloading anonymously. ([eaecad3](defilantech/LLMKube@eaecad3))
- **upgrade:** the powermetrics sudoers rule now names the exact command line the agent runs, with the sampling interval rendered in. Upgrading the agent does not change an installed /etc/sudoers.d/llmkube-powermetrics: on hosts that installed it, re-run `make install-powermetrics-sudo` (with `APPLE_POWER_INTERVAL_MS=<ms>` if the agent uses a non-default `--apple-power-interval`), then check with `sudo -ln | grep powermetrics` that the rule ends in `-i` followed by a number. ([d28c55f](defilantech/LLMKube@d28c55f))

### [`v0.10.14`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01014-2026-10-29)

[Compare Source](defilantech/LLMKube@v0.10.13...v0.10.14)

##### Features

- **cli:** send the endpoint API key from llmkube benchmark ([#&#8203;2297](defilantech/LLMKube#2297)) ([df2e008](defilantech/LLMKube@df2e008))
- **router:** add spec.proxy.drainTimeout for the router-proxy shutdown ([#&#8203;2304](defilantech/LLMKube#2304)) ([113e009](defilantech/LLMKube@113e009))
- **router:** opt-in client address and caller header on the audit log ([#&#8203;2288](defilantech/LLMKube#2288)) ([4d1faac](defilantech/LLMKube@4d1faac))

##### Bug Fixes

- **controller:** keep a ModelPool resident the router re-activates during a reclaim ([#&#8203;2290](defilantech/LLMKube#2290)) ([857daf2](defilantech/LLMKube@857daf2))
- **controller:** remove a key dropped from podLabels or podAnnotations from the pods ([#&#8203;2289](defilantech/LLMKube#2289)) ([ca5df08](defilantech/LLMKube@ca5df08))
- **foreman:** route the remaining GitHub-only paths through the forge seams ([#&#8203;2291](defilantech/LLMKube#2291)) ([a333270](defilantech/LLMKube@a333270))
- **metrics:** each binary serves only the metrics it records ([#&#8203;2296](defilantech/LLMKube#2296)) ([de8ca21](defilantech/LLMKube@de8ca21))

##### Documentation

- **modelpool:** leave member replicas out of GitOps manifests ([#&#8203;2300](defilantech/LLMKube#2300)) ([bcbc357](defilantech/LLMKube@bcbc357))

### [`v0.10.13`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01013-2026-10-28)

[Compare Source](defilantech/LLMKube@v0.10.12...v0.10.13)

##### Features

- **agent:** add the llmkube-metal-agent stop-engines command ([2cd2ec9](defilantech/LLMKube@2cd2ec9))
- **agent:** keep Metal engines serving across metal-agent restarts and adopt them on startup (refs [#&#8203;2079](defilantech/LLMKube#2079)) ([2cd2ec9](defilantech/LLMKube@2cd2ec9))
- **release:** sign and notarize the macOS release binaries ([#&#8203;2233](defilantech/LLMKube#2233)) ([f5ff962](defilantech/LLMKube@f5ff962))
- **router:** podLabels and podAnnotations for router-proxy pods ([#&#8203;2285](defilantech/LLMKube#2285)) ([a736e18](defilantech/LLMKube@a736e18))
- **router:** publish per-backend modalities and context length on /v1/models ([#&#8203;2249](defilantech/LLMKube#2249)) ([2887f54](defilantech/LLMKube@2887f54))

##### Bug Fixes

- apply CRDs server-side in make install and make deploy ([#&#8203;2275](defilantech/LLMKube#2275)) ([edaa4f7](defilantech/LLMKube@edaa4f7))
- **chart:** leave --init-container-user off when its value is unset ([#&#8203;2286](defilantech/LLMKube#2286)) ([ac541db](defilantech/LLMKube@ac541db))
- **cli:** pin only single-writer claims and make the cache pod image configurable ([#&#8203;2178](defilantech/LLMKube#2178)) ([2fe250b](defilantech/LLMKube@2fe250b))
- **controller:** annotate router-proxy pods for scraping under --emit-scrape-annotations ([#&#8203;2268](defilantech/LLMKube#2268)) ([07fb7f4](defilantech/LLMKube@07fb7f4))
- **controller:** chown the cache root to the init container's uid when fsGroup is off ([#&#8203;2273](defilantech/LLMKube#2273)) ([c9d86ad](defilantech/LLMKube@c9d86ad))
- **controller:** create no HPA for multiNode services ([#&#8203;2272](defilantech/LLMKube#2272)) ([c8d6f76](defilantech/LLMKube@c8d6f76))
- **controller:** delete FUSE stager Jobs in the foreground without the feature-gated field ([#&#8203;2269](defilantech/LLMKube#2269)) ([e439f60](defilantech/LLMKube@e439f60))
- **controller:** make FUSE model-cache staging safe across stagers and serving pods ([#&#8203;2276](defilantech/LLMKube#2276)) ([6c7c1fe](defilantech/LLMKube@6c7c1fe))
- **controller:** point the router-proxy Service at the port the proxy listens on ([#&#8203;2280](defilantech/LLMKube#2280)) ([1c69118](defilantech/LLMKube@1c69118))
- **controller:** run the stall-probe tests on a fake clock ([#&#8203;2267](defilantech/LLMKube#2267)) ([661856a](defilantech/LLMKube@661856a))
- **router:** tell client cancellations, upstream errors and timeouts apart ([#&#8203;2284](defilantech/LLMKube#2284)) ([3a2ea74](defilantech/LLMKube@3a2ea74))
- use vLLM's real TurboQuant KV cache types in the sample and docs ([#&#8203;2270](defilantech/LLMKube#2270)) ([3b4ffb6](defilantech/LLMKube@3b4ffb6))

##### Documentation

- **upgrade:** metal-agent engines now keep running when the agent stops and the next agent adopts them. To roll back to an older agent, run `launchctl bootout gui/$(id -u)/com.llmkube.metal-agent` and then `llmkube-metal-agent stop-engines` with the current binary before installing the older one, or every model is loaded twice. `make uninstall-metal-agent` stops the engines for you. Adopted engines keep the flags they started with; run stop-engines after the bootout to apply new defaults. ([2cd2ec9](defilantech/LLMKube@2cd2ec9))

### [`v0.10.12`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01012-2026-10-27)

[Compare Source](defilantech/LLMKube@v0.10.11...v0.10.12)

##### Features

- **api:** add MetalNode, verified by a controller dial-back, with the agent reporting through its Lease ([#&#8203;2235](defilantech/LLMKube#2235)) ([fccc1bc](defilantech/LLMKube@fccc1bc))
- **cli:** add llmkube metal join and leave, with agent token renewal ([#&#8203;2246](defilantech/LLMKube#2246)) ([b1f6602](defilantech/LLMKube@b1f6602))
- **cli:** add llmkube metal token create and metal remove ([#&#8203;2236](defilantech/LLMKube#2236)) ([f54414a](defilantech/LLMKube@f54414a))

##### Bug Fixes

- **controller:** keep a ModelPool activation while its member is held at zero ([#&#8203;2241](defilantech/LLMKube#2241)) ([8fc4143](defilantech/LLMKube@8fc4143))
- **controller:** map sglangConfig.enablePrefixCaching onto --disable-radix-cache ([#&#8203;2259](defilantech/LLMKube#2259)) ([d499aa4](defilantech/LLMKube@d499aa4))
- **controller:** stage under the cache lock when the init image's stat has no -f ([#&#8203;2260](defilantech/LLMKube#2260)) ([52241de](defilantech/LLMKube@52241de))
- **router:** list each backend on /v1/models under the id BackendNameMatch routes ([#&#8203;2257](defilantech/LLMKube#2257)) ([4627cfc](defilantech/LLMKube@4627cfc))
- **router:** report moves off a quarantined pod as reason="quarantined" instead of reason="not_ready" ([faa7a5f](defilantech/LLMKube@faa7a5f))
- **router:** rewrite vLLM and SGLang backends to the name they serve ([#&#8203;2258](defilantech/LLMKube#2258)) ([3d39111](defilantech/LLMKube@3d39111))
- validate multi-file cache staging on shared volumes ([#&#8203;2185](defilantech/LLMKube#2185)) ([e6eab85](defilantech/LLMKube@e6eab85))

### [`v0.10.11`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01011-2026-10-25)

[Compare Source](defilantech/LLMKube@v0.10.10...v0.10.11)

##### Features

- **api:** accept fp8, float16 and bfloat16 in vllmConfig.kvCacheDtype ([#&#8203;2227](defilantech/LLMKube#2227)) ([31eb943](defilantech/LLMKube@31eb943))
- **api:** add spec.imagePullPolicy to InferenceService ([#&#8203;2208](defilantech/LLMKube#2208)) ([ec54d28](defilantech/LLMKube@ec54d28))
- **controller:** move the SGLang defaults to v0.5.22 and report removed attention backends ([#&#8203;2221](defilantech/LLMKube#2221)) ([33f9bb1](defilantech/LLMKube@33f9bb1))
- **controller:** move the vLLM default to v0.32.0 and check every flag against it ([#&#8203;2223](defilantech/LLMKube#2223)) ([0144a4c](defilantech/LLMKube@0144a4c))
- **controller:** run every llama.cpp default from one pinned build, b11854 ([#&#8203;2220](defilantech/LLMKube#2220)) ([052840e](defilantech/LLMKube@052840e))
- **router:** weigh LoadAware placement by estimated prefill tokens in flight ([#&#8203;2193](defilantech/LLMKube#2193)) ([a33735c](defilantech/LLMKube@a33735c))

##### Bug Fixes

- **controller:** render vllmConfig.speculative as --speculative-config ([#&#8203;2210](defilantech/LLMKube#2210)) ([191dcbd](defilantech/LLMKube@191dcbd))
- **controller:** report sglangConfig tool-call parsers SGLang does not register ([#&#8203;2228](defilantech/LLMKube#2228)) ([90fed2e](defilantech/LLMKube@90fed2e))

### [`v0.10.10`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01010-2026-10-24)

[Compare Source](defilantech/LLMKube@v0.10.9...v0.10.10)

##### Features

- **controller:** add --model-cache-prep to skip the root cache prep ([#&#8203;2206](defilantech/LLMKube#2206)) ([8bcb973](defilantech/LLMKube@8bcb973))
- **router:** compile dataPlane: AgentGateway to AgentgatewayModel ([#&#8203;2162](defilantech/LLMKube#2162)) ([26b1ef4](defilantech/LLMKube@26b1ef4))

##### Bug Fixes

- **controller:** report a prefetch as downloading while its downloader runs ([#&#8203;2207](defilantech/LLMKube#2207)) ([7699514](defilantech/LLMKube@7699514))
- **router:** let every waiter of a failed ModelPool swap see its outcome ([#&#8203;2203](defilantech/LLMKube#2203)) ([b95db62](defilantech/LLMKube@b95db62))

### [`v0.10.9`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0109-2026-10-23)

[Compare Source](defilantech/LLMKube@v0.10.8...v0.10.9)

##### Features

- **api:** affinity and topology spread for the router proxy, rolling without surge under hard placement ([#&#8203;2200](defilantech/LLMKube#2200)) ([3a61700](defilantech/LLMKube@3a61700))
- **router:** proxy the Anthropic Messages API like chat completions ([#&#8203;2201](defilantech/LLMKube#2201)) ([86223aa](defilantech/LLMKube@86223aa))

##### Bug Fixes

- **router:** retry stale keep-alive connections and quarantine a pod only after two failures ([#&#8203;2197](defilantech/LLMKube#2197)) ([b279053](defilantech/LLMKube@b279053))

### [`v0.10.8`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0108-2026-10-23)

[Compare Source](defilantech/LLMKube@v0.10.7...v0.10.8)

##### Features

- **api:** type the vLLM tool-call and reasoning parsers and admit GLM on SGLang ([#&#8203;2168](defilantech/LLMKube#2168)) ([fd64c72](defilantech/LLMKube@fd64c72))
- **controller:** add InferenceService spec.hubOffline for serving pods ([#&#8203;2188](defilantech/LLMKube#2188)) ([a911cb2](defilantech/LLMKube@a911cb2))
- **controller:** keep 3 old ReplicaSets for LLMKube-managed Deployments by default ([#&#8203;2182](defilantech/LLMKube#2182)) ([5e1722f](defilantech/LLMKube@5e1722f))
- **router:** honour endpoint nodePort and externalTrafficPolicy, add proxy runtimeClassName ([#&#8203;2183](defilantech/LLMKube#2183)) ([a8f9e57](defilantech/LLMKube@a8f9e57))
- **router:** on pools under 3 pods, move a session only when its pod is KV-full ([#&#8203;2192](defilantech/LLMKube#2192)) ([f9b934c](defilantech/LLMKube@f9b934c))
- **router:** remember LoadAware sessions and move them only under real pressure ([#&#8203;2180](defilantech/LLMKube#2180)) ([1e8ce5a](defilantech/LLMKube@1e8ce5a))

##### Bug Fixes

- **router:** answer a ModelPool swap past swapBudget with 503 and Retry-After ([#&#8203;2181](defilantech/LLMKube#2181)) ([7c0d84e](defilantech/LLMKube@7c0d84e))
- **router:** release request bodies once sent and raise the proxy's default memory limit ([#&#8203;2196](defilantech/LLMKube#2196)) ([140fd6b](defilantech/LLMKube@140fd6b))

### [`v0.10.7`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0107-2026-10-23)

[Compare Source](defilantech/LLMKube@v0.10.6...v0.10.7)

##### Features

- **chart:** install the metal-agent RBAC from the Helm chart ([#&#8203;2165](defilantech/LLMKube#2165)) ([521e502](defilantech/LLMKube@521e502))
- **controller:** roll a multi-replica GPU service one pod at a time ([#&#8203;2164](defilantech/LLMKube#2164)) ([c4073cf](defilantech/LLMKube@c4073cf))
- **router:** load-aware, session-sticky placement for endpoint backends ([#&#8203;2166](defilantech/LLMKube#2166)) ([74cfb0e](defilantech/LLMKube@74cfb0e))
- **router:** the router proxy tracks endpoint backends' pods itself ([#&#8203;2159](defilantech/LLMKube#2159)) ([57070e1](defilantech/LLMKube@57070e1))

### [`v0.10.6`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0106-2026-10-22)

[Compare Source](defilantech/LLMKube@v0.10.5...v0.10.6)

##### Features

- **controller:** prefetch per cache claim, gated by the InferenceService ([#&#8203;2138](defilantech/LLMKube#2138)) ([d76f398](defilantech/LLMKube@d76f398))
- **router:** per-backend Endpoint Picker for dataPlane: AgentGateway ([#&#8203;2157](defilantech/LLMKube#2157)) ([c1bbe28](defilantech/LLMKube@c1bbe28))

##### Bug Fixes

- **cli:** read the model cache from its own pod in cache list ([#&#8203;2154](defilantech/LLMKube#2154)) ([996abe9](defilantech/LLMKube@996abe9))
- **controller:** refuse to repoint a same-name Service ([#&#8203;2149](defilantech/LLMKube#2149)) ([7965a1b](defilantech/LLMKube@7965a1b))
- **hf-listing:** read a tree size only when it is a whole number of bytes ([#&#8203;2153](defilantech/LLMKube#2153)) ([6e6723d](defilantech/LLMKube@6e6723d))
- **model:** report the staged total instead of status.size "0" ([#&#8203;2150](defilantech/LLMKube#2150)) ([f6710e7](defilantech/LLMKube@f6710e7))

### [`v0.10.5`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0105-2026-10-21)

[Compare Source](defilantech/LLMKube@v0.10.4...v0.10.5)

##### Features

- add make gate and make test-affected, a cached parallel round gate ([#&#8203;2106](defilantech/LLMKube#2106)) ([e971675](defilantech/LLMKube@e971675))
- **agent:** admit Metal models against memory already committed to running engines (closes [#&#8203;2080](defilantech/LLMKube#2080)) ([d2d2d43](defilantech/LLMKube@d2d2d43))
- **agent:** record each engine the metal-agent starts in the state directory (refs [#&#8203;2079](defilantech/LLMKube#2079)) ([0224d51](defilantech/LLMKube@0224d51))
- **controller:** add spec.startupTimeout for a bigger startup probe budget ([#&#8203;2129](defilantech/LLMKube#2129)) ([df0e9cd](defilantech/LLMKube@df0e9cd))
- **controller:** set fsGroupChangePolicy OnRootMismatch with the default fsGroup (closes [#&#8203;2058](defilantech/LLMKube#2058)) ([2bd4538](defilantech/LLMKube@2bd4538))

##### Bug Fixes

- **agent:** escalate to SIGKILL when stopping an engine the agent did not spawn ([0224d51](defilantech/LLMKube@0224d51))
- **agent:** keep the pressure block when an eviction teardown completed ([#&#8203;2107](defilantech/LLMKube#2107)) ([fd9ab77](defilantech/LLMKube@fd9ab77))
- **agent:** replace a hung oMLX daemon instead of starting a second one ([#&#8203;2105](defilantech/LLMKube#2105)) ([a16c348](defilantech/LLMKube@a16c348))
- **agent:** restart services evicted under memory pressure once memory recovers ([#&#8203;2093](defilantech/LLMKube#2093)) ([3db0ecc](defilantech/LLMKube@3db0ecc))
- answer 400 for request bodies that are not valid JSON ([#&#8203;2075](defilantech/LLMKube#2075)) ([d9826ff](defilantech/LLMKube@d9826ff))
- **chart:** fail fast on shared cache without default storageclass and support existingClaim ([#&#8203;2016](defilantech/LLMKube#2016)) ([#&#8203;2114](defilantech/LLMKube#2114)) ([c4f7f3d](defilantech/LLMKube@c4f7f3d))
- **chart:** gate llama.cpp InferenceAdmissionStalled on decode progress ([#&#8203;2065](defilantech/LLMKube#2065)) ([5b2f45d](defilantech/LLMKube@5b2f45d))
- **cli:** make cache clear and delete --purge-cache remove entries ([#&#8203;2134](defilantech/LLMKube#2134)) ([2b668e2](defilantech/LLMKube@2b668e2))
- **controller:** keep downloader credentials off curl's command line ([#&#8203;2098](defilantech/LLMKube#2098)) ([2207c2e](defilantech/LLMKube@2207c2e))
- **controller:** mount a memory-backed /dev/shm for multi-process vLLM, SGLang and TGI (closes [#&#8203;2099](defilantech/LLMKube#2099)) ([a68c98f](defilantech/LLMKube@a68c98f))
- **controller:** refuse to overwrite or recreate a same-name Deployment ([#&#8203;2096](defilantech/LLMKube#2096)) ([b90c8aa](defilantech/LLMKube@b90c8aa))
- **model:** canonicalize HF repo IDs so a trailing slash stages ([#&#8203;2128](defilantech/LLMKube#2128)) ([9289ac6](defilantech/LLMKube@9289ac6))
- **modelpool:** stop swap thrash on reversals, idle-probe blindness, and abandoned loads ([#&#8203;2130](defilantech/LLMKube#2130)) ([#&#8203;2135](defilantech/LLMKube#2135)) ([e7dee43](defilantech/LLMKube@e7dee43))
- **model:** serialize downloaders that share a cache directory ([#&#8203;2115](defilantech/LLMKube#2115)) ([13eca01](defilantech/LLMKube@13eca01))
- **sglang:** keep endpoint API keys out of process arguments ([#&#8203;2122](defilantech/LLMKube#2122)) ([7fdae50](defilantech/LLMKube@7fdae50))
- size metal-agent KV cache estimate from GQA attention shape ([#&#8203;2102](defilantech/LLMKube#2102)) ([a119234](defilantech/LLMKube@a119234))

##### Documentation

- cover per-request balancing in the ModelRouter page ([#&#8203;2095](defilantech/LLMKube#2095)) ([9ec4766](defilantech/LLMKube@9ec4766))
- **model-cache:** correct the CephFS fsGroupPolicy claim and the PSA advice ([2bd4538](defilantech/LLMKube@2bd4538))
- state differentiation, audience, and project status up front ([#&#8203;2100](defilantech/LLMKube#2100)) ([ee27e20](defilantech/LLMKube@ee27e20))
- **upgrade:** every InferenceService that uses the default pod security context rolls once on upgrade, because the generated pod template gains `fsGroupChangePolicy: OnRootMismatch` (a full model reload for large models) ([2bd4538](defilantech/LLMKube@2bd4538))
- **upgrade:** single-node vLLM, SGLang and TGI services with more than one GPU (or a tensor, pipeline or data-parallel size above 1) that do not mount /dev/shm themselves roll once on upgrade, because the generated pod template gains a memory-backed /dev/shm volume ([a68c98f](defilantech/LLMKube@a68c98f))
- **upgrade:** the metal-agent now admits a model only if it fits in the Mac's budget minus the memory committed to models already running, so a Mac whose models add up to more than its budget refuses the extra ones; a Model memoryBudget or memoryFraction above the Mac's budget is clamped to it unless the agent runs with --allow-model-memory-budget-override ([d2d2d43](defilantech/LLMKube@d2d2d43))
- **upgrade:** the metal-agent now writes a record per engine under --state-dir/engines (both relay and legacy mode) and fails an engine start when it cannot write it; make sure the state directory (default \~/Library/Application Support/llmkube/metal-agent) is writable by the agent's user ([0224d51](defilantech/LLMKube@0224d51))

### [`v0.10.4`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0104-2026-10-16)

[Compare Source](defilantech/LLMKube@v0.10.3...v0.10.4)

##### Features

- **agent:** cap the bytes a single Model download may write ([#&#8203;2088](defilantech/LLMKube#2088)) ([3852411](defilantech/LLMKube@3852411))
- **api:** endpoint API key and externalTrafficPolicy for InferenceService ([#&#8203;2043](defilantech/LLMKube#2043)) ([27a9214](defilantech/LLMKube@27a9214))
- **controller:** detect a stalled watch path with a lease canary, bounded reconciles and stall alerts ([#&#8203;2072](defilantech/LLMKube#2072)) ([d00e0cf](defilantech/LLMKube@d00e0cf))
- **controller:** detect keyed pool siblings and vLLM env-var keys in ModelRouter ([#&#8203;2068](defilantech/LLMKube#2068)) ([78889f0](defilantech/LLMKube@78889f0))
- **controller:** detect ModelRouter backends keyed through --api-key on the command line ([#&#8203;2063](defilantech/LLMKube#2063)) ([3674acc](defilantech/LLMKube@3674acc))
- **controller:** stage HF mirror repos with spec.files and expand globs from the repo listing ([#&#8203;2071](defilantech/LLMKube#2071)) ([9e5d7d3](defilantech/LLMKube@9e5d7d3))

##### Bug Fixes

- **agent:** health restarts stop the unhealthy engine first, wait for repeated failures, and back off ([#&#8203;2090](defilantech/LLMKube#2090)) ([407e02e](defilantech/LLMKube@407e02e))
- **agent:** keep a Metal service running when its Model lookup fails transiently ([#&#8203;2089](defilantech/LLMKube#2089)) ([e22eeab](defilantech/LLMKube@e22eeab))
- **controller:** merge DRA pod scheduling fields and reject multiNode with resourceClaims ([#&#8203;2062](defilantech/LLMKube#2062)) ([9319745](defilantech/LLMKube@9319745))
- no device-plugin GPU limit when a Model uses DRA resourceClaims ([#&#8203;2060](defilantech/LLMKube#2060)) ([8c3e434](defilantech/LLMKube@8c3e434))

##### Documentation

- APISIX in front of LLMKube ([#&#8203;2053](defilantech/LLMKube#2053)) ([16286c6](defilantech/LLMKube@16286c6))
- **metal:** fix install commands and update the quickstart for relay mode ([#&#8203;2087](defilantech/LLMKube#2087)) ([8bc7c6f](defilantech/LLMKube@8bc7c6f))
- use an extension-less link in the GPU fault guide ([#&#8203;2056](defilantech/LLMKube#2056)) ([780e710](defilantech/LLMKube@780e710))

### [`v0.10.3`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0103-2026-10-13)

[Compare Source](defilantech/LLMKube@v0.10.2...v0.10.3)

##### Features

- **api:** accept SGLang's Blackwell attention backends; flag flash_attn ([#&#8203;2033](defilantech/LLMKube#2033)) ([2385fd4](defilantech/LLMKube@2385fd4))
- **chart:** alert on GPU faults the device plugin ignores ([#&#8203;2031](defilantech/LLMKube#2031)) ([e4697ef](defilantech/LLMKube@e4697ef))
- **controller:** add opt-in vLLM engine-stall liveness probe ([#&#8203;2022](defilantech/LLMKube#2022)) ([429434a](defilantech/LLMKube@429434a))
- **controller:** apply the Model's HF_ENDPOINT mirror to runtime downloads ([#&#8203;2039](defilantech/LLMKube#2039)) ([cbececa](defilantech/LLMKube@cbececa))
- **controller:** stall-aware liveness for SGLang pods; share the detector ([#&#8203;2029](defilantech/LLMKube#2029)) ([212b014](defilantech/LLMKube@212b014))
- **router:** serve POST /v1/completions on the router-proxy ([#&#8203;2019](defilantech/LLMKube#2019)) ([1724a02](defilantech/LLMKube@1724a02))

##### Bug Fixes

- **controller:** bind SGLang dual-stack so it can become Ready on IPv4 clusters ([#&#8203;2028](defilantech/LLMKube#2028)) ([7d87c7e](defilantech/LLMKube@7d87c7e))
- **controller:** bound multiNode recreates and keep the failure evidence ([#&#8203;1996](defilantech/LLMKube#1996)) ([3331610](defilantech/LLMKube@3331610))
- **controller:** refuse to overwrite or delete a same-name HPA ([#&#8203;2010](defilantech/LLMKube#2010)) ([09affe5](defilantech/LLMKube@09affe5))
- **controller:** report a missing CA ConfigMap instead of wedging pods ([#&#8203;1995](defilantech/LLMKube#1995)) ([c624e02](defilantech/LLMKube@c624e02))
- **controller:** report AutoscalingReady=True when the HPA is managed ([#&#8203;2012](defilantech/LLMKube#2012)) ([b877ee5](defilantech/LLMKube@b877ee5))
- **controller:** split reconcileMultiNodeGroup under the gocyclo limit ([#&#8203;2000](defilantech/LLMKube#2000)) ([023a5c4](defilantech/LLMKube@023a5c4))
- **controller:** stop the prefetch status loop while the CA ConfigMap is missing ([#&#8203;2037](defilantech/LLMKube#2037)) ([b0b3cc6](defilantech/LLMKube@b0b3cc6))
- **foreman-agent:** make the repo package's git env hermetic and forward GIT_CONFIG\_\* ([#&#8203;2014](defilantech/LLMKube#2014)) ([62837d9](defilantech/LLMKube@62837d9))
- **router:** compile AgentGateway mode against Gateway API Inference Extension v1 ([#&#8203;1990](defilantech/LLMKube#1990)) ([ce817ba](defilantech/LLMKube@ce817ba))
- **router:** refuse to adopt same-name router-proxy objects ([#&#8203;2008](defilantech/LLMKube#2008)) ([e8f190a](defilantech/LLMKube@e8f190a))
- **router:** remove router-proxy objects for routers not in Proxy mode ([#&#8203;2006](defilantech/LLMKube#2006)) ([c04f824](defilantech/LLMKube@c04f824))
- **router:** stop truncating streams at the header timeout; quarantine stalled streams ([#&#8203;2026](defilantech/LLMKube#2026)) ([c4335c9](defilantech/LLMKube@c4335c9))
- **router:** time only upstream silence in the stream idle guard ([#&#8203;2035](defilantech/LLMKube#2035)) ([933426c](defilantech/LLMKube@933426c))

##### Documentation

- **air-gapped:** cover vLLM and SGLang safetensors checkpoints ([#&#8203;1993](defilantech/LLMKube#1993)) ([4d3fa70](defilantech/LLMKube@4d3fa70))
- replace deprecated Strix Halo GTT sizing with TTM parameters ([#&#8203;2003](defilantech/LLMKube#2003)) ([09d8d73](defilantech/LLMKube@09d8d73))

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xNTEuMiIsInVwZGF0ZWRJblZlciI6IjQ0LjE4MS45IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://git.erwanleboucher.dev/eleboucher/homelab/pulls/2331
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TestRemoteURL fails under a global git insteadOf rewrite; runGit drops GIT_CONFIG_* overrides

1 participant