Skip to content

feat(controller): run every llama.cpp default from one pinned build, b11854 - #2220

Merged
Defilan merged 1 commit into
defilantech:mainfrom
Defilan:chore/llamacpp-pin-b11854-2214
Oct 24, 2026
Merged

feat(controller): run every llama.cpp default from one pinned build, b11854#2220
Defilan merged 1 commit into
defilantech:mainfrom
Defilan:chore/llamacpp-pin-b11854-2214

Conversation

@Defilan

@Defilan Defilan commented Oct 24, 2026

Copy link
Copy Markdown
Member

What

Every default llama.cpp image now comes from one pinned upstream build, b11854: the CPU default (was the floating :server tag), the NVIDIA image (was server-cuda-b10068, from 2026-07-18) and router mode (also the floating tag).

Why

With a floating CPU tag, two rollouts of one service could run different llama.cpp servers. With CPU and CUDA pinned to builds three months apart, a flag could work or fail depending on whether the Model declares a GPU. Build 11146 removing --mlock is that kind of break (#1916).

Fixes #2214

How

  • llamaCppBuild = "b11854" in runtime_llamacpp.go. llamaCppCPUImage (server-b11854), llamaCppCUDAImage (server-cuda-b11854) and LlamaCppRouterBackend.DefaultImage all derive from it.
  • b11854 is the newest build with both server- and server-cuda- images on GHCR. Upstream doesn't publish images for every build: the newer builds up to today's b11861 have no CUDA image yet.
  • Router mode wasn't named in the issue, but it used the same floating tag, so it moves too.
  • TestLlamaCppDefaultImagesShareOneBuild fails if any of the three defaults leaves the shared build or drops the per-build suffix.
  • The cache-prep tripwire (cache_prep_tripwire_test.go) now sets the image its hashes were recorded with (:server). It guards the prep policy, and should not move with image bumps. Its hashes are unchanged, which also shows the image is the only part of that pod template this PR changes.
  • Two exact-match Deployment tests compare against llamaCppCPUImage instead of the old literal.
  • The spec.image field doc no longer names a tag. docs/air-gapped-quickstart.md, docs/dgx-spark-microk8s.md and the B200 matrix note move from b10068/:server to b11854.

Upgrade note: every llama.cpp InferenceService without spec.image or runtimeImages.llamacpp rolls once onto b11854. Air-gapped users who mirror :server need server-b11854 instead.

Testing

  • TestLlamaCppDefaultImagesShareOneBuild: failed before the change (undefined llamaCppBuild) and passes after. Setting only llamaCppCUDAImage back to server-cuda-b10068 makes it fail with want the per-build tag for b11854.
  • Flag parity: every flag the llama.cpp builders can emit (37) appears in llama-server --help from both the server-b11854 and server-cuda-b11854 images. No flag that b11146 had is missing in b11854.
  • Live on a GB10 (DGX Spark, arm64, driver 580.173.02) through the deployed 0.10.9 operator with spec.image set to each candidate:
    • CPU server-b11854: /v1/models 200, chat 200 with the correct answer, /metrics exposes the llamacpp: series.
    • CUDA server-cuda-b11854 (--n-gpu-layers 99): the same, about 360 tok/s decode after a one-time JIT on the first request (the image has no sm_121 code, as the dgx-spark-microk8s.md note already says).
  • internal/controller suite: passes in 167 s (GINKGO_SEED=4242). go test ./api/... ./pkg/cli/... passes.
  • golangci-lint v2.12.2: 0 issues on darwin and linux.
  • make generate manifests chart-crds foreman-chart-crds federation-chart-crds is clean against the commit (controller-gen v0.19.0, matching main).

AI assistance

Assisted-by: Claude Code (Claude Opus 5.5; roles: picked the build, made the change and tests, ran the live checks above)

Verified by the author: Read diffs and supervised build

Checklist

  • Tests added/updated
  • make test passes locally (internal/controller, api, pkg/cli)
  • make lint passes locally
  • Commit messages follow conventional commits
  • All commits are signed off (git commit -s) per DCO
  • AI assistance (if any) is disclosed above, per CONTRIBUTING.md
  • Documentation updated (if user-facing change)

@Defilan
chore(deps): run every llama.cpp default from one pinned build, b11854
f48dd4b 
The CPU default was the floating ghcr.io/ggml-org/llama.cpp:server tag and
the NVIDIA default was server-cuda-b10068 from 2026-07-18, so two rollouts of
one CPU service could run different servers, and a flag could work or fail
depending on whether the Model declares a GPU. Router mode also used the
floating tag.

llamaCppBuild now names one upstream build, and the CPU default, the CUDA
image and router mode all derive their tags from it. A test fails if any of
the three moves to a different build or drops the per-build suffix.

b11854 is the newest build with both server- and server-cuda- images on GHCR
(upstream does not publish images for every build). Every flag the llama.cpp
builders can emit is in its --help, the same as b11146, and on a GB10 both
images served chat through the operator: CPU and CUDA, the CUDA one at about
360 tok/s after its first-request JIT. The CUDA image is still a CUDA 12.8.1
build with no native sm_100 code, so the B200 caveat stands.

The cache-prep tripwire now sets the image its hashes were recorded with, so
it keeps guarding the prep policy instead of tracking image bumps; its hashes
are unchanged. Docs that named b10068 or :server move to b11854.

Refs defilantech#2214

Signed-off-by: Christopher Maher <chris@mahercode.io>
@Defilan Defilan added dependencies Pull requests that update a dependency file area/runtime Runtime backend and inference engine support component/controller Related to the operator controller docker Docker related changes labels Oct 24, 2026
@codecov

codecov Bot commented Oct 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Defilan Defilan changed the title chore(deps): run every llama.cpp default from one pinned build, b11854 feat(controller): run every llama.cpp default from one pinned build, b11854 Oct 24, 2026
@Defilan
Defilan merged commit 052840e into defilantech:main Oct 24, 2026
31 checks passed
@github-actions github-actions Bot mentioned this pull request Oct 24, 2026
Merged
hbjydev pushed a commit to hbjydev/phoebe that referenced this pull request Oct 26, 2026
hayden-bot git.hayden.moe Instance
fix(image): update image ghcr.io/defilantech/charts/llmkube (0.10.7 →…
4e169c4 
… 0.10.11) (#981)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [ghcr.io/defilantech/charts/llmkube](https://github.com/defilantech/LLMKube) | HelmChart | patch | `0.10.7` → `0.10.11` |

---

### Release Notes

<details>
<summary>defilantech/LLMKube (ghcr.io/defilantech/charts/llmkube)</summary>

### [`v0.10.11`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01011-2026-10-25)

[Compare Source](defilantech/LLMKube@v0.10.10...v0.10.11)

##### Features

- **api:** accept fp8, float16 and bfloat16 in vllmConfig.kvCacheDtype ([#&#8203;2227](defilantech/LLMKube#2227)) ([31eb943](defilantech/LLMKube@31eb943))
- **api:** add spec.imagePullPolicy to InferenceService ([#&#8203;2208](defilantech/LLMKube#2208)) ([ec54d28](defilantech/LLMKube@ec54d28))
- **controller:** move the SGLang defaults to v0.5.22 and report removed attention backends ([#&#8203;2221](defilantech/LLMKube#2221)) ([33f9bb1](defilantech/LLMKube@33f9bb1))
- **controller:** move the vLLM default to v0.32.0 and check every flag against it ([#&#8203;2223](defilantech/LLMKube#2223)) ([0144a4c](defilantech/LLMKube@0144a4c))
- **controller:** run every llama.cpp default from one pinned build, b11854 ([#&#8203;2220](defilantech/LLMKube#2220)) ([052840e](defilantech/LLMKube@052840e))
- **router:** weigh LoadAware placement by estimated prefill tokens in flight ([#&#8203;2193](defilantech/LLMKube#2193)) ([a33735c](defilantech/LLMKube@a33735c))

##### Bug Fixes

- **controller:** render vllmConfig.speculative as --speculative-config ([#&#8203;2210](defilantech/LLMKube#2210)) ([191dcbd](defilantech/LLMKube@191dcbd))
- **controller:** report sglangConfig tool-call parsers SGLang does not register ([#&#8203;2228](defilantech/LLMKube#2228)) ([90fed2e](defilantech/LLMKube@90fed2e))

### [`v0.10.10`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01010-2026-10-24)

[Compare Source](defilantech/LLMKube@v0.10.9...v0.10.10)

##### Features

- **controller:** add --model-cache-prep to skip the root cache prep ([#&#8203;2206](defilantech/LLMKube#2206)) ([8bcb973](defilantech/LLMKube@8bcb973))
- **router:** compile dataPlane: AgentGateway to AgentgatewayModel ([#&#8203;2162](defilantech/LLMKube#2162)) ([26b1ef4](defilantech/LLMKube@26b1ef4))

##### Bug Fixes

- **controller:** report a prefetch as downloading while its downloader runs ([#&#8203;2207](defilantech/LLMKube#2207)) ([7699514](defilantech/LLMKube@7699514))
- **router:** let every waiter of a failed ModelPool swap see its outcome ([#&#8203;2203](defilantech/LLMKube#2203)) ([b95db62](defilantech/LLMKube@b95db62))

### [`v0.10.9`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0109-2026-10-23)

[Compare Source](defilantech/LLMKube@v0.10.8...v0.10.9)

##### Features

- **api:** affinity and topology spread for the router proxy, rolling without surge under hard placement ([#&#8203;2200](defilantech/LLMKube#2200)) ([3a61700](defilantech/LLMKube@3a61700))
- **router:** proxy the Anthropic Messages API like chat completions ([#&#8203;2201](defilantech/LLMKube#2201)) ([86223aa](defilantech/LLMKube@86223aa))

##### Bug Fixes

- **router:** retry stale keep-alive connections and quarantine a pod only after two failures ([#&#8203;2197](defilantech/LLMKube#2197)) ([b279053](defilantech/LLMKube@b279053))

### [`v0.10.8`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0108-2026-10-23)

[Compare Source](defilantech/LLMKube@v0.10.7...v0.10.8)

##### Features

- **api:** type the vLLM tool-call and reasoning parsers and admit GLM on SGLang ([#&#8203;2168](defilantech/LLMKube#2168)) ([fd64c72](defilantech/LLMKube@fd64c72))
- **controller:** add InferenceService spec.hubOffline for serving pods ([#&#8203;2188](defilantech/LLMKube#2188)) ([a911cb2](defilantech/LLMKube@a911cb2))
- **controller:** keep 3 old ReplicaSets for LLMKube-managed Deployments by default ([#&#8203;2182](defilantech/LLMKube#2182)) ([5e1722f](defilantech/LLMKube@5e1722f))
- **router:** honour endpoint nodePort and externalTrafficPolicy, add proxy runtimeClassName ([#&#8203;2183](defilantech/LLMKube#2183)) ([a8f9e57](defilantech/LLMKube@a8f9e57))
- **router:** on pools under 3 pods, move a session only when its pod is KV-full ([#&#8203;2192](defilantech/LLMKube#2192)) ([f9b934c](defilantech/LLMKube@f9b934c))
- **router:** remember LoadAware sessions and move them only under real pressure ([#&#8203;2180](defilantech/LLMKube#2180)) ([1e8ce5a](defilantech/LLMKube@1e8ce5a))

##### Bug Fixes

- **router:** answer a ModelPool swap past swapBudget with 503 and Retry-After ([#&#8203;2181](defilantech/LLMKube#2181)) ([7c0d84e](defilantech/LLMKube@7c0d84e))
- **router:** release request bodies once sent and raise the proxy's default memory limit ([#&#8203;2196](defilantech/LLMKube#2196)) ([140fd6b](defilantech/LLMKube@140fd6b))

</details>

---

### Configuration

📅 **Schedule**: (in timezone Europe/London)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xNzguNSIsInVwZGF0ZWRJblZlciI6IjQ0LjE3OC41IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL3BhdGNoIl19-->

Reviewed-on: https://git.hayden.moe/hayden/phoebe/pulls/981
eleboucher pushed a commit to eleboucher/homelab that referenced this pull request Oct 31, 2026
bot-owl @eleboucher
fix(helm): update chart ghcr.io/defilantech/charts/llmkube (0.10.2 ➔ …
a27c13b 
…0.10.15) (#2331)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [ghcr.io/defilantech/charts/llmkube](https://github.com/defilantech/LLMKube) | HelmChart | patch | `0.10.2` → `0.10.15` |

---

> ⚠️ **Warning**
>
> Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/567) for more information.

---

### Release Notes

<details>
<summary>defilantech/LLMKube (ghcr.io/defilantech/charts/llmkube)</summary>

### [`v0.10.15`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01015-2026-10-31)

[Compare Source](defilantech/LLMKube@v0.10.14...v0.10.15)

##### Features

- **agent:** add --engine-env-passthrough to pass named variables to Metal engines ([6d4fba0](defilantech/LLMKube@6d4fba0))
- **agent:** give Metal engines an allowlisted environment (fixes [#&#8203;2314](defilantech/LLMKube#2314)) ([6d4fba0](defilantech/LLMKube@6d4fba0))
- **foreman:** select the forge provider by name, remote host, or default ([#&#8203;2292](defilantech/LLMKube#2292)) ([239411a](defilantech/LLMKube@239411a))
- **foreman:** stream in-turn generation progress ([#&#8203;2313](defilantech/LLMKube#2313)) ([3d57743](defilantech/LLMKube@3d57743))
- **metal-agent:** read only marked Secrets through sourceSecretRef (fixes [#&#8203;1956](defilantech/LLMKube#1956)) ([eaecad3](defilantech/LLMKube@eaecad3))
- **metal:** limit the metal-agent's Service and EndpointSlice writes by admission policy ([#&#8203;2319](defilantech/LLMKube#2319)) ([a8ea5bf](defilantech/LLMKube@a8ea5bf))
- **release:** publish a Homebrew cask for the metal-agent ([#&#8203;2234](defilantech/LLMKube#2234)) ([1b29497](defilantech/LLMKube@1b29497))
- **router:** mirror requests to the other backends of a shadow rule (fixes [#&#8203;2327](defilantech/LLMKube#2327)) ([40266ab](defilantech/LLMKube@40266ab))

##### Bug Fixes

- **agent:** pin the exact powermetrics argument vector in sudoers ([d28c55f](defilantech/LLMKube@d28c55f))
- **controller:** treat a ModelPool resident removed from spec.members as gone ([#&#8203;2310](defilantech/LLMKube#2310)) ([87464ad](defilantech/LLMKube@87464ad))
- **foreman:** bound the Metal engine port read from the agent's EndpointSlice ([#&#8203;2321](defilantech/LLMKube#2321)) ([75837e3](defilantech/LLMKube@75837e3))
- **metal-agent:** re-check a refused InferenceService on a backoff (fixes [#&#8203;2329](defilantech/LLMKube#2329)) ([6f65b30](defilantech/LLMKube@6f65b30))
- **metal-agent:** re-issue an ingress certificate the Mac's clock no longer accepts ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- **metal-agent:** read sourceSecretRef from the Model's namespace ([eaecad3](defilantech/LLMKube@eaecad3))
- **metal-agent:** retry a start that hit a transient source Secret read error ([5e13d5c](defilantech/LLMKube@5e13d5c))
- **metal-agent:** retry a transient remote size probe failure without refusing (fixes [#&#8203;2326](defilantech/LLMKube#2326)) ([5e13d5c](defilantech/LLMKube@5e13d5c))
- **metal:** bind MetalNode Reachable to the Mac with a UID-named certificate (refs [#&#8203;2265](defilantech/LLMKube#2265)) ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- **metal:** expire a MetalNode Reachable result no check has renewed (refs [#&#8203;2265](defilantech/LLMKube#2265)) ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- **metal:** read MetalNode verifier finish times on the controller's clock (refs [#&#8203;2265](defilantech/LLMKube#2265)) ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- **metal:** trust only the reported certificate in the MetalNode dial-back ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- preserve Foreman audit and transcript records per attempt ([#&#8203;2311](defilantech/LLMKube#2311)) ([3e838f9](defilantech/LLMKube@3e838f9))
- **router:** dispatch only to backends with a rule's required capabilities (fixes [#&#8203;2328](defilantech/LLMKube#2328)) ([e533192](defilantech/LLMKube@e533192))
- **router:** honor the weighted rule strategy on the Proxy data plane (fixes [#&#8203;2325](defilantech/LLMKube#2325)) ([7b34a43](defilantech/LLMKube@7b34a43))

##### Documentation

- **macos:** explain why Login Items lists the agent as Defilan Technologies LLC ([#&#8203;2307](defilantech/LLMKube#2307)) ([5b5e17e](defilantech/LLMKube@5b5e17e))
- move Foreman stream guide before references ([#&#8203;2322](defilantech/LLMKube#2322)) ([d856cb1](defilantech/LLMKube@d856cb1))
- **upgrade:** a MetalNode is Reachable only when the Mac's metal-agent reports an ingress certificate naming its MetalNode. After upgrading the controller, every Mac shows Reachable=False with reason AgentUpgradeRequired, and its hostIP is cleared, until its metal-agent is upgraded too; upgrade the agents promptly. Nothing routes to hostIP yet, so serving is unaffected. The dial-back verifier image's curl must be 7.57 or later with a TLS library that accepts a self-signed certificate as its own trust anchor; the default curlimages/curl image does. ([7e7fbaa](defilantech/LLMKube@7e7fbaa))
- **upgrade:** Metal engines now inherit only PATH, HOME, USER, LOGNAME, TMPDIR, LANG, TZ, LC\_\*, HF_HOME and HF_HUB_CACHE from the agent's environment, plus the variables the agent sets for each runtime. If an engine relies on another variable from the agent's launchd environment (for example a GGML or MLX tuning variable, or proxy settings for llama-server's own `-hf` download), list it in `--engine-env-passthrough`. Engines adopted from a previous agent process keep their old environment until they restart; run `llmkube-metal-agent stop-engines` after the bootout to apply it at once. ([6d4fba0](defilantech/LLMKube@6d4fba0))
- **upgrade:** on the Proxy data plane, a ModelRouter rule with match.requiredCapabilities now sends requests only to the backends that advertise all of them, under every strategy. A primary-fallback rule no longer sends to, or falls back to, a backend that lacks them; when every capable backend fails, the request fails instead of landing on an incapable one. ([e533192](defilantech/LLMKube@e533192))
- **upgrade:** on the Proxy data plane, ModelRouter rules with strategy: shadow now send a copy of each request to the rule's other backends, including external ones, which bill those requests. A shadow rule with one backend now fails validation, and one that mirrors to a ModelPool member fails the compile and keeps the previous config. ([40266ab](defilantech/LLMKube@40266ab))
- **upgrade:** on the Proxy data plane, ModelRouter rules with strategy: weighted now split requests by backend weight instead of sending all of them to the first backend. weight: 0 drains a backend, matching the Gateway data plane. A weighted rule whose backends cannot take traffic now fails validation, and a weighted rule across two members of one ModelPool fails the compile and keeps the previous config. ([7b34a43](defilantech/LLMKube@7b34a43))
- **upgrade:** the Metal agent now reads a Model's spec.sourceSecretRef only when the Secret is labeled inference.llmkube.dev/model-source-credentials=true; otherwise the InferenceService is refused with SourceSecretNotAllowed. Before upgrading the agent, label each credentials Secret your Metal Models use: `kubectl label secret <name> -n <namespace> inference.llmkube.dev/model-source-credentials=true`. For one release, `--allow-unlabeled-source-secrets` keeps the old behavior with a warning; it is deprecated. A Hugging Face source whose Secret cannot be read is now refused instead of downloading anonymously. ([eaecad3](defilantech/LLMKube@eaecad3))
- **upgrade:** the powermetrics sudoers rule now names the exact command line the agent runs, with the sampling interval rendered in. Upgrading the agent does not change an installed /etc/sudoers.d/llmkube-powermetrics: on hosts that installed it, re-run `make install-powermetrics-sudo` (with `APPLE_POWER_INTERVAL_MS=<ms>` if the agent uses a non-default `--apple-power-interval`), then check with `sudo -ln | grep powermetrics` that the rule ends in `-i` followed by a number. ([d28c55f](defilantech/LLMKube@d28c55f))

### [`v0.10.14`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01014-2026-10-29)

[Compare Source](defilantech/LLMKube@v0.10.13...v0.10.14)

##### Features

- **cli:** send the endpoint API key from llmkube benchmark ([#&#8203;2297](defilantech/LLMKube#2297)) ([df2e008](defilantech/LLMKube@df2e008))
- **router:** add spec.proxy.drainTimeout for the router-proxy shutdown ([#&#8203;2304](defilantech/LLMKube#2304)) ([113e009](defilantech/LLMKube@113e009))
- **router:** opt-in client address and caller header on the audit log ([#&#8203;2288](defilantech/LLMKube#2288)) ([4d1faac](defilantech/LLMKube@4d1faac))

##### Bug Fixes

- **controller:** keep a ModelPool resident the router re-activates during a reclaim ([#&#8203;2290](defilantech/LLMKube#2290)) ([857daf2](defilantech/LLMKube@857daf2))
- **controller:** remove a key dropped from podLabels or podAnnotations from the pods ([#&#8203;2289](defilantech/LLMKube#2289)) ([ca5df08](defilantech/LLMKube@ca5df08))
- **foreman:** route the remaining GitHub-only paths through the forge seams ([#&#8203;2291](defilantech/LLMKube#2291)) ([a333270](defilantech/LLMKube@a333270))
- **metrics:** each binary serves only the metrics it records ([#&#8203;2296](defilantech/LLMKube#2296)) ([de8ca21](defilantech/LLMKube@de8ca21))

##### Documentation

- **modelpool:** leave member replicas out of GitOps manifests ([#&#8203;2300](defilantech/LLMKube#2300)) ([bcbc357](defilantech/LLMKube@bcbc357))

### [`v0.10.13`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01013-2026-10-28)

[Compare Source](defilantech/LLMKube@v0.10.12...v0.10.13)

##### Features

- **agent:** add the llmkube-metal-agent stop-engines command ([2cd2ec9](defilantech/LLMKube@2cd2ec9))
- **agent:** keep Metal engines serving across metal-agent restarts and adopt them on startup (refs [#&#8203;2079](defilantech/LLMKube#2079)) ([2cd2ec9](defilantech/LLMKube@2cd2ec9))
- **release:** sign and notarize the macOS release binaries ([#&#8203;2233](defilantech/LLMKube#2233)) ([f5ff962](defilantech/LLMKube@f5ff962))
- **router:** podLabels and podAnnotations for router-proxy pods ([#&#8203;2285](defilantech/LLMKube#2285)) ([a736e18](defilantech/LLMKube@a736e18))
- **router:** publish per-backend modalities and context length on /v1/models ([#&#8203;2249](defilantech/LLMKube#2249)) ([2887f54](defilantech/LLMKube@2887f54))

##### Bug Fixes

- apply CRDs server-side in make install and make deploy ([#&#8203;2275](defilantech/LLMKube#2275)) ([edaa4f7](defilantech/LLMKube@edaa4f7))
- **chart:** leave --init-container-user off when its value is unset ([#&#8203;2286](defilantech/LLMKube#2286)) ([ac541db](defilantech/LLMKube@ac541db))
- **cli:** pin only single-writer claims and make the cache pod image configurable ([#&#8203;2178](defilantech/LLMKube#2178)) ([2fe250b](defilantech/LLMKube@2fe250b))
- **controller:** annotate router-proxy pods for scraping under --emit-scrape-annotations ([#&#8203;2268](defilantech/LLMKube#2268)) ([07fb7f4](defilantech/LLMKube@07fb7f4))
- **controller:** chown the cache root to the init container's uid when fsGroup is off ([#&#8203;2273](defilantech/LLMKube#2273)) ([c9d86ad](defilantech/LLMKube@c9d86ad))
- **controller:** create no HPA for multiNode services ([#&#8203;2272](defilantech/LLMKube#2272)) ([c8d6f76](defilantech/LLMKube@c8d6f76))
- **controller:** delete FUSE stager Jobs in the foreground without the feature-gated field ([#&#8203;2269](defilantech/LLMKube#2269)) ([e439f60](defilantech/LLMKube@e439f60))
- **controller:** make FUSE model-cache staging safe across stagers and serving pods ([#&#8203;2276](defilantech/LLMKube#2276)) ([6c7c1fe](defilantech/LLMKube@6c7c1fe))
- **controller:** point the router-proxy Service at the port the proxy listens on ([#&#8203;2280](defilantech/LLMKube#2280)) ([1c69118](defilantech/LLMKube@1c69118))
- **controller:** run the stall-probe tests on a fake clock ([#&#8203;2267](defilantech/LLMKube#2267)) ([661856a](defilantech/LLMKube@661856a))
- **router:** tell client cancellations, upstream errors and timeouts apart ([#&#8203;2284](defilantech/LLMKube#2284)) ([3a2ea74](defilantech/LLMKube@3a2ea74))
- use vLLM's real TurboQuant KV cache types in the sample and docs ([#&#8203;2270](defilantech/LLMKube#2270)) ([3b4ffb6](defilantech/LLMKube@3b4ffb6))

##### Documentation

- **upgrade:** metal-agent engines now keep running when the agent stops and the next agent adopts them. To roll back to an older agent, run `launchctl bootout gui/$(id -u)/com.llmkube.metal-agent` and then `llmkube-metal-agent stop-engines` with the current binary before installing the older one, or every model is loaded twice. `make uninstall-metal-agent` stops the engines for you. Adopted engines keep the flags they started with; run stop-engines after the bootout to apply new defaults. ([2cd2ec9](defilantech/LLMKube@2cd2ec9))

### [`v0.10.12`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01012-2026-10-27)

[Compare Source](defilantech/LLMKube@v0.10.11...v0.10.12)

##### Features

- **api:** add MetalNode, verified by a controller dial-back, with the agent reporting through its Lease ([#&#8203;2235](defilantech/LLMKube#2235)) ([fccc1bc](defilantech/LLMKube@fccc1bc))
- **cli:** add llmkube metal join and leave, with agent token renewal ([#&#8203;2246](defilantech/LLMKube#2246)) ([b1f6602](defilantech/LLMKube@b1f6602))
- **cli:** add llmkube metal token create and metal remove ([#&#8203;2236](defilantech/LLMKube#2236)) ([f54414a](defilantech/LLMKube@f54414a))

##### Bug Fixes

- **controller:** keep a ModelPool activation while its member is held at zero ([#&#8203;2241](defilantech/LLMKube#2241)) ([8fc4143](defilantech/LLMKube@8fc4143))
- **controller:** map sglangConfig.enablePrefixCaching onto --disable-radix-cache ([#&#8203;2259](defilantech/LLMKube#2259)) ([d499aa4](defilantech/LLMKube@d499aa4))
- **controller:** stage under the cache lock when the init image's stat has no -f ([#&#8203;2260](defilantech/LLMKube#2260)) ([52241de](defilantech/LLMKube@52241de))
- **router:** list each backend on /v1/models under the id BackendNameMatch routes ([#&#8203;2257](defilantech/LLMKube#2257)) ([4627cfc](defilantech/LLMKube@4627cfc))
- **router:** report moves off a quarantined pod as reason="quarantined" instead of reason="not_ready" ([faa7a5f](defilantech/LLMKube@faa7a5f))
- **router:** rewrite vLLM and SGLang backends to the name they serve ([#&#8203;2258](defilantech/LLMKube#2258)) ([3d39111](defilantech/LLMKube@3d39111))
- validate multi-file cache staging on shared volumes ([#&#8203;2185](defilantech/LLMKube#2185)) ([e6eab85](defilantech/LLMKube@e6eab85))

### [`v0.10.11`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01011-2026-10-25)

[Compare Source](defilantech/LLMKube@v0.10.10...v0.10.11)

##### Features

- **api:** accept fp8, float16 and bfloat16 in vllmConfig.kvCacheDtype ([#&#8203;2227](defilantech/LLMKube#2227)) ([31eb943](defilantech/LLMKube@31eb943))
- **api:** add spec.imagePullPolicy to InferenceService ([#&#8203;2208](defilantech/LLMKube#2208)) ([ec54d28](defilantech/LLMKube@ec54d28))
- **controller:** move the SGLang defaults to v0.5.22 and report removed attention backends ([#&#8203;2221](defilantech/LLMKube#2221)) ([33f9bb1](defilantech/LLMKube@33f9bb1))
- **controller:** move the vLLM default to v0.32.0 and check every flag against it ([#&#8203;2223](defilantech/LLMKube#2223)) ([0144a4c](defilantech/LLMKube@0144a4c))
- **controller:** run every llama.cpp default from one pinned build, b11854 ([#&#8203;2220](defilantech/LLMKube#2220)) ([052840e](defilantech/LLMKube@052840e))
- **router:** weigh LoadAware placement by estimated prefill tokens in flight ([#&#8203;2193](defilantech/LLMKube#2193)) ([a33735c](defilantech/LLMKube@a33735c))

##### Bug Fixes

- **controller:** render vllmConfig.speculative as --speculative-config ([#&#8203;2210](defilantech/LLMKube#2210)) ([191dcbd](defilantech/LLMKube@191dcbd))
- **controller:** report sglangConfig tool-call parsers SGLang does not register ([#&#8203;2228](defilantech/LLMKube#2228)) ([90fed2e](defilantech/LLMKube@90fed2e))

### [`v0.10.10`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#01010-2026-10-24)

[Compare Source](defilantech/LLMKube@v0.10.9...v0.10.10)

##### Features

- **controller:** add --model-cache-prep to skip the root cache prep ([#&#8203;2206](defilantech/LLMKube#2206)) ([8bcb973](defilantech/LLMKube@8bcb973))
- **router:** compile dataPlane: AgentGateway to AgentgatewayModel ([#&#8203;2162](defilantech/LLMKube#2162)) ([26b1ef4](defilantech/LLMKube@26b1ef4))

##### Bug Fixes

- **controller:** report a prefetch as downloading while its downloader runs ([#&#8203;2207](defilantech/LLMKube#2207)) ([7699514](defilantech/LLMKube@7699514))
- **router:** let every waiter of a failed ModelPool swap see its outcome ([#&#8203;2203](defilantech/LLMKube#2203)) ([b95db62](defilantech/LLMKube@b95db62))

### [`v0.10.9`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0109-2026-10-23)

[Compare Source](defilantech/LLMKube@v0.10.8...v0.10.9)

##### Features

- **api:** affinity and topology spread for the router proxy, rolling without surge under hard placement ([#&#8203;2200](defilantech/LLMKube#2200)) ([3a61700](defilantech/LLMKube@3a61700))
- **router:** proxy the Anthropic Messages API like chat completions ([#&#8203;2201](defilantech/LLMKube#2201)) ([86223aa](defilantech/LLMKube@86223aa))

##### Bug Fixes

- **router:** retry stale keep-alive connections and quarantine a pod only after two failures ([#&#8203;2197](defilantech/LLMKube#2197)) ([b279053](defilantech/LLMKube@b279053))

### [`v0.10.8`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0108-2026-10-23)

[Compare Source](defilantech/LLMKube@v0.10.7...v0.10.8)

##### Features

- **api:** type the vLLM tool-call and reasoning parsers and admit GLM on SGLang ([#&#8203;2168](defilantech/LLMKube#2168)) ([fd64c72](defilantech/LLMKube@fd64c72))
- **controller:** add InferenceService spec.hubOffline for serving pods ([#&#8203;2188](defilantech/LLMKube#2188)) ([a911cb2](defilantech/LLMKube@a911cb2))
- **controller:** keep 3 old ReplicaSets for LLMKube-managed Deployments by default ([#&#8203;2182](defilantech/LLMKube#2182)) ([5e1722f](defilantech/LLMKube@5e1722f))
- **router:** honour endpoint nodePort and externalTrafficPolicy, add proxy runtimeClassName ([#&#8203;2183](defilantech/LLMKube#2183)) ([a8f9e57](defilantech/LLMKube@a8f9e57))
- **router:** on pools under 3 pods, move a session only when its pod is KV-full ([#&#8203;2192](defilantech/LLMKube#2192)) ([f9b934c](defilantech/LLMKube@f9b934c))
- **router:** remember LoadAware sessions and move them only under real pressure ([#&#8203;2180](defilantech/LLMKube#2180)) ([1e8ce5a](defilantech/LLMKube@1e8ce5a))

##### Bug Fixes

- **router:** answer a ModelPool swap past swapBudget with 503 and Retry-After ([#&#8203;2181](defilantech/LLMKube#2181)) ([7c0d84e](defilantech/LLMKube@7c0d84e))
- **router:** release request bodies once sent and raise the proxy's default memory limit ([#&#8203;2196](defilantech/LLMKube#2196)) ([140fd6b](defilantech/LLMKube@140fd6b))

### [`v0.10.7`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0107-2026-10-23)

[Compare Source](defilantech/LLMKube@v0.10.6...v0.10.7)

##### Features

- **chart:** install the metal-agent RBAC from the Helm chart ([#&#8203;2165](defilantech/LLMKube#2165)) ([521e502](defilantech/LLMKube@521e502))
- **controller:** roll a multi-replica GPU service one pod at a time ([#&#8203;2164](defilantech/LLMKube#2164)) ([c4073cf](defilantech/LLMKube@c4073cf))
- **router:** load-aware, session-sticky placement for endpoint backends ([#&#8203;2166](defilantech/LLMKube#2166)) ([74cfb0e](defilantech/LLMKube@74cfb0e))
- **router:** the router proxy tracks endpoint backends' pods itself ([#&#8203;2159](defilantech/LLMKube#2159)) ([57070e1](defilantech/LLMKube@57070e1))

### [`v0.10.6`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0106-2026-10-22)

[Compare Source](defilantech/LLMKube@v0.10.5...v0.10.6)

##### Features

- **controller:** prefetch per cache claim, gated by the InferenceService ([#&#8203;2138](defilantech/LLMKube#2138)) ([d76f398](defilantech/LLMKube@d76f398))
- **router:** per-backend Endpoint Picker for dataPlane: AgentGateway ([#&#8203;2157](defilantech/LLMKube#2157)) ([c1bbe28](defilantech/LLMKube@c1bbe28))

##### Bug Fixes

- **cli:** read the model cache from its own pod in cache list ([#&#8203;2154](defilantech/LLMKube#2154)) ([996abe9](defilantech/LLMKube@996abe9))
- **controller:** refuse to repoint a same-name Service ([#&#8203;2149](defilantech/LLMKube#2149)) ([7965a1b](defilantech/LLMKube@7965a1b))
- **hf-listing:** read a tree size only when it is a whole number of bytes ([#&#8203;2153](defilantech/LLMKube#2153)) ([6e6723d](defilantech/LLMKube@6e6723d))
- **model:** report the staged total instead of status.size "0" ([#&#8203;2150](defilantech/LLMKube#2150)) ([f6710e7](defilantech/LLMKube@f6710e7))

### [`v0.10.5`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0105-2026-10-21)

[Compare Source](defilantech/LLMKube@v0.10.4...v0.10.5)

##### Features

- add make gate and make test-affected, a cached parallel round gate ([#&#8203;2106](defilantech/LLMKube#2106)) ([e971675](defilantech/LLMKube@e971675))
- **agent:** admit Metal models against memory already committed to running engines (closes [#&#8203;2080](defilantech/LLMKube#2080)) ([d2d2d43](defilantech/LLMKube@d2d2d43))
- **agent:** record each engine the metal-agent starts in the state directory (refs [#&#8203;2079](defilantech/LLMKube#2079)) ([0224d51](defilantech/LLMKube@0224d51))
- **controller:** add spec.startupTimeout for a bigger startup probe budget ([#&#8203;2129](defilantech/LLMKube#2129)) ([df0e9cd](defilantech/LLMKube@df0e9cd))
- **controller:** set fsGroupChangePolicy OnRootMismatch with the default fsGroup (closes [#&#8203;2058](defilantech/LLMKube#2058)) ([2bd4538](defilantech/LLMKube@2bd4538))

##### Bug Fixes

- **agent:** escalate to SIGKILL when stopping an engine the agent did not spawn ([0224d51](defilantech/LLMKube@0224d51))
- **agent:** keep the pressure block when an eviction teardown completed ([#&#8203;2107](defilantech/LLMKube#2107)) ([fd9ab77](defilantech/LLMKube@fd9ab77))
- **agent:** replace a hung oMLX daemon instead of starting a second one ([#&#8203;2105](defilantech/LLMKube#2105)) ([a16c348](defilantech/LLMKube@a16c348))
- **agent:** restart services evicted under memory pressure once memory recovers ([#&#8203;2093](defilantech/LLMKube#2093)) ([3db0ecc](defilantech/LLMKube@3db0ecc))
- answer 400 for request bodies that are not valid JSON ([#&#8203;2075](defilantech/LLMKube#2075)) ([d9826ff](defilantech/LLMKube@d9826ff))
- **chart:** fail fast on shared cache without default storageclass and support existingClaim ([#&#8203;2016](defilantech/LLMKube#2016)) ([#&#8203;2114](defilantech/LLMKube#2114)) ([c4f7f3d](defilantech/LLMKube@c4f7f3d))
- **chart:** gate llama.cpp InferenceAdmissionStalled on decode progress ([#&#8203;2065](defilantech/LLMKube#2065)) ([5b2f45d](defilantech/LLMKube@5b2f45d))
- **cli:** make cache clear and delete --purge-cache remove entries ([#&#8203;2134](defilantech/LLMKube#2134)) ([2b668e2](defilantech/LLMKube@2b668e2))
- **controller:** keep downloader credentials off curl's command line ([#&#8203;2098](defilantech/LLMKube#2098)) ([2207c2e](defilantech/LLMKube@2207c2e))
- **controller:** mount a memory-backed /dev/shm for multi-process vLLM, SGLang and TGI (closes [#&#8203;2099](defilantech/LLMKube#2099)) ([a68c98f](defilantech/LLMKube@a68c98f))
- **controller:** refuse to overwrite or recreate a same-name Deployment ([#&#8203;2096](defilantech/LLMKube#2096)) ([b90c8aa](defilantech/LLMKube@b90c8aa))
- **model:** canonicalize HF repo IDs so a trailing slash stages ([#&#8203;2128](defilantech/LLMKube#2128)) ([9289ac6](defilantech/LLMKube@9289ac6))
- **modelpool:** stop swap thrash on reversals, idle-probe blindness, and abandoned loads ([#&#8203;2130](defilantech/LLMKube#2130)) ([#&#8203;2135](defilantech/LLMKube#2135)) ([e7dee43](defilantech/LLMKube@e7dee43))
- **model:** serialize downloaders that share a cache directory ([#&#8203;2115](defilantech/LLMKube#2115)) ([13eca01](defilantech/LLMKube@13eca01))
- **sglang:** keep endpoint API keys out of process arguments ([#&#8203;2122](defilantech/LLMKube#2122)) ([7fdae50](defilantech/LLMKube@7fdae50))
- size metal-agent KV cache estimate from GQA attention shape ([#&#8203;2102](defilantech/LLMKube#2102)) ([a119234](defilantech/LLMKube@a119234))

##### Documentation

- cover per-request balancing in the ModelRouter page ([#&#8203;2095](defilantech/LLMKube#2095)) ([9ec4766](defilantech/LLMKube@9ec4766))
- **model-cache:** correct the CephFS fsGroupPolicy claim and the PSA advice ([2bd4538](defilantech/LLMKube@2bd4538))
- state differentiation, audience, and project status up front ([#&#8203;2100](defilantech/LLMKube#2100)) ([ee27e20](defilantech/LLMKube@ee27e20))
- **upgrade:** every InferenceService that uses the default pod security context rolls once on upgrade, because the generated pod template gains `fsGroupChangePolicy: OnRootMismatch` (a full model reload for large models) ([2bd4538](defilantech/LLMKube@2bd4538))
- **upgrade:** single-node vLLM, SGLang and TGI services with more than one GPU (or a tensor, pipeline or data-parallel size above 1) that do not mount /dev/shm themselves roll once on upgrade, because the generated pod template gains a memory-backed /dev/shm volume ([a68c98f](defilantech/LLMKube@a68c98f))
- **upgrade:** the metal-agent now admits a model only if it fits in the Mac's budget minus the memory committed to models already running, so a Mac whose models add up to more than its budget refuses the extra ones; a Model memoryBudget or memoryFraction above the Mac's budget is clamped to it unless the agent runs with --allow-model-memory-budget-override ([d2d2d43](defilantech/LLMKube@d2d2d43))
- **upgrade:** the metal-agent now writes a record per engine under --state-dir/engines (both relay and legacy mode) and fails an engine start when it cannot write it; make sure the state directory (default \~/Library/Application Support/llmkube/metal-agent) is writable by the agent's user ([0224d51](defilantech/LLMKube@0224d51))

### [`v0.10.4`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0104-2026-10-16)

[Compare Source](defilantech/LLMKube@v0.10.3...v0.10.4)

##### Features

- **agent:** cap the bytes a single Model download may write ([#&#8203;2088](defilantech/LLMKube#2088)) ([3852411](defilantech/LLMKube@3852411))
- **api:** endpoint API key and externalTrafficPolicy for InferenceService ([#&#8203;2043](defilantech/LLMKube#2043)) ([27a9214](defilantech/LLMKube@27a9214))
- **controller:** detect a stalled watch path with a lease canary, bounded reconciles and stall alerts ([#&#8203;2072](defilantech/LLMKube#2072)) ([d00e0cf](defilantech/LLMKube@d00e0cf))
- **controller:** detect keyed pool siblings and vLLM env-var keys in ModelRouter ([#&#8203;2068](defilantech/LLMKube#2068)) ([78889f0](defilantech/LLMKube@78889f0))
- **controller:** detect ModelRouter backends keyed through --api-key on the command line ([#&#8203;2063](defilantech/LLMKube#2063)) ([3674acc](defilantech/LLMKube@3674acc))
- **controller:** stage HF mirror repos with spec.files and expand globs from the repo listing ([#&#8203;2071](defilantech/LLMKube#2071)) ([9e5d7d3](defilantech/LLMKube@9e5d7d3))

##### Bug Fixes

- **agent:** health restarts stop the unhealthy engine first, wait for repeated failures, and back off ([#&#8203;2090](defilantech/LLMKube#2090)) ([407e02e](defilantech/LLMKube@407e02e))
- **agent:** keep a Metal service running when its Model lookup fails transiently ([#&#8203;2089](defilantech/LLMKube#2089)) ([e22eeab](defilantech/LLMKube@e22eeab))
- **controller:** merge DRA pod scheduling fields and reject multiNode with resourceClaims ([#&#8203;2062](defilantech/LLMKube#2062)) ([9319745](defilantech/LLMKube@9319745))
- no device-plugin GPU limit when a Model uses DRA resourceClaims ([#&#8203;2060](defilantech/LLMKube#2060)) ([8c3e434](defilantech/LLMKube@8c3e434))

##### Documentation

- APISIX in front of LLMKube ([#&#8203;2053](defilantech/LLMKube#2053)) ([16286c6](defilantech/LLMKube@16286c6))
- **metal:** fix install commands and update the quickstart for relay mode ([#&#8203;2087](defilantech/LLMKube#2087)) ([8bc7c6f](defilantech/LLMKube@8bc7c6f))
- use an extension-less link in the GPU fault guide ([#&#8203;2056](defilantech/LLMKube#2056)) ([780e710](defilantech/LLMKube@780e710))

### [`v0.10.3`](https://github.com/defilantech/LLMKube/blob/HEAD/CHANGELOG.md#0103-2026-10-13)

[Compare Source](defilantech/LLMKube@v0.10.2...v0.10.3)

##### Features

- **api:** accept SGLang's Blackwell attention backends; flag flash_attn ([#&#8203;2033](defilantech/LLMKube#2033)) ([2385fd4](defilantech/LLMKube@2385fd4))
- **chart:** alert on GPU faults the device plugin ignores ([#&#8203;2031](defilantech/LLMKube#2031)) ([e4697ef](defilantech/LLMKube@e4697ef))
- **controller:** add opt-in vLLM engine-stall liveness probe ([#&#8203;2022](defilantech/LLMKube#2022)) ([429434a](defilantech/LLMKube@429434a))
- **controller:** apply the Model's HF_ENDPOINT mirror to runtime downloads ([#&#8203;2039](defilantech/LLMKube#2039)) ([cbececa](defilantech/LLMKube@cbececa))
- **controller:** stall-aware liveness for SGLang pods; share the detector ([#&#8203;2029](defilantech/LLMKube#2029)) ([212b014](defilantech/LLMKube@212b014))
- **router:** serve POST /v1/completions on the router-proxy ([#&#8203;2019](defilantech/LLMKube#2019)) ([1724a02](defilantech/LLMKube@1724a02))

##### Bug Fixes

- **controller:** bind SGLang dual-stack so it can become Ready on IPv4 clusters ([#&#8203;2028](defilantech/LLMKube#2028)) ([7d87c7e](defilantech/LLMKube@7d87c7e))
- **controller:** bound multiNode recreates and keep the failure evidence ([#&#8203;1996](defilantech/LLMKube#1996)) ([3331610](defilantech/LLMKube@3331610))
- **controller:** refuse to overwrite or delete a same-name HPA ([#&#8203;2010](defilantech/LLMKube#2010)) ([09affe5](defilantech/LLMKube@09affe5))
- **controller:** report a missing CA ConfigMap instead of wedging pods ([#&#8203;1995](defilantech/LLMKube#1995)) ([c624e02](defilantech/LLMKube@c624e02))
- **controller:** report AutoscalingReady=True when the HPA is managed ([#&#8203;2012](defilantech/LLMKube#2012)) ([b877ee5](defilantech/LLMKube@b877ee5))
- **controller:** split reconcileMultiNodeGroup under the gocyclo limit ([#&#8203;2000](defilantech/LLMKube#2000)) ([023a5c4](defilantech/LLMKube@023a5c4))
- **controller:** stop the prefetch status loop while the CA ConfigMap is missing ([#&#8203;2037](defilantech/LLMKube#2037)) ([b0b3cc6](defilantech/LLMKube@b0b3cc6))
- **foreman-agent:** make the repo package's git env hermetic and forward GIT_CONFIG\_\* ([#&#8203;2014](defilantech/LLMKube#2014)) ([62837d9](defilantech/LLMKube@62837d9))
- **router:** compile AgentGateway mode against Gateway API Inference Extension v1 ([#&#8203;1990](defilantech/LLMKube#1990)) ([ce817ba](defilantech/LLMKube@ce817ba))
- **router:** refuse to adopt same-name router-proxy objects ([#&#8203;2008](defilantech/LLMKube#2008)) ([e8f190a](defilantech/LLMKube@e8f190a))
- **router:** remove router-proxy objects for routers not in Proxy mode ([#&#8203;2006](defilantech/LLMKube#2006)) ([c04f824](defilantech/LLMKube@c04f824))
- **router:** stop truncating streams at the header timeout; quarantine stalled streams ([#&#8203;2026](defilantech/LLMKube#2026)) ([c4335c9](defilantech/LLMKube@c4335c9))
- **router:** time only upstream silence in the stream idle guard ([#&#8203;2035](defilantech/LLMKube#2035)) ([933426c](defilantech/LLMKube@933426c))

##### Documentation

- **air-gapped:** cover vLLM and SGLang safetensors checkpoints ([#&#8203;1993](defilantech/LLMKube#1993)) ([4d3fa70](defilantech/LLMKube@4d3fa70))
- replace deprecated Strix Halo GTT sizing with TTM parameters ([#&#8203;2003](defilantech/LLMKube#2003)) ([09d8d73](defilantech/LLMKube@09d8d73))

</details>

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xNTEuMiIsInVwZGF0ZWRJblZlciI6IjQ0LjE4MS45IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://git.erwanleboucher.dev/eleboucher/homelab/pulls/2331
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area/runtime Runtime backend and inference engine support component/controller Related to the operator controller dependencies Pull requests that update a dependency file docker Docker related changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[STORY] Pin the llama.cpp CPU and CUDA defaults to the same current build instead of :server and b10068

1 participant

@Defilan