--resident-budget-gib: a budget clamped to available RAM then fails the safety check (0.1.31)

Version: 0.1.31 (tag v0.1.31), Windows 11 26200, RTX 5090 32 GB, Ryzen 9 5950X, 64 GB DDR4 at the time.
Model: Unsloth UD-Q4_K_XL, imported by hand following docs/UNSLOTH_Q4.md, with "--resident-budget-gib", "40" in the config.

What happens

The engine clamps the budget, fills it, and then refuses to start with the same number:

FileExpertSource: --resident-budget-gib 40.00 is more than the RAM has room for (31.37 GiB available minus 4 GiB headroom): 27.37 GiB
FileExpertSource: RAM budget 27.37 GiB: 9384 of the 48.69 GiB of experts the GPU cache does not hold, by profile rank; the rest are read from the files
strata generate: CPU expert residency: FileExpertSource: resident complement 27.37 GiB exceeds available RAM (31.37 GiB) minus the 4 GiB safety headroom

The docs say "On 64 GB, 40. The engine clamps a budget larger than the RAM it finds free", but with that setting the
engine did not start here. An explicit smaller budget (26) works around it.

Likely cause

I read src/core/expert_source.cpp but did not step through it in a debugger.

The budget branch (around line 1029) calls available_memory_bytes(physical) once and clamps budget_bytes to
room = physical - headroom. It then packs the ranked experts while at <= budget_bytes, so bytes can end up equal
to room or a few bytes under it. Right after that (around line 1069) available_memory_bytes is called again,
and bytes > physical - headroom is checked against the new reading. If available RAM drops by a few MB between the
two calls (the engine's own allocations, the OS file cache, another process), a budget clamped to the limit fails.
Both log lines show 31.37 because the drop is smaller than the two-decimal rounding.

Some ways to fix it, whichever you prefer: reuse the first reading for the second check when a budget was given; clamp
the budget to room minus a small margin such as 256 MiB; or use the clamped value as the hard check's limit.

Repro

We saw it once. It should happen whenever the budget gets clamped and available RAM shrinks even slightly before the
second check, so it may be intermittent.

Start of strata-ud-q4_k_xl.log up to the failure

strata generate: PCIe probe: 28.2 GB/s host->device -> pcie_frac 0.55 (default 0.55)
strata generate: this CPU has no AVX-512: the expert kernels run on AVX-2 (multi-token for the i-quant gate/up rows)
strata generate: native pack: E:\strata-data\packs\ud-q4_k_xl experts (largest blob 3.99 MB), token embedding type 8 in mapped host memory (644 MiB)
strata generate: 1416 MiB of weights loaded from E:\strata-data\packs\ud-q4_k_xl (303 canonical tensors skipped: served natively)
strata generate: 301 native projection matrices, 2975.00 MiB of weights
strata generate: profile E:\strata-src\data\expert-profile.bin: 24576 ranked pairs, built for 24576 slots
strata generate: PLE on, table 320001536 rows of E:\strata-data\models-unsloth\UD-Q4_K_XL\Qwen3.8-Flash-Next-UD-Q4_K_XL-00002-of-00004.gguf
strata mtp: draft layer loaded, 814 MiB of VRAM (experts 675, dense 111), files read in 0.47 s (1655 MiB/s)
strata generate: experts via mmap (--mmap-experts; the GGUF shards in place, no experts.bin)
strata generate: experimental native Q5_K head, 675430400 bytes
strata generate: expert cache auto: 24.00 GiB free, 700 MiB reserved (+281 MiB for the draft head) -> 6195 slots
strata generate: expert cache 7893 slots, 23.04 GiB of VRAM; policy is
strata generate: the GPU computes the experts in the cache; it rounds differently from the CPU,
                 so a reply can differ slightly from a run without the cache (same quality:
                 bench/results/2026-09-27-cache-parity).
                 PROFILE, ranked by routing frequency, no eviction.
strata generate: pre-filled 7893 of 7893 slots from the profile; slot 0 verified
strata generate: routing-aware prefetch of the file tier on (the next layer's router)
strata generate: R4 hit path ON - resident experts are computed on the GPU
strata generate: 15 expert-pool workers + the host thread
strata generate: session is up (engine 0.1.31)
strata generate: token graph hit path: 7893 resident experts, decided on the device
FileExpertSource: --resident-budget-gib 40.00 is more than the RAM has room for (31.37 GiB available minus 4 GiB headroom): 27.37 GiB
FileExpertSource: RAM budget 27.37 GiB: 9384 of the 48.69 GiB of experts the GPU cache does not hold, by profile rank; the rest are read from the files
strata generate: CPU expert residency: FileExpertSource: resident complement 27.37 GiB exceeds available RAM (31.37 GiB) minus the 4 GiB safety headroom

Still there in 0.1.32: src/core/expert_source.cpp did not change between v0.1.31 and v0.1.32.

The new Unsloth setup flow makes it easier to run into. With --family unsloth, resident_budget_gib() in setup.py
rounds the total physical RAM and subtracts 24, so a 64 GB PC gets --resident-budget-gib 40. The engine checks the
budget against available RAM, which on this machine (64 GB at the time) was 31.37 GiB. A 40 GiB budget gets clamped
to that limit, and the clamped budget is what failed the second check in the log above. A 64 GB user who keeps the
setup default may see the same error at start.

Thanks, confirmed. The engine lowers the budget to the free RAM it reads first, then reads free RAM a second time for the safety check, and a few MB of difference between the two readings makes it fail. Being fixed for the next release: the check will use the same reading with a small margin, and if RAM is still short it will warn and read the rest of the experts through the file cache instead of stopping. Setup will also get a --resident-budget-gib option. Keeping this open until the release.

Same failure on a different model and quantization, on 64 GB of RAM. It looks less like a clamping bug and more like the available-RAM estimate depending on what the previous run left behind.

Machine: Ryzen 7 5700X3D, 64 GB DDR4-2666, RTX 5060 Ti 16 GB, Windows 11, engine 0.1.31 and 0.1.32. Model: orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF (IQ4_XS), 60.94 GiB of routed experts, pack built from the three shards (native_experts.txt v4, 303 matrices served natively).

--resident-budget-gib 40 starts and fills:

RAM budget 40.00 GiB: 16131 of the 53.40 GiB of experts the GPU cache does not hold, by profile rank; page-locked and mapped

--resident-budget-gib 50 refuses, and it refuses with the complement, not the budget:

FileExpertSource: resident complement 46.70 GiB exceeds available RAM (50.70 GiB) minus the 4 GiB safety headroom

The same session had just completed a run at 40, so "50.70 GiB available" is not a property of the machine — it moves with what is still page-locked from the previous run. On this machine 40 is the usable ceiling and 50 is past it, and the number that gets printed as available is smaller than the 64 GB the machine has.

One more data point for whoever picks this up: on this machine the budget mode is not the fast path either. experts.bin in mmap 38.1 tok/s against 25.4 with --resident-budget-gib 40 on the same file, and the two costs are in the log — adaptive tier 2688 experts swapped into the VRAM tier (every 4 rounds, 7.580 ms/round), and the CPU pool reading the page-locked complement at 15.5 GB/s against 27.4 GB/s from the OS page cache. Full table in #416.

tests: golden --yes / Enter configs on five PCs (#403 #406 #364 #384)

engine: a RAM budget clamped to the free RAM no longer fails its own …

setup: recommend, never force - longer contexts, more GPUs, a bigger …

docs: the RAM rule recommends 128K, the low-RAM mode on several GPUs,…

Engine 0.1.33: a portable image encoder again (#411 #412), setup reco…

Fixed in 0.1.33: a clamped budget leaves 256 MiB of margin and the safety check uses the same RAM reading; a budget that can't be kept at all is a warning (the experts are read from the files). Setup takes --resident-budget-gib N. Thanks!