[BlueKingMuch]

On Windows, hipMemGetInfo reports the card's size minus this process's own allocations: ROCclr's PAL backend asks Windows for this process's usage only, so what the desktop and other programs hold on the card is not subtracted. On a card that also drives the desktop the engine counts the ~930 MiB those hold as free, --expert-cache auto fills the card past what Windows lets the process keep in VRAM, and WDDM moves part of the engine out to system RAM. On an RX 6800 with setup's configuration this PR takes decode from 30.5 to 41.4 tok/s (medians of 24 requests each).

What changes

• Windows gives every process a video memory budget that does account for the others (DXGI QueryVideoMemoryInfo). On Windows HIP builds, cudaMemGetInfo, which the HIP shim maps to hipMemGetInfo, now lowers hipMemGetInfo's free figure by what the budget withholds: the card's size minus the budget (799 MiB on this card while the process is small, about 1.8 GiB once it holds 11 GiB).
• Everything sized from that figure stays within the budget. For --expert-cache auto the existing check after the slots are written does the rest ("only 8 MiB free once the slots are written ... shrinking"), since the budget itself drops once the process holds most of the card (probe below).
• Lowering HIP's own figure, rather than taking the budget minus DXGI's usage, keeps memory the HIP runtime has freed but still caches counted as free, the way HIP counts it (in the probe below DXGI still counts 1946 MiB as used after everything was freed).
• dxgi.dll is loaded at run time and the card's adapter is found by its LUID, so nothing new is linked. Without dxgi.dll, the adapter, or a budget below the card's size, hipMemGetInfo's figure stands.
• The first query logs it once: strata: Windows budgets 15569 of this card's 16368 MiB for this process; free VRAM is counted within that (STRATA_WDDM_BUDGET=0: off). STRATA_WDDM_BUDGET=0 turns it off.
• CUDA and Linux builds are unchanged: the code is under #if defined(STRATA_USE_HIP) && defined(_WIN32), and the shim header is only used by HIP builds.

It is on by default, since without it setup's own configuration overfills a card that drives a display; if you prefer it opt-in, that is the off lambda in mem_get_info.

No kernel changes. What changes is the free-VRAM figure and with it how many experts sit in VRAM, as with any other --expert-cache size.

Measured

RX 6800 16 GB (gfx1030, drives the desktop), Ryzen 7 5800X3D, PCIe 4.0 x16, Windows 11 (build 26300), HIP SDK 7.2, Visual Studio 2026 Build Tools. A local test build of 0.1.31 (9259cad) with #356, #311 (rebased onto 0.1.31), this commit, #373, #377, an earlier version of #376 (the same Release flags), and a scratch reserve for verify windows (branch win-scratch on my fork; not proposed, since 0.1.31 did not hang without it). Every comparison in the table is within that build: STRATA_WDDM_BUDGET=0 against unset.

Qwen3.8-Flash-Next IQ3_XXS with the configuration setup wrote: --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --max-context 131072 --kv int8 --kv-resident 32768, MTP draft head, pcie_frac 0.55 from the probe. Two engine starts per variant, each with four passes over tools/calibrate.py's three prompts, 128 tokens each; the card's memory from Windows' GPU counters after loading.

	budget off (STRATA_WDDM_BUDGET=0)	budget on (default)
free VRAM the cache sizing starts from	10.69 GiB	9.91 GiB
expert cache after the shrink check	9.83 GiB (6072 slots)	8.05 GiB (4960 slots)
the card in use, all programs	16132 of 16368 MiB	15230 MiB
the engine's dedicated VRAM	15198 MiB	14296 MiB
its shared GPU memory, above the budget-on start's	+929 MiB	(reference)
decode, median of 24 requests	30.5 tok/s	41.4 tok/s (+36 %)
decode per prompt (median of 8 each)	30.7 / 29.4 / 30.0	47.1 / 40.1 / 41.4
decode, slowest / fastest request	26.8 / 31.9	37.8 / 47.7

With the budget off the expert cache is 1.78 GiB larger; 902 MiB more of the engine is in VRAM (dedicated) and 929 MiB more in system memory (shared). The two starts of each variant gave the same sizes and memory figures, and their decode medians are within 0.4 tok/s of each other. The prompt path borrows the same 4.44 GiB of cache slots in both and the prompt chunk stays 8192 tokens; long prompts were not measured.

--vram-reserve-mib alone does not help: on a 0.1.30 build, --vram-reserve-mib 1500 still left the card at 16037 MiB, since the reserve is taken from the same figure.

The budget on this card, from a standalone probe that allocates in steps of 1 GiB (256 MiB past 14 GiB), in MiB:

held by the probe	0	10240	11264	14336	16128	all freed
hipMemGetInfo free	16227	5974	4950	1878	86	16214
DXGI budget	15569	15569	14515	14543	14538	15569
DXGI usage	141	10394	11418	14490	16282	1946

The allocations past the budget all succeed; whether and how much of that Windows then moves out depends on what else wants the card (in the engine runs above, 929 MiB).

To reproduce: STRATA_WDDM_BUDGET=0 against unset, comparing the engine's dedicated and shared GPU memory between the two (Task Manager's details view or the GPU Process Memory counters). Shared memory is large in both (the registered expert arena counts there); the difference is what moved.

Tested on this card only, with the desktop on it. On a card without a display the budget should sit closer to the card's size, so less would change there; not tested, nor with more than one GPU.

🤖 Generated with Claude Code

https://claude.ai/code/session_015Ld5YDzuKW1hgMVwwgRZDP

[Niko1221]

Released in 0.1.37 as is (rebased; the device listing that main gained since kept beside it). Merged under your name (GitHub doesn't show it as merged because it went in by hand). Thank you!