vulkan: models larger than device memory (27B Q4_0 on an 11 GiB card) — no host spill

[geisten]

Follow-up to #458 (known limitation).

Problem

The Vulkan path needs the whole model in device memory. The qwen3.8-27B Q4_0 (16 GB) loads and runs on a 21 GiB device (RADV iGPU, GEIST_VK_DEVICE=1; bit-identical to cpu_scalar), but on an 11 GiB RTX 2080 Ti the load fails with an out-of-memory error. There is no spill to host memory.

Proposal (needs a decision)

• Per-tensor placement: keep what fits in VRAM, put the rest into host-visible device memory (HOST_VISIBLE|HOST_COHERENT, read over PCIe by the shaders) — simplest, decode is bandwidth-bound by the spilled share.
• Layer-granular offload: first N layers in VRAM, remaining layers streamed or computed from host memory. More work, better bandwidth use.
• At minimum: a clear, early capacity error (weights + KV + scratch vs. VkPhysicalDeviceMemoryProperties budget) instead of an allocation failure mid-load.
• GEIST_VK_VRAM_BUDGET knob for tests.

Related: weights_device_copy (#458) already removed the duplicate host arena, so the device budget is the only remaining constraint.

Acceptance

• 27B Q4_0 loads and decodes on the 2080 Ti (with a reported tok/s), or fails at load with a message naming the needed vs. available bytes.

[geisten]

#605 implements the "at minimum" part of this issue: a clear capacity error and the GEIST_VK_VRAM_BUDGET knob.

Every device-local allocation is checked against its heap before it is made. A model that does not fit fails at load with the MiB needed, the MiB in use and the limit. A KV cache that does not fit fails at session creation the same way.

Open question, needs your decision: how should the 27B on the 2080 Ti actually run?

1. Per-tensor placement (recommended as the first step). Weights that no longer fit in VRAM go to HOST_VISIBLE|HOST_COHERENT memory, and the shaders read them over PCIe. The new vram_used/budget accounting already provides the decision point, so the change stays local to vk_resolve_weight. Decode would become bandwidth-bound by the spilled share.
2. Layer-granular offload (first N layers in VRAM). More work and better bandwidth use, but it needs arch support.

A side finding: the weight registry frees its VRAM copies only when the backend is destroyed. geist_model_destroy releases nothing. Loading a second model (a different host buffer) on the same backend therefore accumulates device memory. Reloading the same buffer reuses the copies. This fits with #469 (registry).

---

Generated by Claude Code