vulkan: count GPU fallbacks per site, add GEIST_VK_STRICT (#474) - #604
Merged
Merged
Conversation
Work that leaves the GPU used to show up only as a slowdown. Each site is now counted in vk_state.fallbacks (declined fused ops, host views and copies of mapped memory, host-path linears) and printed under GEIST_VK_VERBOSE; host-path weights are summed at resolve and reported once when the first of them runs. GEIST_VK_STRICT=1 turns every fallback into GEIST_E_BACKEND with an error naming the site and refuses host-path weights at resolve. A failed argmax dispatch now returns the dispatch status instead of GEIST_E_UNSUPPORTED, which sent the arch to a host scan of logits the GPU never wrote. New test_backend_vulkan_fallback_unit covers every declining fused op, host access, the argmax failure, host-path weights, and decodes the in-memory llama and Qwen3.5 fixtures with zero fallbacks, strict mode on and off. Added to both Vulkan CI lists. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nb4xVBy5t3fTdSStHPPKPg
…474) Items 1 and 6 of #474. A failed submit set seq_failed, but buffer_map still handed out the mapping, so the arch read whatever the batch had not written. buffer_map now takes the failure and returns nullptr with GEIST_E_BACKEND as the backend error; the host-path linear and a host buffer copy check it too. Every transformer caller that maps a buffer turns nullptr into an error instead of dereferencing it (kv_rotate_rows and the KIVI drain now return a status for that). vk_seq_flush stays void: the sticky flag already carries the result to every readback, and a status there would touch all its callers for no extra coverage. The push constants, dispatch sizes and offsets in ops.c narrowed with (uint32_t) casts, so a size past 4G elements wrapped silently. They now go through vk_ckd_u32 (checked.h convention: true when it does not fit), and the op returns GEIST_E_INVALID_ARG via vk_too_wide. vk_tensor_gpu and the x ring do the same for element offsets. resolve_weight checks max_m x n_in against the x ring, so a weight whose batch could not be staged fails the load instead of every linear_t call declining to a host linear, and creates the ring and the argmax word there rather than on the hot path. caps.max_m is VK_MAX_M. Two memcpy(dst, nullptr, 0) on the first registry grow (UBSan) are guarded. New test_backend_vulkan_narrowing_unit: buffer_map and download after a forced QueueSubmit failure, vk_ckd_u32 bounds, attention refusing a q_offset and sliding_window past UINT32_MAX, and the ring check at resolve. Added to both Vulkan CI lists. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nb4xVBy5t3fTdSStHPPKPg
6 of 7 tasks
vulkan: report a failed submit at buffer_map; checked uint32 narrowing; size the x ring at resolve (#474 items 1, 6)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nb4xVBy5t3fTdSStHPPKPg
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Covers items 4 and 9 of #474. Item 4 surfaces the silent fallbacks. Item 9 reports host-path weights.
Before, when a fused op left the GPU, the only sign was a slowdown. This covers declined shaders, host loops over mapped memory, host buffer copies, and weights with no GPU kernel. A failed argmax dispatch also sent the arch to a host scan of logits that the GPU never wrote.
After this PR, every such site is counted per site.
GEIST_VK_VERBOSEprints the counters at destroy. The first host-path linear prints how many weights and MiB take that path. WithGEIST_VK_STRICT=1, each fallback becomesGEIST_E_BACKEND, with an error that names the site, and host-path weights are refused at load. A failed argmax dispatch now returns the dispatch error.Changes
vk_fallback(st, site)andenum vk_fb(sequence.c, vk_internal.h).GEIST_E_UNSUPPORTED, orGEIST_E_BACKENDunder strict mode.linear_t,linear_t_pair,embedding_lookup_scaled,argmax_f32,kv_append_f16andattn_qgate_splitgo through thin wrappers. A decline is counted once per call;linear_t_pairis not double-counted.vk_tensor_hostand the host branch ofvk_buffer_copycount the fallback and refuse it under strict mode.vk_resolve_host: host-path weights (Q3_K, large F16/BF16, rows that are not whole blocks) are summed at resolve and refused under strict mode.vk_w_cpu_mNcounts each call and prints the one-line summary on the first one.vk_argmax_f32returns the dispatch status on failure.Testing
New
tests/test_backend_vulkan_fallback_unit.c, added to both Vulkan CI lists. It checks:GEIST_E_BACKEND, not counted as a fallback);It also decodes the in-memory llama and Qwen3.5-hybrid fixtures with zero fallbacks at every site, with strict mode off and on.
make test-unit, release: 92 passed, 32 skipped.test_backend_vulkan_ops_unitfails (act_quant and Hadamard checks). It fails the same way on unmodified main on lavapipe and is not in the CI list.make format-checkandmake check-headersare clean.API impact
include/geist.h. The change is internal to the Vulkan backend;GEIST_VK_STRICTis a new environment knob.🤖 Generated with Claude Code
https://claude.ai/code/session_01Nb4xVBy5t3fTdSStHPPKPg
Generated by Claude Code