[yujiteshima]

Stacked on #2 — the base is that PR's branch, so the diff here is just this one commit. Merge #2 first; GitHub will retarget this to main automatically. See the note at the end for why the two cannot merge in either order.

The problem

52 vk* call sites, 26 of which return a VkResult, and none were checked. A failed call left an unusable handle in place and execution carried on. That showed up in three ways, in increasing order of nastiness.

1. The VM died with no Ruby-level error. With no Vulkan driver present, vkCreateInstance failed, the null instance went to vkEnumeratePhysicalDevices, and the process aborted — measured, exit 134 (SIGABRT), nothing catchable. The same shape existed in create_buffer: the memory-type search defaulted to index 0 when nothing was both host-visible and host-coherent, so map_buffer returned NULL straight into a write loop.

2. The error named the wrong cause. vkCreateComputePipelines documents that it sets the handle to VK_NULL_HANDLE on failure — which ensure_pipeline read as "the .spv is missing". A shader the driver had rejected was reported as Run make -C shader first., sending the user to re-run a build step that had already run. This is not hypothetical for #1: if V3DV rejects bitfieldReverse in fft_stage, that is exactly the message the Pi user would have got.

3. The numbers were quietly wrong. An unchecked vkQueueSubmit meant a lost device never signalled its fence, vkWaitForFences returned immediately, and the caller read the unwritten buffer back as ordinary Floats. For the streaming spectrum in #1, that is a threshold decision made on data that was never computed.

The change

VK_CHECK / gpu_check raise a RuntimeError naming the call and the VkResult:

RuntimeError: vkCreateInstance(...) failed: incompatible driver (VkResult -9)

The result-code table is written out by hand rather than pulled from the SDK's vk_enum_string_helper.h, to keep the only link dependency the loader.

• gpu_init, dispatch_compute and map_buffer now take mrb_state so they can raise. map_buffer no longer returns NULL.
• The memory-type search raises instead of falling back to index 0.
• Pipeline state is recorded per pipeline (PIPE_MISSING_SPV vs PIPE_CREATE_FAILED), so the two cases give opposite advice.
• A failed gpu_init sets init_failed instead of being retried on every later operation, which would leak a fresh context each time.

Unwinding

mrb_raise longjmps, so anything already allocated has to be released before the check. create_buffer releases each earlier step before handing the failure to gpu_check, and dispatch_compute's record/submit/wait sequence funnels into one exit that frees the fence, command buffer and descriptor set and then raises.

The subtler consequence: buffers are now wrapped before the dispatch that fills them, because the GC can only free what it owns. That also closes the pre-existing leak when GPU::SFloat.cast is handed a non-numeric element and mrb_as_float raises mid-loop. narray_sum's partial buffer is wrapped for the same reason, so destroy_buffer is now only the finalizer plus create_buffer's own unwind.

Two silent-truncation guards in the same family

• GPU::SFloat.new rejects a size past UINT32_MAX. Previously the cast to uint32_t wrapped: GPU::SFloat.new(2**32 + 10) returned a 10-element array.
• A dispatch past maxComputeWorkGroupCount is rejected rather than left to the driver. Worth being precise here: on lavapipe the oversized dispatch runs and returns the correct answer — I checked — which is exactly why the limit is easy to miss in testing while a driver bounded by a hardware register may behave differently. GPU.info gains :max_workgroups so callers and bug reports can see the bound.

Verified

On Mesa lavapipe, building mruby 3.x with the gem (no warnings):

• 46 tests pass in test/narray_test.rb (2 new).
• test/shader_error_test.rb (new) passes in both modes. It is a separate file because it has to break the process-wide GPU context — running it alongside the main suite would leave every later test without pipelines. CONTRIBUTING.md documents how to run it.
• Checked by hand that the no-device path now raises and leaves the VM alive, where the same script previously aborted the process.

Not run on Pi 5 / V3D or a discrete GPU, same as #2.

Why this is stacked rather than parallel

dispatch_compute gained an mrb_state parameter, and #2's FFT code calls it from 12 sites. Whichever of the two lands second would fail to compile against the other, so they cannot merge in either order. I confirmed this commit is otherwise self-contained by cherry-picking it onto main alone: two trivial context conflicts, and it builds clean and passes 30/30 tests — but that stripped version knows nothing about rfft, so it is not a substitute for merging in order.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JAgXRCCQ43jwXzeXKS6hG4

---

Generated by Claude Code

[yujiteshima]

Verified on real hardware

Ran this branch (so #2 + #3) on a physical GPU.

Environment: Apple M5 GPU via MoltenVK, Vulkan 1.1.334, macOS 15 (Darwin 25.5.0), mruby 4.0.0 (d350b8d), clang.

Build	no warnings
test/narray_test.rb	45 passed, 0 failed, 1 skipped
test/shader_error_test.rb	1 passed, 0 failed
examples/fft_spectrum.rb	unchanged from #2 — bins 50 and 200 at 262144 and 65536

The skip is array past one dispatch, reporting device allows 274877906944 elements per dispatch. So this device's maxComputeWorkGroupCount is far too large to reach and the guard correctly steps aside — which is the count difference against the 46 you saw on lavapipe, not a regression.

The crash is really fixed

Measured the headline case both ways, same script, same machine, driver hidden with VK_DRIVER_FILES=/nonexistent.json:

begin
  p GPU::SFloat.new(8).to_a
rescue => e
  puts "rescued: #{e.class}: #{e.message}"
end
puts "VM still alive"

On main:

[Vulkan Loader] ERROR: vkEnumeratePhysicalDevices: Invalid instance
exit code 134   (SIGABRT — nothing printed, nothing catchable)

On this branch:

rescued: RuntimeError: vkCreateInstance(&inst_info, NULL, &g_ctx.instance) failed: incompatible driver (VkResult -9)
VM still alive
exit code 0

Independently reproduced before reading this PR, incidentally — I hit the same exit 134 while reading gpu_vulkan.c, then found this branch already had the fix. The error message naming both the call site and the decoded VkResult makes the cause obvious at a glance.

Merge order

Confirmed the stacking is real: this branch contains #2's FFT work, so #2 first, then this.

Still open

Raspberry Pi 5 / V3D. The board is powered down; I will run both branches there before merging. That is also where the bitfieldReverse question from #2 gets its real answer — MoltenVK handles it correctly (see my comment on #2), but V3DV is the driver that matters for the target.

🤖 Generated with Claude Code