[nicolas-abril]

Why

A bang runs in rounds. Every time a stage's joins finish and the next stage forks, that is a new round. On main each round goes back to the host: cube_run reads the frontier (H_CURSOR), picks which passes to launch from its size, commits one command buffer and waits. So the GPU sits idle for a full host round trip on every dependent stage.

#1431, #1435, #1436 and #1437 ran into this with a 4-bit matvec for LLM decoding (about 97 dependent stages per token). They worked around it with format-specific kernels and a q4go/q4wait queue. This PR removes the round trip in the runtime, so it works for any bang.

Change

bend2/comp.ts only.

• The device picks the next round. The pack (pass 2), which is the last pass of a round, also takes the next frontier. Lane 0 swaps H_CURSOR into H[H_FRONT] and writes the group count for each of the five dispatches at H_GROUPS (turn_set). These are the same rules gpu_run used to apply on the host (turn_groups). A typedef check makes sure both fields fit before H_ROOT_DONE.
• Metal: each pass is an indirect dispatch from those counts, so a pass with no work launches no groups. Each round gets its own command buffer, with three in flight (GPU_DEEP). They are chained by one static MTLEvent whose value keeps going up, and each has its own autorelease pool. The host stops queueing once it reads an empty frontier from the heap it was given, then waits for the buffers still in flight. Rounds queued after the frontier empties dispatch nothing. There is no batch size and no timing rule, and nothing carries over between bangs.
• CUDA: each bang launches 1, 2, 4 … 64 rounds between syncs until the frontier is empty, starting from 1 every bang. Every kernel gets its dispatch slot at and returns at once when turn_groups(H[H_FRONT], at) says there is no work. This replaces the inverted pass→slot formula. TURN_ROWS(at) is the one group-size rule, shared by turn_groups and the CUDA launch.
• The host reads the cursor once, when the bang starts. The CPU path is unchanged.

Measurements

PAR-GPU on the cluster, from gates/perf.ts (one Mac mini per cell, the gate's own build and timing). Main is 8f86f735, the PR is 1ac32994. Each side ran the gate once. Outputs match on every bench.

bench	main	this PR	ratio
mandelbrot	0.069 s	0.060 s	0.87x
tree-bitonic	1.766 s	1.582 s	0.90x
histogram	1.497 s	1.375 s	0.92x
nbody	0.061 s	0.057 s	0.93x
merkle	0.078 s	0.073 s	0.94x
kmeans	0.682 s	0.644 s	0.94x
gameoflife	0.084 s	0.080 s	0.95x
editdist	0.511 s	0.488 s	0.95x
tree-matmul	0.910 s	0.882 s	0.97x
raytrace	0.267 s	0.260 s	0.97x
queens	2.300 s	2.260 s	0.98x
symreg	0.585 s	0.577 s	0.99x
tree-radix	1.131 s	1.118 s	0.99x
lexer	2.640 s	2.620 s	0.99x
bfs	0.904 s	0.904 s	1.00x
hashmap	1.235 s	1.239 s	1.00x
terrain	0.313 s	0.314 s	1.00x

• No bench is slower than main, and every cell is under its pin.
• SEQ-CPU and PAR-CPU are unchanged (0.99–1.00x).
• PAR-GPU memory goes up by about 0.3–0.5 MB per bench (for example 13.9M to 14.4M), which is about 3%. Every cell stays within its limit.
• Checker benches match main: compute 0.92x, defs 1.03x, generics 1.13x (main reads 1.11x on the same gate), proofs 0.98x, trees 1.04x.
• The number of command buffers in flight was picked on an Apple M4 desktop: 2 in flight was about the same as main, 3 was faster, and 4 or 8 did no better than 3.

Gates

At 1ac32994, on top of 648e4456: repo 54/54, ping 61/61, test 1655/1655 and perf 124/124 (test and perf run on the cluster). comp.ts is 64,885 ttok, under its 66,000 cap.

Not tested

• CUDA: no CUDA machine was available, and the CUDA path has not even been compiled. Changed there: the kernel takes a third argument at, the launch uses TURN_ROWS(at), and gpu_pass doubles from 1 each bang.
• Linux and the hub were not run.

Notes

• A lookup table inside turn_groups (u32 g[5], indexed at runtime) made queens 8% and lexer 4% slower on the GPU, so it stays written as conditionals. The slot→pass table {0, 0, 1, 3, 2} exists only in gpu_run.
• Metal has no device-side frontier check. A pass with no work already has an indirect count of 0.

🤖 Generated with Claude Code

[nicolas-abril]

@totaland , can you tell me if this is enough for your use case?