[wakamex]

Fixes #1393. Fixes #1394. Follow-up to #1439, which #1809 reverted.

Today on main, a lane on the GPU has a 2,048-word stack (STAK_LEN), and once it fills, the next call posts ERR_DEEP and the run stops. A non-tail List.append under ! fails at 1,023 frames (#1393), an unbalanced fork tree such as two!(365n) fails (#1394), and Unbalanced Tree Search (UTS) (uts-compact) exits with ERR_DEEP on the GPU. The host has no such limit.

This PR lets a full lane keep going. When a call finds too little room left, lane_spill copies the lane's whole stack to a heap block and leaves a 3-word exit frame at the stack's base. When the lane returns down to that frame, FID_EXIT copies the stack back and returns into the continuation on top of it.

What each program pays

#1809 reverted #1439 because lane_spill joined the device code of every ! program and made the Metal GPU build 17% slower on symreg and merkle. #1439 inlined the check and a 4-argument lane_spill call at every call site, and added a second unspill loop to FID_EXIT. Here a call site costs one compare and a jump, and the spill itself runs in one cold place:

• The check sits in emit_jump, after the callee's arguments are in r0..rn, so the caller's locals are dead and its whole frame is already on the stack.
• On the device, WL_ROOM(F) is if (sp >= <stack end - room>) WL_JMP((F) | 1u << 16). FIDs stay under 65,536, so bit 16 never names a segment. work_loop's default: case clears the bit, calls lane_spill and dispatches F. This avoids goto, which the Metal Shading Language does not support (section 1.5.4 of the 4.1 specification): this PR's first version jumped to an sp_spill: label, and Metal refused to build it ("labeled statements are not supported in Metal").
• FID_EXIT unspills through its existing WL_ARGS + heap_free + WL_TAKE block, testing for a spilt frame with WL_SPILT(wf).

Every ! program with a non-tail call still carries lane_spill and that default: branch in its device code, symreg and merkle included. On Metal both now build faster than on main, and no program runs slower (tables below).

Why a lane never writes past its stack

A spilled stack comes back whole, so a lane can resume near the top of its 2,048 words, and the continuation then pushes the next call's frame before any check runs. One frame of room is therefore not enough. emit_fork records room, the most words one fork chain in the program pushes (its held words plus one fid per call), and WL_ROOM spills when less than room remains. Every def is entered with at least room words free, so its whole chain fits even when it resumes from a spill, and a spilled stack never holds more than STAK_LEN words. room is 8 in merkle and 449 in the new test.

Effects on shared code

• Host: WL_ROOM expands to nothing and WL_SPILT to 0. objdump -d shows the same host machine code as main for symreg, merkle, queens, raytrace and tree-matmul, apart from addresses.
• Device: every task return through FID_EXIT now tests wf == FID_EXIT, in every ! program.
• Device: nothing posts ERR_DEEP any more. Runaway recursion under ! now spills 16 KB blocks until the heap runs out, then fails with ERR_HEAP. The comment in tests/io/stack_fault_trap.bend that described the device posting ERR_DEEP is updated; that test exercises the host stack and its output is unchanged.

comp.ts goes from 66,000 to 66,377 ttok (+377). Main is at the 66,000 cap.

Tests

tests/compile/bang_lane_spill.bend covers four cases:

• a 5,000-deep List.append (Metal: non-tail recursion under a bang faults at 1023 list elements; CPU completes through 100000 #1393);
• two!(365n) (Metal: one ! over a two-leaf fork tree (a 365-fork spine beside a 3-fork spine) faults; each leaf alone, a balanced 365+365, and --gpu off all pass #1394);
• a 600-deep fork whose first spill comes at its second call, with the first call's words still on the stack;
• tall, a fork that holds six 40-word records across its first call and hands one to each later call, so its frames add up to 449 words. Built with one frame of room (WIDE), the second tall! faults with bend: device fault on CUDA.

All five lines match on Metal, on CUDA, natively on the CPU, interpreted, and as JS under Bun. All 40 runnable tests that use ! pass on an M4 mini (Metal) and on an RTX 3090 (CUDA); main fails only bang_lane_spill.

Metal

Measured on a rented Mac mini M4 (10-core GPU, macOS 26.7.1), main (de26a91) against this PR, alternating main and PR in every round.

Build time, median of 7 runs per side, the same commands as #1809 (lower is better):

Program	Step	Main	This PR	Change
symreg	bend main.bend -o main	0.776 s	0.761 s	1.9% shorter
symreg	./main +RTS --gpu-build	0.267 s	0.252 s	5.7% shorter
merkle	bend main.bend -o main	0.706 s	0.699 s	1.0% shorter
merkle	./main +RTS --gpu-build	0.163 s	0.156 s	4.2% shorter

In every row but merkle's bend -o the slowest PR run is faster than the fastest main run; there the two ranges overlap by 1 ms.

Runtime of the 17 programs under bench/runtime, in 4 blocks of 4 runs (A B B A, then B A A B). A block's ratio is main's time over this PR's, so above 1x means the PR is faster; the table gives the median block ratio and its 95% bootstrap interval. All 17 print the same output as main.

Program	Main (s)	Result	Median ratio (95% interval)
bfs	0.86	no detected change	1.001x (0.946x to 1.042x)
editdist	0.52	no detected change	1.001x (0.955x to 1.012x)
gameoflife	0.13	no detected change	1.013x (0.959x to 1.055x)
hashmap	1.34	no detected change	0.991x (0.955x to 1.009x)
histogram	1.43	1.1% faster	1.011x (1.007x to 1.017x)
kmeans	0.72	no detected change	1.006x (0.988x to 1.012x)
lexer	2.82	5.3% faster	1.053x (1.014x to 1.100x)
mandelbrot	0.11	no detected change	0.983x (0.938x to 1.058x)
merkle	0.12	no detected change	1.026x (0.989x to 1.037x)
nbody	0.10	no detected change	1.006x (0.964x to 1.020x)
queens	2.53	no detected change	1.007x (0.989x to 1.039x)
raytrace	0.33	no detected change	0.989x (0.962x to 1.028x)
symreg	0.69	11.1% faster	1.111x (1.014x to 1.144x)
terrain	0.32	no detected change	1.055x (0.882x to 1.119x)
tree-bitonic	1.97	3.1% faster	1.031x (1.003x to 1.098x)
tree-matmul	0.83	no detected change	1.013x (0.861x to 1.175x)
tree-radix	1.20	1.5% faster	1.015x (1.005x to 1.028x)

No program gets slower. I have not found why symreg (11.1%) and lexer (5.3%) get faster.

CUDA build time and registers

On an RTX 3090 (CUDA 13.1, sm_86), ./main +RTS --gpu-build with CUDA_CACHE_DISABLE=1 for the 17 programs under bench/runtime, alternating main and PR in 4 blocks, totals 23.7 s on main and 22.9 s with this PR. Single runs vary by up to 15% (main against itself), and only tree-matmul builds shorter in every block (3,183 to 2,699 ms, medians). Registers per thread of bend_dev (cuobjdump --dump-resource-usage) go up for symreg (64 to 72), tree-bitonic (72 to 80), editdist (56 to 64), gameoflife (70 to 77), terrain (60 to 66) and nbody (113 to 118), down for merkle, mandelbrot and raytrace (by 2) and tree-matmul (92 to 86), and stay the same for the other 7.

CUDA runtime

bend-bench paired (0f6d45f) checks each output, warms up, then runs 8 blocks of 4 runs (A B B A, then B A A B) on the same RTX 3090 with a 4 GB heap, against main (de26a91). A block's ratio is main's time over this PR's, so above 1x means the PR is faster; the table gives the median block ratio and its 95% bootstrap interval.

Program	Blocks	Result	Median ratio (95% interval)
bfs	8	no detected change	0.999x (0.987x to 1.009x)
editdist	8	10.0% faster	1.100x (1.088x to 1.114x)
gameoflife	8	4.6% faster	1.046x (1.041x to 1.053x)
hashmap	8	no detected change	1.018x (0.961x to 1.055x)
kmeans	8	3.2% slower	0.968x (0.959x to 0.991x)
lexer	8	no detected change	1.008x (0.987x to 1.018x)
mandelbrot	8	no detected change	0.994x (0.979x to 1.023x)
merkle	8	2.1% slower	0.979x (0.966x to 0.999x)
nbody	8	no detected change	0.989x (0.983x to 1.003x)
queens	8	0.8% slower	0.992x (0.966x to 0.998x)
raytrace	8	no detected change	0.993x (0.972x to 1.004x)
symreg	8	no detected change	0.993x (0.963x to 1.011x)
terrain	8	no detected change	1.002x (0.981x to 1.030x)
tree-bitonic	8	no detected change	0.995x (0.986x to 1.015x)
tree-matmul	8	15.9% faster	1.159x (1.021x to 1.214x)
tree-radix	8	no detected change	0.975x (0.934x to 1.012x)

kmeans gets 3.2% slower, merkle 2.1% and queens 0.8%. editdist (10.0%), gameoflife (4.6%) and tree-matmul (15.9%) get faster, and I have not found why.

uts-compact passes with this PR (293367 0); main exits with ERR_DEEP, so it has no main time to compare against.

Original description

Fixes #1393. Fixes #1394. Follow-up to #1439 and #1809.

Today on main (de26a91, after #1809 reverted #1439), emit_fork emits WL_ROOM(fr.length) before pushing each sequential continuation frame, and WL_ROOM(N) posts ERR_DEEP and returns 0 on DEVICE once a lane's 2,048-word (STAK_LEN) stack fills. A non-tail List.append under ! faults at 1,023 frames (#1393), an unbalanced fork tree such as two!(365n) faults (#1394), and Unbalanced Tree Search (UTS) (uts-compact) exits with ERR_DEEP on the GPU.

In #1439 (40402b4), WL_ROOM(N, D) called a 4-argument lane_spill(e, sp, N, D) at every sequential call site while the caller segment's local variables were still live, followed by a second branch if (!(sp = ...)) { err_post(e.mem, ERR_DEEP); return 0; }, and WL_CASE(FID_EXIT) added a second WL_ARGS + heap_free + WL_TAKE block alongside continuation dispatch. Even with FAR on lane_spill, expanding a 4-argument call with live caller locals and two branches at every WL_ROOM site (27 device sites in symreg, 8 in merkle, 47 in tree-matmul) plus a second unspill loop in WL_CASE(FID_EXIT) grew the device control-flow graph, slowed bend -o's GPU build step (./main +RTS --gpu-build) by 17% on symreg and merkle on Metal (#1809), and slowed CUDA execution on queens (6.7% slower), symreg (7.4% slower), tree-matmul (20.6% slower), and tree-bitonic (50.3% slower).

This PR moves the stack-room check into emit_jump, after the sequential frame is pushed and after the callee arguments are loaded into r0..rn, and routes every WL_ROOM(F) overflow on DEVICE to a single cold sp_spill: label at the bottom of switch (fid) in work_loop:

• emit_fork removes WL_ROOM(fr.length) before STK(j) = w and passes push = true to emit_jump(sc, cargs, c.k!, !fork && c.b, true). No other emit_jump caller passes push = true, so tail jumps and self-tail loops without a frame push emit no WL_ROOM.
• emit_jump with push = true emits WL_ROOM(seg_ref(sc, fid)) right after r0..rn are assigned, before WL_JMP or WL_AGAIN. Self-recursive calls in emit_fork still set sc.seg.spin = true and take WL_AGAIN(sc.seg.fid) on the normal path when the stack is not full.
• Because WL_ROOM runs after STK(j) = w; WL_PUSHN(fr.length) and after r0..rn are loaded, all caller segment local variables are dead and the caller's entire continuation frame is already on the stack below sp. In Metal: a full lane stack spills to the heap, so non-tail recursion under ! no longer faults at 1023 frames (#1393, #1394) #1439, WL_ROOM ran before STK(j) = w; WL_PUSHN(fr.length), so spilling at a second fork call (such as walk's second call in tests/compile/bang_lane_spill.bend, where two words from the first call are already on the stack) would insert the exit frame between the earlier call's stack words and the new frame's words unless lane_spill took D and copied those D words back above the exit frame. Emitting WL_ROOM after WL_PUSHN places the exit frame above the complete continuation frame, so WL_CASE(FID_EXIT) restores all of it before returning into the continuation, and lane_spill(e, sp) needs neither N nor D.
• On !DEVICE, WL_ROOM(F) expands to nothing (previously if (DEVICE && ...) inside every host segment, which Clang folded away; objdump -d shows byte-for-byte identical host machine code on SEQ-CPU and PAR-CPU).
• On DEVICE, WL_ROOM(F) is if (sp >= e.mem + STAT_OFF - 247 * CUBE) { fid = (F); goto sp_spill; }. Because comp.ts caps each continuation segment's parameter count at WIDE (247), a single frame push adds at most WIDE + 1 (248) words, so STAK_LEN - WIDE (1,801 words) is a static watermark: starting below 1,801 words, pushing a frame reaches at most 2,048 words (STAK_LEN), and every word below sp belongs to a complete pushed frame.
• At the bottom of switch (fid) in work_loop on DEVICE, inside default: after return 0;, sp_spill: sp = lane_spill(e, sp); calls FAR DEV Term* lane_spill(Env e, DEV Term* sp) at a single call site where only e, sp, fid, and r0..rn are live, then exits switch (fid) to loop back to WL_SPIN -> switch (fid) without placing a label between WL_SPIN and switch (fid). lane_spill computes sb = e.alc + (STAK_OFF - ALC_OFF), copies the sn = (sp - sb) / CUBE words to a heap block of cls_fit(STAK_LEN), and writes the 3-word exit frame (term_tsk(FID_EXIT, wa), sn, FID_EXIT) at sb, returning sb + 3 * CUBE.
• In WL_CASE(FID_EXIT), unspilling shares the existing WL_ARGS + heap_free + WL_TAKE block with continuation dispatch under if (cont != TERM_HOLE && ((DEVICE && wf == FID_EXIT) || fid_resw(wf))), ending with if (DEVICE && wf == FID_EXIT) { WL_RETN(n); }.

comp.ts goes from 66,000 to 66,268 ttok (+268 ttok, vs +283 in #1439; main is at the 66,000 cap after #1809 kept #1439's token cuts and #1542 used the 283 tokens).

GPU build time (--gpu-build) and register usage

Measured under NVIDIA Runtime Compilation (NVRTC) on an RTX 3090 (CUDA 13.1, sm_86) across 4 interleaved ABBA/BAAB blocks against base 8409268 (identical to de26a91 outside tools/bend-fmt-lsp) and #1439 (40402b4), both with a warm ~/.nv/ComputeCache (which times the NVRTC C++ -> PTX front-end) and with CUDA_CACHE_DISABLE=1 (which times the full uncached C++ -> PTX -> SASS ptxas compile). Register counts come from cuobjdump --dump-resource-usage on bend_dev; lower build time and lower register count are better:

Benchmark	Device WL_ROOM sites	Warm front-end: main / #1439 / PR (ms)	Uncached full compile: main / #1439 / PR (ms)	Uncached PR vs main	Registers (main / #1439 / PR)
symreg	27	264.8 / 273.7 / 271.9	1,431.9 / 1,286.9 / 1,345.2	-6.1%	64 / 72 / 72
merkle	8	261.3 / 261.7 / 266.0	942.0 / 902.6 / 904.3	-4.0%	64 / 68 / 64
raytrace	4	269.2 / 287.5 / 262.4	1,233.5 / 1,246.7 / 1,321.9	+7.2%	92 / 90 / 96
uts-compact	2	255.1 / 271.2 / 267.4	828.4 / 824.9 / 844.5	+1.9%	80 / 80 / 80
tree-matmul	47	320.7 / 350.0 / 347.2	2,671.5 / 2,120.5 / 2,343.7	-12.3%	92 / 80 / 86
queens	5	268.7 / 248.3 / 264.6	946.3 / 925.1 / 959.7	+1.4%	96 / 88 / 96
tree-bitonic	11	248.6 / 288.7 / 285.0	1,115.9 / 1,089.1 / 1,052.8	-5.7%	72 / 80 / 80
lexer	6	-	954.6 / 938.2 / 919.8	-3.7%	72 / 70 / 72

Across all 75 build steps in bend-bench gpu fast (including the three compile-stress programs stress-flatmany, stress-flatchain, and stress-loopchain), total build time is 74.7 s on main vs 75.9 s on this PR, and no step slows by more than 10% and 1 s.

GPU runtime

Measured with bend-bench paired (0f6d45f, interleaved ABBA/BAAB blocks of 4 runs each after correctness check and warmup on RTX 3090, CUDA 13.1, 4 GB heap) against base 8409268; a median block ratio above 1x means the candidate is faster than main:

Workload	Blocks	#1439 (40402b4) vs main	This PR vs main	This PR 95% bootstrap interval	This PR verdict
lexer	8	-	0.997x	0.993x to 1.004x	no detected change
merkle	8	0.996x (within noise)	0.996x	0.994x to 1.014x	no detected change
queens	8	0.933x (6.7% slower)	1.016x	0.983x to 1.037x	no detected change
symreg	8	0.926x (7.4% slower)	0.986x	0.973x to 1.008x	no detected change
tree-bitonic	8	0.497x (50.3% slower)	0.999x	0.947x to 1.014x	no detected change
tree-matmul	8	0.794x (20.6% slower)	1.101x	1.018x to 1.192x	10.1% faster
uts-compact	-	293367 0 (passes)	293367 0 (passes)	-	passes (main fails with ERR_DEEP)

Across the remaining 12 workloads in paired (4 ABBA/BAAB blocks each), 5 are faster (kmeans 1.141x, summation 1.135x, editdist 1.086x, gameoflife 1.058x, mandelbrot 1.021x) and 7 show no detected change (bfs 0.999x, hashmap 1.031x, nbody 1.000x, terrain 1.010x, tree-radix 0.990x, pricing 0.999x, game-search 0.984x).

tests/compile/bang_lane_spill.bend passes on both GPU (+RTS --gpu on) and CPU (+RTS --gpu off), printing 1543298507, 2063974529, and 1878402805.

Other approaches tried

Before settling on goto sp_spill into default: after r0..rn, I tested six alternatives against base 8409268:

Approach	Effect on build or runtime	Why it was rejected
Check sp at top of work_loop before switch (fid)	queens 0.953x (4.7% slower), symreg 0.952x (4.8% slower)	Runs on every WL_RETN and continuation jump, loading FID_T[fid].resw on every work_loop iteration
Direct sp = lane_spill(e, sp) at each WL_ROOM in emit_fork	merkle 64 -> 72 regs, queens 0.977x (2.3% slower)	cargs are still live across WL_ROOM in emit_fork
Direct sp = lane_spill(e, sp) at each WL_ROOM in emit_jump	symreg 64 -> 80 regs, queens 4.0% slower than goto sp_spill	Expanding a call site at all 27 WL_ROOM sites in symreg and 5 in queens raises register pressure in recursive segments
Inline lane_spill body at sp_spill:	queens 96 -> 106 regs, queens --gpu-build +15.6%	Inlining the heap_alloc + copy loop into work_loop bloats work_loop's register allocation
Place if (0) { sp_spill:; ... } above switch (fid)	lexer 0.786x (21.4% slower), queens fast-gpu 96 -> 108 regs	Placing a label between WL_SPIN and switch (fid) reduced NVRTC jump-table cloning (BRX 7 -> 5) in lexer; moving sp_spill: inside default: after return 0; restores BRX 7 and lexer 0.997x
Separate if (DEVICE && wf == FID_EXIT) unspill block in WL_CASE(FID_EXIT)	queens 96 -> 108 regs, symreg --gpu-build +11.6%	Second WL_ARGS + heap_free + WL_TAKE loop in FID_EXIT increases register pressure and build time

[wakamex]

my clanker opened this without me telling them to

to be fair to them, they seem confident about it:

### Only Open Items (Nothing Blocking)

1. `ttok` cap in [repo.ts](file:///code/bend/gates/repo.ts): `main` is at `66,000 / 66,000` for [comp.ts](file:///code/bend/bend2/comp.ts) because `#1809` kept `#1439`'s token cuts and `#1542` consumed that headroom. This PR adds `+268` `ttok` (`66,268`, vs `+283` in `#1439`). Per [AGENTS.md](file:///code/bend/AGENTS.md), only Taelin changes the cap in [repo.ts](file:///code/bend/gates/repo.ts) or golf-trims [comp.ts](file:///code/bend/bend2/comp.ts) if he wants it under `66,000`.
2. Apple M4 Metal `--gpu-build` gate (`gates/perf.ts`): we verified `--gpu-build` on CUDA (`NVRTC` + `ptxas`), where the 27-site `symreg` and 8-site `merkle` builds that regressed by 17% on Metal in `#1809` are now `-6.1%` and `-4.0%` faster uncached (`+2.7%` and `+1.8%` warm front-end), because the 27 inlined 4-argument calls and branches were replaced by `if (sp >= ...) { fid = F; goto sp_spill; }` and a single `lane_spill(e, sp)` call site. The Metal compiler on the mini cluster will confirm the exact `apple_m4` numbers when Taelin runs `gates/perf.ts`.

[wakamex]

needs more work, gonna rent a Mac to test

[wakamex]

okay I rented a Mac ($15 minimum, can't rent for less than a day, thanks Apple) and updated the PR