[jschmied]

Part of #2005 (step 1b, the protocol layer of section 6.4). Draft, stacked on #2012: the first three commits are #2012's (c55fd8d3c, 442f61776, c68421b38), and the last three are this PR. #2012 has since gained fe6aeebc7 (its frame magic now reads TFST on the wire). Once #2012 lands, I'll rebase this PR onto it so it carries only its own commits.

Host code only: no engine and no GPU. It defines what a pipeline stage and rank 0 say to each other on #2012's link, and ships golden bytes of every message, so a CUDA and a Metal stage can be built separately against the same bytes.

What it adds (zig/src/fabric/stage/)

• payload.zig: every description, ready and plan payload starts with an 8-byte prefix: magic 0x50534654 (its little-endian bytes read TFSP), a u16 protocol version, the message and a reserved byte. fabric: the pipeline stage link, a TCP ring that carries each round's plan and hidden rows, with a shared key (Linux and macOS, #2005 step 1) #2012's frame header stays at version 1.
• boot.zig: a stage's boot description and its ready report, and rank 0's checks:
  • the description: rank and world, engine version, backend, KV dtype, half-open layer range, boundary (hidden width, row dtype, residual planes, drafter width), shapes (prompt chunk, max rows, --parallel, context), owned tap layers, and capacity (KV rows, recurrent states);
  • check refuses the first mismatch by its name: OtherRank, OtherVersion, OtherBackend, OtherKvDtype, OtherRange, OtherWidth, OtherDtype, OtherPlanes, OtherDraftWidth, OtherShapes, OtherTaps, ShortCapacity;
  • the layout check covers all stages together: RangeGap, RangeOverlap, RangeShort, RangeBeyond.
• plan.zig:
  • the round plan: streams in ascending id order, each with an op (open, continue, close, cancel, resume), first position, rows, accepted count and resume point, plus the chunk cuts;
  • the rows frame's layout: plane-major rows in plan order, then the drafter's fp32 tap sums;
  • the bound checks on decode, each refused by its name. No length from the wire is trusted before it is checked against the payload's size and the stage's limits.
• keepalive.zig: rank 0 sends an empty plan every 2 s when idle, and a stage that misses three declares the ring dead. Pure state, tested with injected time.
• fixtures/: 43 golden files, each good message plus planted bad ones, and a README with every layout and the error each bad file must give. fixtures_test.zig re-encodes each good file byte for byte and checks that each bad one is refused with its named error.

The choices section 6.4 left open are written into RFC #2005, section 6.4, under "v0's decisions": half-open ranges, stream order, what a chunk cut and a resume point are, one engine version string, and the boot capacity floor (KV rows ≥ the context, recurrent states ≥ --parallel) that check enforces.

Checks (GB10)

• zig build test-stage: 33 named tests (18 from fabric: the pipeline stage link, a TCP ring that carries each round's plan and hidden rows, with a shared key (Linux and macOS, #2005 step 1) #2012, 15 new), all pass. The runner reports 35/35 because it also counts two unnamed test {} blocks that only import the other files. test-stage-compile builds for Linux and macOS on aarch64.
• lean: 0 new problems. zig build test, the CI set (test-kernel-contract test-golden test-server-security test-pull-security fuzz-cortheon) and git diff --check pass.

[ashhart]

Thanks @jschmied, this is a careful protocol layer: every length is checked before it's used, every mismatch has its own error, and the golden files mean a CUDA and a Metal stage can be built separately against the same bytes.

I read commits 4 and 5 on current main, with #2012's three commits under them. Everything applies, the lean check is unchanged, and nothing outside fabric/stage imports it. A few small things:

• Once fabric: the pipeline stage link, a TCP ring that carries each round's plan and hidden rows, with a shared key (Linux and macOS, #2005 step 1) #2012 lands, could you rebase so this PR carries only its own two commits? The body says one of the commits is fabric: the pipeline stage link, a TCP ring that carries each round's plan and hidden rows, with a shared key (Linux and macOS, #2005 step 1) #2012's, but three are.
• The magic 0x54465350 is called "TFSP", but written little-endian its bytes read "PSFT", as the README's bytes show. It's worth fixing one way or the other before anyone builds a stage against the fixtures.
• check takes kv_rows ≥ context and states ≥ parallel as the capacity floor. Could the v0 decisions say that's the intended floor, and link where those decisions are written, since no docs file changes here?
• I count 18 tests at fabric: the pipeline stage link, a TCP ring that carries each round's plan and hidden rows, with a shared key (Linux and macOS, #2005 step 1) #2012's head and 33 after this PR. The 15 new ones match; the "20" doesn't.

We'll run zig build test-stage on a GB10 and on a Mac when it's ready to land after #2012.

[jschmied]

Thanks @ashhart. All four are in, at 05a8853ab:

• Stack: the body now names fabric: the pipeline stage link, a TCP ring that carries each round's plan and hidden rows, with a shared key (Linux and macOS, #2005 step 1) #2012's three commits, c55fd8d3c, 442f61776 and c68421b38. Once fabric: the pipeline stage link, a TCP ring that carries each round's plan and hidden rows, with a shared key (Linux and macOS, #2005 step 1) #2012 lands, I'll rebase so this PR carries only its own commits.
• Magic: the payload magic is now 0x50534654, so its little-endian bytes on the wire read TFSP, and all 43 fixtures are rewritten. It's a new commit, 05a8853ab, so the earlier two stay as you read them. fabric: the pipeline stage link, a TCP ring that carries each round's plan and hidden rows, with a shared key (Linux and macOS, #2005 step 1) #2012's frame magic 0x54465354 has the same issue: its bytes read TSFT. Should I fix it there (to 0x54535654, bytes TFST) before fabric: the pipeline stage link, a TCP ring that carries each round's plan and hidden rows, with a shared key (Linux and macOS, #2005 step 1) #2012 lands, or leave the frame as it is?
• Capacity floor: RFC RFC: a layer-range (pipeline) layout across mixed nodes: one hidden row per token over Ethernet, GB10 and Mac in one model #2005, section 6.4, "v0's decisions", now states it as the intended boot floor, KV rows ≥ the context and recurrent states ≥ --parallel, with per-request admission later in step 7. The PR body links there.
• Test count: you're right, 33 named tests, 18 from fabric: the pipeline stage link, a TCP ring that carries each round's plan and hidden rows, with a shared key (Linux and macOS, #2005 step 1) #2012 and 15 new. My "20/35" was the runner's count, which also includes two unnamed test {} blocks that only pull in the other files. The body says so now.

Checks on 05a8853ab (GB10): zig build test-stage passes and test-stage-compile builds for Linux and macOS; lean 424 on the base, 424 on the head, 0 new; zig build test, the CI set and git diff --check pass.

[ashhart]

Thanks @jschmied, all four are in. The magic's little-endian bytes now read TFSP, and none of the 43 fixtures still carries the old order. The capacity floor and the link to the v0 decisions are in, and so are the test counts. On the frame magic: yes, please fix it in #2012 before it lands, so both read the same way on the wire. Once #2012 is in and this is rebased onto it, we'll run test-stage on a GB10 and on a Mac and land it.

[jschmied]

Thanks @ashhart. The frame magic is fixed in #2012, at fe6aeebc7:

• The magic is now 0x54534654, so its bytes on the wire read TFST. The value I suggested above, 0x54535654, was wrong: its bytes read TVST.
• The frame test now checks the first four bytes of an encoded header against "TFST", and it fails with the old value.
• zig build test (which runs test-stage), the CI set and diff-check are green, and the lean check is unchanged (0 new).
• Both PR bodies say this now.

Once #2012 lands, I'll rebase this PR onto it so it carries only its own commits.

[ashhart]

Thanks @jschmied. Nothing is left on this PR's own commits. One thing before it can land: CI's fmt job fails on fabric/stage/secret.zig and frame.zig. Those come from #2012, and #2012 shows the same failure, so a zig fmt there fixes both. Once #2012 lands and this is rebased down to its own three commits, we'll run test-stage on a GB10 and on a Mac and land it.