[volfco]

Part of #2180.

What changes

• cuda_snapshot_layout.zig: with the flag on, the plan carries the arena head in two parts around the dead half of
  the double-buffered KDA recurrent state (halfOf: the layers' parity when every layer agrees, both halves
  otherwise). Plan.initFor builds a restore's plan from the kept state's own parity; Plan.withHalf is the explicit
  form for tests.
• cuda_snapshots.zig: a restore uses the kept state's parity (s.cur); the slot's dead half is left as it was.
• cuda_ranks.zig: the setting (0 or 1, a typo refused by name) and its fingerprint field (layout 20 as assigned in the review (16 is kda_pad on main, 18 and 19 are glm cuda: TF_GLM_L2PF reads a decode window's next weights into L2 on a side stream, on by default on a GB10 #2162's and glm cuda: a prompt step takes two chunks while no other stream decodes, each one's exchanges under the other's sites #1423's shapes), kv_live_half after kda_pad; 23 words before capacity, so four bytes of padding, which nothing reads since the fields compare by name).
• Tests: cuda_snapshot_layout_test.zig (the hole's position and lengths for both parities on a real state of virtual
  addresses, every later part unchanged; the given parity against the slot's; halfOf's fallbacks) and
  cuda_ranks_test.zig (the setting's parse and agreement; the field in the coverage list).
• Docs: the recipe's bullet, a changelog line under Unreleased, and the receipt docs/receipts/glm-kv-live-half-d3e06e76/.

Off by default. No kernel, arithmetic or wire change. 71,303,168 bytes a rank less in every kept state on
GLM-5.3-Flash at TP=2 (34 KDA layers, 32 heads a rank, fp32 128x128 a head).

Why it is safe

The dead half is the state before the last prompt chunk or decode window: the forward reads recAt(cur) and writes
recAt(1 - cur), every chunk and every committed window flips the bit, a keep is refused while a window is pending,
and commit's partial-accept replay reads cur and the window's saved rows into 1 - cur. Nothing reads the dead
half after a restore, so leaving it out of the plan changes no restored computation.

Receipt

Two GB10 (NVIDIA GB10, sm_121, 128 GB shared memory) at TP=2 over a direct RoCE link, Ubuntu 24.04.4, kernel
6.17.0-1018-nvidia, driver 580.159.03, Zig 0.17.0, nvcc 13.0, checkpoint Mia-AiLab/GLM-5.3-Flash-EXL3-4bpw-TensorFold-Ablit
revision 57edefd2, context 65,536, one stream, prompt cache in pinned host memory (16 GiB). Binary built from
d3e06e76c, sha256 in binaries.txt, identical on both ranks.

Four nine-turn greedy conversations on the same binary (3k to 21k tokens; six main turns, a raw keep, a raw restore
and a later restore): whole head and live half, drafts off and drafts on.

The receipt's whole-head arm (TF_GLM_KV_LIVE_HALF=0) is the default path, unchanged by this PR, so it stands in for
main.

• Replies: identical in every turn across the four runs (compare.json).
• Drafts: drafted / accepted / rounds identical turn by turn between the two drafts-on runs (45/30/17, 39/31/16,
  41/26/21, 41/24/23, 42/29/18, 43/24/24, 38/33/14, 35/20/28, 44/29/18); fewer accepted than drafted in every turn,
  so the partial-accept commit path ran on the live half.
• Kept-state sizes, the engine's own prompt cache: lines (MiB a state, rank 0):

keep after turn	whole head	live half	whole head, drafts on	live half, drafts on
1	190	122	194	126
5	296	228	308	240
9	412	344	436	368
nine states	2716	2104	2838	2226

Every keep 68 MiB (71,303,168 bytes) smaller; 22.5 % fewer MiB over the nine states with drafts off, 21.6 % with
drafts on.

• Time to first token: within one run's noise of the whole head in every turn (differences under 0.25 s, no
  consistent sign). No speed claim is made.

Checks

• zig fmt --check clean on the changed files; python3 tools/zig/lean_check.py unchanged against the branch's base
  (424 problems before and after, none in the changed files).
• zig build test-cuda-host passes at the base and on the branch; a failure planted in the new layout test is
  reported by name (one of 2,289 tests) and the restored tree passes.
• Coverage not in this receipt: other TP sizes, concurrent streams, paged sequences (TF_KV_PAGES), the
  page-sharing store (TF_GLM_KV_SHARE), --kv-cache fp8. The head's split is the same code for them.
• ATC: atc overlap --json (ATC 0.7.0) run before opening from a worktree at f869cd189 against upstream main
  8094ec962: 64 open pull requests share files with this change (28 changed files, at most four shared with any one
  pull request); every conflict came back unknown and complete: false (pullFilesUnavailable, timeLimit), as
  the clone has no upstream remote to fetch refs/pull/<n>/head from, so no test merge ran. atc check 2180 --json:
  not claimed, no other pull requests. Not claimed with atc claim; this pull request's Part of #2180 is the claim
  per docs/atc/SKILL.md. Happy to claim or start a session if you want it on the radar.

🤖 Generated with Claude Code

[volfco]

Rebased onto upstream main 8094ec962.

Upstream's c309bb354 had taken fingerprint layout 16 for kda_pad, so the branch now uses layout 17, with kv_live_half placed after kda_pad in the struct, the Field enum, of(), the mismatch list, explain and the test's field list. That makes 23 u32 words before capacity (upstream's 22 plus this one), so capacity now sits after four bytes of padding; the ranks compare the fingerprint field by field, not as bytes, so nothing reads the padding.

The receipt's tested commit d3e06e76c differs from the rebased tree by that fingerprint reconciliation and the changelog line's position only. New commits: c39cff503 (the change) and 5f0d07fb1 (the receipt, with the rebase noted in its commit.txt and README). zig fmt --check and zig build test-cuda-host pass on the rebased tree.

[ashhart]

Thanks @volfco. Off by default the plan is main's, and the nine-turn TP=2 receipt is the evidence we'd want: the same
replies and draft counts turn by turn, and every keep 68 MiB smaller. Two small things for a follow-up, not this PR:
the flag is read twice, by Settings.read and by liveHalfEnabled(), and passing the parsed setting down would keep
one reader. And the combinations you list as untested (TF_KV_PAGES, TF_GLM_KV_SHARE, fp8 KV) stay worth a run
when you have one. Ready.

[ashhart]

One thing on the layout number, @volfco: main is at 16, #2162 is queued and takes 18, and #1423 has agreed 19 after it. So please take 20, and carry #2162's l2pf field alongside yours when you rebase once #2162 has landed. Your field stays yours; each fingerprint shape needs its own number.

[volfco]

Done: the fingerprint layout is 20 now (5914a28), with the comment on the field naming 18 and 19 as #2162's and #1423's shapes. I will carry #2162's l2pf field alongside kv_live_half when I rebase after #2162 lands. Host tests and zig fmt --check pass on the branch; the PR and issue bodies say 20 as well.

[volfco]

Rebased onto main 8abb30f (the conflict was the ranks test file: main's TF_GLM_OVERLAP_PIECES test and this branch's TF_GLM_KV_LIVE_HALF test were added at the same place; both are kept, main's first). Layout stays 20; the commits are 0119e48 (the change), 23e9981 (the receipt) and 50e8cb2 (the layout number and the receipt's rebase note). Host tests and zig fmt --check pass on the rebased tree.

[ashhart]

Thanks @volfco. The rebase onto layout 20 is right, and the restore building its plan from the kept state's own cur bits is exactly what we wanted to see. It's queued to land after #2162, whose layout number comes below yours. Could you add one sentence to the body saying the whole-head arm in your receipt is the default path, so it stands in for main? A single reader for the flag is still welcome as a follow-up, not needed for this one.

[volfco]

Added to the body under Receipt: the whole-head arm (TF_GLM_KV_LIVE_HALF=0) is the default path, unchanged by this PR, so it stands in for main. I'll send the single flag reader as a follow-up.

[ashhart]

Thanks @volfco, that's the sentence. It's queued behind #2162; when you rebase onto it, the recipe's hunk will need fresh context too, since main has a new KV-pages paragraph just above yours.

[volfco]

Rebased onto main 8abb30f → 29d6e8b, as queued behind #2162 (still open, so the l2pf carry is not in yet; layout stays 20 with 18 and 19 still named for #2162 and #1423).

The only conflict was the recipe hunk, as expected: main's new KV-pages paragraph (--prompt-cache-gib before the pool, from the kv_pool work) now sits exactly where the bullet went. Resolved with main's paragraph first, then TF_GLM_KV_LIVE_HALF, then TF_GLM_KV_HOST — both kept.

New commits: f43f279d2 (the change), 20de4e747 (the receipt and the recipe), 98af39c63 (the layout number and the receipt's rebase note, now naming both bases). The PR's Zig files are byte-identical to the previously tested tree, so the receipt's evidence stands; zig fmt --check and zig build test-cuda-host will be confirmed by this push's CI, and I will carry #2162's l2pf field alongside kv_live_half when I rebase after it lands.

[ashhart]

Thanks @volfco, the rebase is right: the recipe bullet now sits after main's KV-pages paragraph, and the Zig is
unchanged from the tested tree. One thing for the rebase after #2162: the CHANGELOG line has drifted into the 1.2.2
section (it sits beside #1577's entry, which shipped in 1.2.2). Please move it under ## Unreleased and add
(#2181) to it.

[volfco]

Rebased onto main c70d67bb2, now that #2162 has landed. Changes:

• The CHANGELOG line now sits under ## Unreleased with (#2181).
• kv_live_half sits beside glm cuda: TF_GLM_L2PF reads a decode window's next weights into L2 on a side stream, on by default on a GB10 #2162's l2pf in Settings and Fingerprint, with layout 20 (18 is glm cuda: TF_GLM_L2PF reads a decode window's next weights into L2 on a side stream, on by default on a GB10 #2162's, 19 stays reserved for glm cuda: a prompt step takes two chunks while no other stream decodes, each one's exchanges under the other's sites #1423).
• The recipe bullet sits after main's TF_GLM_PROMPT_TAIL paragraph.

The Zig differs from the tested tree only in that merge. The rebased tree builds natively on GB10 (sm_121). Ready for merge.

[ashhart]

Thanks @volfco, the rebase is right. kv_live_half sits beside #2162's l2pf at layout 20, the recipe bullet and the CHANGELOG line are where they belong, and with the variable unset the kept plan is main's. Three small things before it lands:

1. A run on this head (ce40dd63). The receipt is from d3e06e76c, before glm cuda: TF_GLM_L2PF reads a decode window's next weights into L2 on a side stream, on by default on a GB10 #2162's L2 prefetch went on by default on GB10. On two GB10s at TP=2, on the checkpoint the recipe names (the receipt's is a modified "Ablit" variant, and receipts live in the repository), the nine-turn conversation with drafts on, in three arms:

   • the variable on;
   • the variable off;
   • a fresh run with --prompt-cache-gib 0.

   Post the reply hashes and cached_tokens turn by turn (on should equal both off and fresh), the off arm against main c70d67bb2 too, and each rank's kept-state sizes (on exactly 71,303,168 bytes smaller, off equal to main).

2. Two local names in the receipt's scripts: $HOME/tensorfold-kernel-night/runtime in launch-pr.sh, and /tmp/atlas-bench.lock in launch-pr.sh and validate-pr.sh. Please neutralise them like the other redactions, and regenerate SHA256SUMS. lazy_receipt.py's docstring also names RUNBOOK-lazy.md and STATUS-lazy.md, which aren't in the repository.

3. The body: "Part of GLM on CUDA: a kept prompt state carries the dead half of the double-buffered KDA state (71 MB a rank) #2180" rather than "Closes GLM on CUDA: a kept prompt state carries the dead half of the double-buffered KDA state (71 MB a rank) #2180", since the single flag reader and the untested combinations are still to come.

We'll set the CHANGELOG ending at landing.

[volfco]

Thanks @ashhart, all three are done, at 3434be78 (rebased onto main 8f5eb7579; the branch's own diff is byte-identical to the tested head 555f052d outside CHANGELOG.md, and main's changes since ca854daf3 touch no GLM file).

1. Run on this head: docs/receipts/glm-kv-live-half-555f052d/. Two GB10s at TP=2, brandonmusic/GLM-5.3-Flash-tr3-4bpw (rev a5fee929), the nine-turn conversation, drafts on, four arms:
   • Replies: identical in all nine turns across on, off, fresh (--prompt-cache-gib 0) and main ca854daf3, and draft statistics are identical too.
   • cached_tokens: on equals off and main turn by turn (0, 2048, … 18432); fresh is 0 throughout.
   • Kept-state sizes: every on keep is exactly 68 MiB = 71,303,168 bytes smaller than its off twin on rank 0, and off equals main state for state. Rank 1 runs no prompt cache of its own and prints no sizes. Its dead half follows from the same plan, and the receipt says so rather than claiming a measurement.
   • Host tests: test-cuda-host fails the same 7 build-runner steps at main and on the branch. Each failing binary passes when run directly.
2. Local names: $NCCL_LIB_DIR and $GPU_LOCK replace them in both receipts' scripts, also in start-ranks-pr.sh and stop-ranks.sh. The lazy_receipt.py docstring no longer names the runbooks, and SHA256SUMS is regenerated.
3. Body: now "Part of GLM on CUDA: a kept prompt state carries the dead half of the double-buffered KDA state (71 MB a rank) #2180".

The recipe bullet cites both receipts, and names the earlier one's checkpoint as modified.

[ashhart]

Thanks, @volfco. All three are in: the four-arm run on this head with replies, draft statistics and cached_tokens equal, and every kept state exactly 71,303,168 bytes smaller with the flag on; the redacted scripts; and "Part of #2180". Layout 20 is still free on main. Two small things:

1. $HOME/kvprobe is still in launch2.sh:6 and validate2.sh:10-11, and in the earlier receipt's launch-pr.sh:6 and server logs. Please neutralise it like the others and regenerate SHA256SUMS.
2. A state kept on disk (--spill-gib) with the flag on: is it refused by a server with the flag off, or misread? If it isn't refused, a word in the disk identity or a refusal would close it.

The CHANGELOG ending is ours at landing, as said.