[bong-water-water-bong]

convert/model: store Zamba2's shared transformer block once per block (engine ggml-org#1036)

Every hybrid layer of a Zamba2 model runs the same transformer block, and the checkpoint stores
that block only num_mem_blocks times (twice at 7B: model.layers.{6,11}.shared_transformer.*).
Our converter wrote one copy per hybrid layer instead — 13 copies of two blocks — so
ref/Zamba2-7B-Instruct-bf16.gguf is 10.977 B parameters / 20.45 GiB for a 7.357 B / 14.8 GB
checkpoint, and every quant arm inherits the inflation.

This writes each block once and makes the loader point the other layers at it.

The change

conversion/zamba2.py — the sharing rule was already computed (visit % num_mem_blocks, sources
= the first num_mem_blocks hybrid layers); the code then duplicated at the end of it. A block
tail is now written once, at the block's source layer. Which tails are shareable is derived from
the checkpoint's own adapter tensors, not hardcoded: a tail that has a per-layer adapter is
per-layer after merging and stays per layer. ffn_up carries the per-layer MLP adapter
(feed_forward.gate_up_proj_adapter_list.<visit>, 13 of them at 7B), so it is not shared.

src/models/zamba2.cpp — the shared set is created TENSOR_NOT_REQUIRED. After the layer loop,
the hybrid layers that actually carry the block are the sources, and every other hybrid layer
aliases attn_post_norm / wq / wk / wv / wo / ffn_norm / ffn_down from
sources[visit % num_blocks]. No new GGUF key, no graph change, no new op.

Old files keep working with the same code, on the old path. A file written before this change
has a copy in every hybrid layer, so every layer is a source, |sources| == |hybrid|, and each
alias is to itself. A file that stores no copy at all is refused once with a named assertion
instead of failing per tensor, and the sources must be the first hybrid layers of the file —
that is what makes the modulo sound, and it is asserted rather than assumed.

Measured

Block multiplicity in the current ref/Zamba2-7B-Instruct-bf16.gguf (13 copies of each tensor,
head/mid/tail sha256 per tensor):

tensor	distinct values
attn_q, attn_k, attn_v, attn_output	2
post_attention_norm, ffn_norm, ffn_down	2
ffn_up (baked MLP adapter)	13
ssm_mix	13

and the 2 distinct values follow visit % num_mem_blocks per layer ([0,1,0,1,...] over the
13 hybrid layers), which is exactly the rule the loader now uses.

Converter, run over the real 7B-Instruct checkpoint's shapes (~/work/zarch/proof/1036-conv-check.py,
reads safetensors headers only):

	tensors	parameters
before	848	10,977,000,000-ish (20.45 GiB BF16)
after	771	8,433,440,592 (15.71 GiB BF16)

= 2,543,439,360 params reclaimed: one block minus gate_up (q+k+v 51,380,224 each +
attn_output 25,690,112 + ffn_down 51,380,224 + the two norms) = 231,221,760, times the 11
duplicate layers. ffn_up and ssm_mix are still written once per hybrid layer.

2.7B fixture, new layout built from the old GGUF (~/work/zarch/proof/1036-prune.py, which derives
the block assignment from the file and refuses one that is not visit % k): 569 -> 520 tensors,
3.828 B -> 3.003 B params, 7.14 -> 5.60 GiB, the 14 source-layer tensors byte-identical to the
originals and all 49 duplicates gone.

Equivalence, same binary, CPU (GGML_HRX=OFF), 4 threads:

arm	file	PPL, 16 x 512	greedy, 24 tok, temp 0
A	old layout (identity path)	16.7937 +/- 0.93864	The capital of France is Paris.
B	new layout	16.7937 +/- 0.93864	The capital of France is Paris.

and B against A's own saved logits, llama-perplexity --kl-divergence over the same 16 chunks
(8192 tokens): Mean KLD 0.000000 +/- 0.000000, Same top p 100.000 +/- 0.000 %,
Δp RMS 0.001 %. Not "close" — identical to the last digit, which is what dropping genuinely
duplicated bytes has to give.

Arm A reproducing 16.7937 also re-proves the identity path: the published 2.7B BF16 number was
taken before this change.

Same two files on the device (gfx1151, --device HRX0 -ngl 99, under gpu-hold):

	STRICT placement	PPL, 1 chunk
A, old layout	exit 0 — every node on HRX	13.7674
B, new layout	exit 0 — every node on HRX	13.7674

GGML_HRX_STRICT=1 aborts on the first node that is not placed on HRX, so exit 0 is the strong form of "no CPU
fallback"; both layouts give it, and the device PPL agrees to every printed digit. New layout throughput, -r 1 on a
die shared with other lanes: pp512 363.23 tok/s, tg64 15.15 tok/s — indicative only.

test-llama-archs -a zamba2: NMSE 0.00e+00 vs CPU, roundtrip OK.

Not reclaimed here (cause 2 stays open)

The remaining 1,130,364,928 params — 11 copies of gate_up (102,760,448 each, 2.26 GiB) — cannot
be dropped without changing the graph: the adapter has to be applied at run time instead of being
baked. That is a separate decision, so engine ggml-org#1036 stays open for it; this PR closes cause 1 only.

Limits of what was run

• The new layout is proven end to end on the 2.7B, whose checkpoint weights are gone, so its
  new-layout file is built by pruning the old GGUF rather than by re-running the converter. The
  converter's side is checked against the real 7B-Instruct checkpoint (emitted tensor set and
  parameter total above) but the 7B file itself is pi-zquant's to rebuild, sequenced after their
  current sweep reads ref/.
• The device arm is a regression check, not evidence for the saving: the change moves tensors between layers and
  never changes an op, so identical placement is the expected result. It was measured once, -r 1, on a die that
  was busy with other lanes for the ~45 minutes the queue needed — the tok/s above is therefore indicative and
  should not be compared with anything. Placement and device PPL are equalities, and those are comparable.
• The 2.7B arm was taken at ~0.09-7 tok/s because the box was shared and thermally throttled; the
  numbers that matter are equalities, not tok/s.

[bong-water-water-bong]

#203 ready for review at 9ad387f

Cause 1 of engine ggml-org#1036, fork side. The block is stored once per block and the loader aliases the
rest; old files take the identity path, which is proved rather than argued: same binary, old 2.7B
layout vs new 2.7B layout gives PPL 16.7937 both, Mean KLD 0.000000 +/- 0.000000 over 16 chunks
(8192 tokens), Same top p 100.000 %, identical greedy string.

Two things a reviewer should push on: (a) the sources must be the first hybrid layers of the file
or the modulo is meaningless -- that is asserted, not assumed; (b) ffn_up is deliberately not
shared (13 distinct per-layer baked MLP adapters), so this reclaims 2.543 B params rather than the
3.67 B the issue's first comment implies -- the arithmetic correction is on ggml-org#1036, and the residual
1.13 B needs cause 2.

[bong-water-water-bong]

VERDICT: READY at 9ad387fb9 — the loader and the converter agree tail for tail and source layer for source layer, and both file generations resolve correctly.

The two halves match exactly. The loader aliases seven tensors (attn_post_norm, wq, wk, wv, wo, ffn_norm, ffn_down) and the converter's _SHARED_TAILS is the same seven (input_layernorm.weight, pre_ff_layernorm.weight, self_attn.q/k/v/o_proj.weight, feed_forward.down_proj.weight), with ffn_up excluded on both sides because the per-layer gate_up_proj_adapter_list is baked into it — the loader keeps ffn_up at flag 0 (required, per layer) while every shared tail becomes TENSOR_NOT_REQUIRED.

The modulo is sound, which is the thing that could have been silently wrong. The converter builds _shared_block_src = _hybrid_layers[:num_mem_blocks] (line 55) and maps visit % num_mem_blocks to a block (line 51); the loader asserts shared_src[k] == shared_use[k] — i.e. the copies are a prefix of the hybrid layers in visit order — and then aliases sources[v % sources.size()]. Those are the same mapping, and the assert is what makes the environment's % equal the converter's.

Both file generations land correctly. Old file: every hybrid layer carries a copy, so every layer is a source, |sources| == |hybrid| == |use|, and v % n == v makes every alias the identity — the old path is preserved exactly, as claimed. New file: sources are the first num_mem_blocks hybrid layers and the rest alias into them. A file with no copy at all is refused once by the two named asserts rather than once per tensor.

Compiled here (CPU Release, -DGGML_HRX=OFF): src/models/zamba2.cpp builds to zamba2.cpp.o with zero warnings and zero errors in the whole build, and python3 -m py_compile conversion/zamba2.py is clean.

Not verified by me: the end-to-end conversion. There is no Zamba2 checkpoint on this box, so the parameter/tensor-count claims (10.977 B → 7.357 B, 13 copies → 2, and every quant arm inheriting the inflation) are the author's. What I can say is that the code that produces them is consistent with the code that consumes them, which is where this class of change usually breaks.

Two residual notes, neither blocking:

1. The loader's modulo trusts sources.size() == num_mem_blocks, and the GGUF carries no num_mem_blocks. A file produced by this converter always satisfies it; a hand-built or partially-shared file where the copies are a correct prefix but fewer than the true block count would alias wrong weights silently. The prefix assert catches the ordering error but not that one. Writing num_mem_blocks into the GGUF hparams would close it whenever the format is next touched.
2. TENSOR_NOT_REQUIRED is per tensor while the "no copy at all" assert keys on wq only. A file that carries wq but omits, say, ffn_down everywhere would alias a null pointer into every layer that needs it instead of hitting the named assert. Again converter-impossible; cheap to widen the assert to the whole set.

[bong-water-water-bong]

Device arm added to the body (the section that said it was not measured).

2.7B, old layout vs new layout, --device HRX0 -ngl 99 under gpu-hold, gfx1151:

	GGML_HRX_STRICT=1	device PPL (1 chunk)
old layout	exit 0	13.7674
new layout	exit 0	13.7674

Exit 0 under STRICT means no node of the graph was placed on anything but HRX, so the aliased graph keeps the
property #178 established (zamba2 runs with 0 nodes off the device) rather than degrading it — which is what has to
be true if the change only moves ffn_down/attn_* ownership between layers. New-layout throughput -r 1 on a
shared die: pp512 363.23, tg64 15.15 tok/s — indicative, not comparable to anything.

Two things I owe the reviewer as honest reporting rather than a clean story: the first device attempt burned a 1830 s
queue slot on invalid device: HRX0 because I launched the tool directly without this box's HSA/no-SVM environment
(it is a documented trap, and 1bit serve sets it for you), and the second expired after 2700 s because the
scoreboard's gpt-oss-120b job held the die. The third, shrunk to one chunk per layout, got through.