A container the suite can reach at index_bits 6, and a VQ4P arm in run.sh

[mfethe1]

Requested by @Ultron09 in #41: teach make_test_container.py --index-bits 6 and wire the VQ4P engine check into run.sh. This is that PR; the kernel it covers is vq_rows_p6, which #41's AVX-512 work joins.

Why

CLAUDE.md names the trap this closes:

the same applies to --index-bits 6, which exercises a different kernel and a different record fmt

running the suite on one is how the Q4G load path stayed broken through green runs

Until now vq_rows_p6 had no container make check could reach: the synthetic one is always index_bits 8, and a real --index-bits 6 conversion needs the source weights and hours. My own review on #41 said I had tried to close this and could not — this is the closing half, and #41's kernel gets the same checks for free once it lands.

What

--index-bits 6 writes the one combination the engine accepts at 6 (src/model.c refuses anything but 4 stages × 64 entries): records carry the WQ_VQ4P fmt byte, indices packed 4×6 bits into 3 bytes with the same little-endian bit order and [M/B][pos][row] blocking as convert.py's block_indices_packed, and the manifest's expert_quant carries index_bits/fmt/bits_per_weight exactly as convert.py writes them.

The default path emits byte-identical containers: manifest.json gains no key (the engine reads absence as 8), so the rotary fixture's container_sha256 gate is untouched. Verified per-file by sha256 at seed 0, plain and --rope, before and after.

run.sh gains a VQ4P engine (index_bits 6) section running the same four self-comparisons the engine block runs on VQ3R: records through the C structs (test_container now accepts the WQ_VQ4P fmt byte — same size and layout as VQ3R, only the payload packing differs), chunked == token-at-a-time, SIMD vs the CPU baseline, and expert-cache bit-identity. A few MB, milliseconds to build, so it runs on every host CI does.

Evidence

Same tree (2fa0796 on main at 6430585), two platforms:

host	board	VQ4P arm
macOS arm64, clang 17	48 passed, 0 failed, 13 skipped (baseline 44/0/13)	all 4 PASS — NEON vq_rows_p6
Windows 11, Ryzen 7 3700X (Zen 2, no AVX-512), MSYS2 UCRT64 gcc 16.2.0	46 passed, 0 failed, 14 skipped (baseline 42/0/14)	all 4 PASS — scalar vq_rows_p6

Both boards are exactly baseline + 4: no prior PASS moved. The SIMD-vs-baseline check lands in the suite's existing "within fp noise" branch on both hosts, same as the VQ3R arm; the cache check is bit-identical on both. Also: the same-seed VQ4P container is byte-identical across Windows and macOS (6/6 file sha256s equal — the newline="\n" discipline #47 landed, working), and the 4×6 packing round-trips against the engine's P6_J0..P6_J3 macros on 1,000,000 random rows with 0 mismatches.

What this does not claim: quality of the noise codebooks (there is none — it is noise), and nothing about the AVX-512 kernel itself, which stays #41's to land; when it does, this arm covers it on Cascade Lake / Skylake-SP CI runners without further work.

[mfethe1]

Following up on the #41 measurement (dispatched vs WASTE_BACKEND=cpu: max diff 0.41, 162233/163840 over 1e-3 on NEON; @GTSUlteer's 1.23 / 163297 on AVX2): the same comparison on the tiny synthetic container here is max diff 9.5e-07, 0/256 over 1e-3 (mac-arm64, NEON vs CPU). So the arm in this PR passes — but for the reason you predicted, not the reason the check implies: the synthetic weights never land on the discontinuous side of the int8 LUT, so 'within fp noise' is data-luck, not a property of the kernel.

That makes the arm weaker than it looks on real containers, and I'd rather say so on the PR than let a green run carry it. The check that would actually pin it is the one you named in #41 — bit-identity of the int8 table (or of the routes) against the scalar path — since a broken kernel and a single entry rounding the other way are indistinguishable at logits level on 6-bit indices. The LUT is a pure function of the codebook, so it can be checked exhaustively in milliseconds with no container at all: build the table scalar, build it through the kernel's lane path, cmp the 256 (or 4096, per stage) bytes.

If that sounds right I'll send it as a follow-up — test_vq_lut.c, container-free, runs on every platform including the CI-forced synthetic path.

[mfethe1]

Traced this further after PR #41 merged — the dispatch table now has three real vq_rows_p6 implementations (model.c:2129 portable, kda_neon.c:100 NEON, simd_avx512.c:129 AVX-512 VBMI), all consuming the same int8_t* lut8 built once by vq_quant_range (model.c:2054, not itself in the dispatch table — same code path regardless of WASTE_BACKEND).

That rules out the LUT-building step as the source: given the same float activations, vq_quant_range produces the same int8 table and scale on every backend. Given the same table, all three consumers do pure integer accumulation with no float reordering to disagree on — the code's own comment at model.c:2166 says as much. So the divergence has to come from upstream of quantization: WASTE_BACKEND=cpu doesn't only swap vq_rows_p6, it swaps every dispatched kernel in the forward pass, so the float activations feeding that token's vq_quant_range call differ minutely (ordinary SIMD-reduction-order noise, ~1e-6 relative) between a normal run and a CPU-forced one — and vq_quant_range's own scale is max_abs/127, so that ordinary noise gets amplified into a 1-LSB flip for any table entry sitting near a k+0.5 boundary. There's already a documented instance of exactly this amplification in this file (model.c:2365: "a 1e-8 difference... moves every entry near a rounding boundary by one LSB. That measured as 0.68 on a logit"). Real weights and real activations are generic enough that some block, somewhere across 163,840 outputs, always sits near enough to a boundary for this to show — which is also the honest answer to why the synthetic-noise container in this PR doesn't hit it: not enough boundary-adjacent structure in that data, not a property of the kernel being more correct there.

That confirms the fix precisely: the test needs to hold the int8 table fixed (not derived from a live, backend-dependent forward pass) and compare only the integer-accumulation step across backends. Scoped the harness assuming a test_vq_lut.c in the shape already proposed above:

• Build a handful of fixed vqp_arg cases directly (lut8, packed 4×6-bit idx via the P6_J0..P6_J3 macros in simd.h, lscale, unit scale) — including entries deliberately placed at k±0.5 boundaries, since that's the case a live-activation test would only hit by chance.
• Call waste_vq_rows_p6 directly (already extern, declared in kda.c) as the reference, and waste_k.vq_rows_p6 after waste_backend_init(WASTE_BE_AUTO) vs WASTE_BE_FORCE_CPU for whichever accelerated kernel the host has.
• Diff the integer accumulator output exactly — no tolerance, since there's no float step left once the table is fixed.

I don't have AVX-512 VBMI or ARM hardware here to build and run the SIMD arm myself, so I stopped short of writing it rather than ship an untested harness — happy to build the portable-reference half now if useful, or defer entirely if this is already in progress on your end.

[mfethe1]

Built the portable-reference half described above: tests/test_vq_lut.c, container-free, on mfethe1/warp@test/vq4p-lut-portable (0e796c3, based on main@619b666 post-#41).

It holds lut8 and the packed idx fixed by hand (no forward pass involved) and diffs waste_vq_rows_p6's output against an independently-coded reference in the same file — same P6_J0..P6_J3 macros (the on-disk bit layout, not the algorithm under test), separate summation loop, no shared object code with the engine. Four cases: generic table/indices, every 6-bit index value 0-63 exercised across j0..j3 (with an inline self-check that the hand-packing round-trips through the real macros before trusting the case), extreme -127/+127 LUT entries to check the int16 accumulator width, and a non-unit varying output scale so the final f16 fold isn't hidden by a scale of exactly 1.0.

Built via make libwaste.a (needs the Makefile's per-file -mavx2/-mavx512f etc. — a hand-rolled gcc src/*.c throws "target specific option mismatch" on the AVX files) and linked against it on Windows 11 Ryzen 7 3700X, MSYS2 UCRT64 gcc 16.2.0:

PASS generic-table-generic-idx    (64 rows, 4 vectors)
PASS full-index-range             (64 rows, 4 vectors)
PASS extreme-values               (64 rows, 4 vectors)
PASS nonunit-output-scale         (64 rows, 4 vectors)

Exit 0, exact equality (not a tolerance) on every row — correct once the table is fixed, since there's no float step left to disagree on.

Before trusting that result I checked the check itself: swapped one lookup in the reference (P6_J1 for P6_J2) and reran. 3 of 4 cases immediately FAILed with real numeric divergence (e.g. row 0: kernel=12.8125 reference=7.5 diff=5.3125). The 4th (extreme-values) didn't move, for a specific and checkable reason rather than a blind spot in the harness: that case fixes every index byte at zero to isolate accumulator width from addressing, so j1 and j2 are both 0 there and the swap is invisible to it by construction. Restored the correct reference, rebuilt, reconfirmed all 4 PASS.

What this doesn't cover: the SIMD arm (kda_neon.c:100, simd_avx512.c:129) against this same fixed-table harness — I don't have AVX-512 VBMI or ARM hardware to build or run it. The portable kernel is the one every host actually falls back to when neither ISA backend is present, so this closes that half on its own; the accelerated paths still need someone with the hardware to point vqp_arg at the same fixed cases and diff against the portable output here (which is now a known-correct oracle) rather than against a live, backend-noisy forward pass.

Branch: https://github.com/mfethe1/warp/tree/test/vq4p-lut-portable

[marcobambini]

This closes a real gap, it was asked for, and I want it. Two changes first, one
of them a regression against your own #42.

Reviewed by running it rather than reading it, and against current main
rather than the base you wrote on — #41 landed after you opened this, so
vq_rows_p6 is now behind the dispatch table. It merges clean and the arm works
there: 53 → 57, exactly +4, no prior check moved. The --rope container's
container_sha256 still matches the shipped fixture, so your byte-identity
claim for the default path holds on this machine too.

---

1. Blocking: cmp -s instead of same() — this is #42 undone

Both new comparisons call cmp -s directly. #42 replaced exactly that pattern
one commit earlier, and the cache check is the same call site by name. With
cmp hidden behind a PATH that holds everything else:

PASS  VQ4P records read through the C structs (6 records over 3 banks)
PASS  VQ4P chunked prefill == token-at-a-time
PASS  VQ4P SIMD backend matches the CPU baseline (within fp noise)
FAIL  VQ4P expert cache changes results
45 passed, 1 failed, 16 skipped

The engine accused of a bit-identity violation because diffutils is not
installed — on UCRT64, which is the platform you wrote #42 for. It is the only
failure in an otherwise clean board.

The SIMD check is fine as it stands: no cmp drops it into the fp branch, which
is a real verdict, and that is what #42 decided for that shape deliberately. It
is only the cache one. Same treatment as the VQ3R cache check directly above:

same "$TMP/p6_seq.bin" "$TMP/p6_cache.bin"
case $? in
    0) ok "VQ4P expert cache is bit-identical to no cache" ;;
    1) no "VQ4P expert cache changes results" ;;
    *) sk "VQ4P expert cache is bit-identical to no cache" "$NO_CMP" ;;
esac

2. Please tighten the tolerance — 1e-3 spends margin you have

I mutated the NEON kernel to find out what the arm can actually see. Same
container, same ids, one edit at a time:

vq_rows_p6_neon	max diff	arm says
intact	9.54e-07	PASS
j3 shift 2 → 3 (whole index extraction wrong)	0.002053	FAIL
T1/T2 swapped (classic port slip)	0.002944	FAIL
one int8 LUT entry, +1 step	0.002944	FAIL

It catches all three, so the arm is not decorative — but a grossly wrong
kernel clears the threshold by 2x while the intact one sits 1000x under it.
There are three orders of magnitude between good and broken here and 1e-3 is
parked at the wrong end of them. 1e-5 would leave 10x over the intact noise
and 300x under the broken signal.

Worth noting the last row, because it is #37's point at synthetic scale: a
single LUT entry rounding the other way is indistinguishable from two stage
tables swapped. Both 0.002944. A tolerance cannot separate "broken" from
"rounded differently" here either — it just happens that at 3 layers neither
reaches 1e-3 by accident. That is an argument for tightening the number now and
for the bit-identity gate #37 and #38 converged on later, not against this PR.

3. The source comment claims something the check does not assert

run.sh's new block says:

the p6 accumulate is integer until the per-block fold, so agreement is exact
rather than within fp noise

The check prints VQ4P SIMD backend matches the CPU baseline (within fp noise)
on this host, and your own evidence table says it lands in the fp branch on both
of yours. The reasoning is right about vq_rows_p6 itself; what reaches the
logits also went through the other dispatched kernels. Someone reading that
comment will think the arm asserts exactness and that a fallback message means
something regressed.

4. One scope line I would like in the comment, not a change request

The container is 4 layers — 3 MoE — 8 experts top-2, hidden 128. The same
comparison on a real index_bits: 6 container, on this machine, with the
kernel this PR covers working correctly:

kimi-linear-vq4p (27 layers) : max diff 0.581133, 162632/163840 over 1e-3
tiny6            (3 MoE)     : max diff 9.54e-07

The arm never touches a real container, so it will not produce that as a false
FAIL — that part of the design is right and I checked it. But it does mean
"this arm covers it on Cascade Lake / Skylake-SP CI runners without further
work" promises more than three layers of noise can carry, on a kernel whose
known failure mode is depth-amplified. A sentence saying the arm bounds gross
errors and not the discontinuity would keep the next reader from over-trusting a
green run — which is the exact failure CLAUDE.md names two lines above the
text you quoted.

---

None of this is about the packing, the fmt byte, test_container, or the
default-path byte-identity, all of which check out. Fix 1, tighten 2, adjust the
comment in 3, and I will merge it.

[mfethe1]

All four addressed in e29d4f7, verified on macOS arm64 (NEON): 48 passed, 0 failed, 13 skipped.

1. cmp -s → same() with the 0/1/* case, identical in shape to the VQ3R cache check directly above it. You were right that it was A missing cmp or python3 said the engine was broken, not the tool absent #42 undone — and at the same call site by name, which is the embarrassing part. The SIMD-vs-CPU check keeps its cmp -s deliberately: no cmp drops it into the fp branch, which is a real verdict, exactly as A missing cmp or python3 said the engine was broken, not the tool absent #42 decided for that shape.

2. Tolerance is now 1e-5. For what it is worth, the arm lands in the fp branch on this host too and passes it — intact max diff is under 1e-5 here as well, consistent with your 9.54e-07.

3. Comment rewritten: the accumulate being integer until the per-block fold is a fact about vq_rows_p6, not about what this check asserts, since the logits also went through the other dispatched kernels. It now says agreement lands in the fp-noise branch and that this is a real verdict, not a regression.

4. Scope sentence added with your kimi-linear number (max diff 0.581133 at 27 layers, kernel correct): the arm bounds gross errors, not the depth-amplified discontinuity mode, and a green run must not be read as more than that.

The single-LUT-entry flip being indistinguishable from a stage-table swap at logits level (both 0.002944) is a cleaner argument for the #37/#38 bit-identity gate than anything I had — noted for the test_vq_lut.c follow-up, where your mutation table sets the bar: three single-edit kernel mutations, all caught, headroom quantified both directions.

[mfethe1]

@marcobambini All four points are addressed in e29d4f7 — item-by-item in the comment above this one: the same() swap with the 0/1/* handling at the cache check (cmp -s kept only where its absence degrades to a real fp verdict, per #42's own rule for that shape), the 1e-5 tolerance on the SIMD-vs-CPU fallback, the rewritten accumulate comment, and the scope note quoting your kimi-linear number. CI is green on the new head across all nine jobs, including the windows-native build+suite and run. Ready for re-review whenever you have a moment.