npu-pack: the q4_1 zero-point nibbles are paired by halves, the codes by even/odd (main, since #829)

Summary. npu/pack_import.cpp's q4_1 tile decoder pairs the zero-point nibbles of each subvolume into rows by
halves, while the codes of the same subvolume pair by even/odd. The shipped pack's own bytes say even/odd is right, so
every container imported since the attention payload landed (#829) carries another row's zero point in half of its rows,
for q/k/v/o and (once it lands) the MLP projections.

Where. origin/main:npu/pack_import.cpp:622-624 (decode_tile) and :656-657 (encode_tile):

const int n0 = r0 + 2 * cp;                       // codes: byte cp -> rows 2cp and 2cp+1
...
const uint8_t z = s[kZerosOff + i];
t.zp[r0 + i]           = z & 0x0F;                // zero points: byte i -> rows i and i+16
t.zp[r0 + 16 + i]      = z >> 4;

Evidence (no device, one file). A correctly paired zero point sits where its 128 codes cluster, so per
(row, 128-block) mean(code) should track zp[row][block]:

                                  pairing     corr      |mean(code)-zp|
decode attn tile L0 (2,176 B)     halves   +0.0229        2.3787
decode attn tile L0 (2,176 B)     evenodd  +0.9970        0.1605
decode fused gate|up L0 (4,288 B) halves   +0.0328        0.9861
decode fused gate|up L0 (4,288 B) evenodd  +0.9621        0.2111
prefill gate L0 (2,176 B)         halves   +0.0620        1.0203
prefill gate L0 (2,176 B)         evenodd  +0.9665        0.2107
prefill down L0 (2,176 B)         halves   +0.0598        0.9468
prefill down L0 (2,176 B)         evenodd  +0.9613        0.2102

A shuffled zp gives corr ~0 and |diff| ~1.01, so the halves reading is noise-level on every shape.

Independently, the pack's two physical layouts of the same weights agree only under even/odd: the fused 4,288-byte
gate|up group (#842) and the prefill file's separate gate/up tensors have identical codes 24/24 layers, identical scales
24/24, and identical zero points 24/24 under even/odd, 0/24 under halves. The row correspondence is pinned by the
codes, so that is not an alignment artefact.

Why the existing checks cannot see it. encode_tile mirrors decode_tile, so "survives decode->encode byte for
byte" (768/768 attention tiles, 2160/2160 MLP groups) is invariant under any permutation applied on both sides, and the
committed goldens were generated by this same decoder.

Fix. Pair the zero nibbles as the codes do (r0 + 2*i low, r0 + 2*i + 1 high) in decode_tile and
encode_tile; then regenerate tests/golden/oga_attn_tile.bin (via tools/oga_attn_tile_golden.py) because it is a
fixture for the current behaviour, and add a check that catches the class: a fixture whose zero points are not
symmetric under the halves/even/odd swap (e.g. one whose eight zero bytes are all distinct), or the code-cluster test
above turned into an assertion.

Impact. w = (code - zp) * scale, so a wrong zero point offsets that row's weights by Δzp x scale — a few int4
steps out of a 16-step range. Wrong numbers, not a crash: import, container read-back and every byte-level gate so far
pass with it.

FACTS §7.17 ("zero index = (col%64)/2") is the correct description; §7.13's "rows 0..15 low, 16..31 high" is the reading
the code inherited, and the shipped bytes contradict it.

Verified at origin/main 906f966a2. The line numbers are exact and the asymmetry is real; three things below are what the fix has to carry with it.

The pin, quoted. Codes pair even/odd — npu/pack_import.cpp:622-624: const int n0 = r0 + 2 * cp; then rows n0 and n0 + 1. Zero points pair by halves — same block: t.zp[r0 + i] = z & 0x0F; / t.zp[r0 + 16 + i] = z >> 4;. encode_tile mirrors it at :655-657, exactly as reported.

Zeros are the only field in that order. Scales decode over kSubvRows rows in natural order (:626-629) and codes use the (cp % 4) + (cp / 4) * 512 byte map, so the fix is those two 2-line blocks — no format change, no header change, no decoder restructuring.

The fixtures are wrong after the fix, and there is a sequencing fact. tests/golden/oga_attn_tile.bin, oga_attn_tile_q4_1.bin and .json are all produced by this decoder (tools/oga_attn_tile_golden.py, which has --check), and tests/npu_pack_import_test.cpp:394,439 compare against them and nothing else — so the PR must regenerate them and add the assertion that catches the class (all-distinct zero bytes, or the code-cluster test as an assertion), or the same permutation stays invisible. Sequencing: PR #842 (open) adds tests/golden/oga_mlp_gu*.bin generated by the current decoder, so #849 should land with or before #842 and both should be regenerated once. Re-running #842's own evidence after the fix is the positive control: the fused 4,288-byte gate|up group vs the prefill file's separate gate/up should be 24/24 under even/odd and 0/24 under halves, the mirror image of today.

Impact on #796 is stacked, not additive. #847 (noncausal mask) is the other measured cause of the same parity gap, so the re-measure order is #849 → #847 → parity.

Owner pi-ogaload (it owns #671/#734/#842/#844/#845); bucket A.

Scoping fact for this issue's priority, measured off the shipped image rather than inferred: nothing already released is
affected. The cycle-14 appliance image's NPU container is GGUF-converted, not npu-pack import output.

$ sudo losetup -o <1BIT-DATA offset> 1bit-os-puppy.img ; mount -o ro
/mnt/c14data/models/Qwen3-0.6B/config.json
  "q4nx_source": { "converter": "tools/gguf_to_q4nx.py", "gguf": "Qwen3-0.6B-Q4_K_M.gguf" },
  "q4nx_weight_layout": "rm"
  model.q4nx  683,820,936 B

So the four NPU models in the image (onebit-models.txt: Qwen3-0.6B/1.7B/4B-NPU-1bit, Qwen3.6-35B-A3B-NPU-1bit) come
through tools/gguf_to_q4nx.py, which does not share decode_tile with the OGA importer. The blast radius of the
halves-paired zero points is therefore containers built with 1bit npu-pack import — the AMD OGA route documented at
docs/npu.md:376-395, i.e. #829's attention import and #842/#844's MLP/final-norm work — and no shipped artifact.

That does not lower the fix's priority much for me: #842 and #844 are the PRs that would ship such a container, and the
defect is silent (both goldens were generated from the same decoder, and decode->encode round-trips are invariant under a
permutation applied twice). But it does mean this is not a reason to re-cut c14.

npu dequant: decode a plain bf16 tensor as bf16, not as 5120-byte q4_…

Closing note (issue keeper): closed by `#879 → f3d9f6c0` — the goldens now regenerate from the shipped pack, which is the acceptance the row asked for

#879's title carries (closes #849); GitHub closed this at 10:45:22Z. Verified device-free at 2fc5c4fd:

python3 tools/oga_attn_tile_golden.py /home/bcloud/models/oga/Qwen2.5-0.5B-Instruct_rai_1.8.0_npu_4K --check
golden matches the pack

That is the check the issue specified, on the pack the bug was found on: the corrected decoder derives tests/golden/oga_attn_tile.bin → oga_attn_tile_q4_1.bin byte-identically from the vendor's own .fconst bytes, so the committed goldens are no longer "generated by the broken decoder and therefore agree with it". The two blocks are paired even/odd now — npu/pack_import.cpp:626-627 (decode) and :659-660 (encode) — with scales untouched at :690.

Residual named in the row and still true: #842's oga_mlp_gu* goldens came from the halves-paired decoder and need regenerating in the same way; that is #842's item, and its layout re-run at 24/24 even/odd is what closes #855's container half.

npu forward: read the batched prefill's GEMM outputs at the kernel's …

npu-pack: check the MLP payload's zero-point pairing too (engine#849'…

npu-pack: record runtime refusals in the manifest, and fix three test…

npu-pack: import the .fconst MLP section (gate|up fused, then down_pr…

Navigation Menu

Repository navigation

npu-pack: the q4_1 zero-point nibbles are paired by halves, the codes by even/odd (main, since #829) #849

Closing note (issue keeper): closed by `#879 → f3d9f6c0` — the goldens now regenerate from the shipped pack, which is the acceptance the row asked for

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

npu-pack: the q4_1 zero-point nibbles are paired by halves, the codes by even/odd (main, since #829) #849

Description

Activity

bong-water-water-bong commented on Nov 2, 2026

bong-water-water-bong commented on Nov 3, 2026

bong-water-water-bong commented on Nov 3, 2026

Closing note (issue keeper): closed by #879 → f3d9f6c0 — the goldens now regenerate from the shipped pack, which is the acceptance the row asked for

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Closing note (issue keeper): closed by `#879 → f3d9f6c0` — the goldens now regenerate from the shipped pack, which is the acceptance the row asked for