[anon761]

What

Several ordinary GGUFs of Qwen3.8-Flash-Next ship per_layer_token_embd.weight in a format the PLE reader refuses at start:

file	PLE table
Unsloth UD-Q6_K_XL	Q8_0 (170-byte rows, ~54 GB)
Swift-1.5 Q4_K_L	Q8_0
a community Q5_K_M	Q5_1 (120-byte rows)

strata generate: per_layer_token_embd.weight is Q5_1, not IQ4_NL, Q5_0 or FP8 (I8)

The rest of these files already has a path in the engine (Q5_K/Q6_K/Q5_1/Q8_0 experts, the K-quant projections, --compat-bf16). For the Q5_K_M file the table was the only thing keeping it from starting (end to end below); the Q8_0 tables are covered by the parity test on the real files.

• dequantize_q5_1 in strata/artifact/dequant.hpp (ggml's dequantize_row_q5_1: unsigned 5-bit codes times d plus the block minimum m); Q8_0 rows use the existing dequantize_q8_0
• PleTable decodes Q5_1 / Q8_0 rows on both readers. The direct reader already takes the row size at runtime (since the FP8 table), so no reader change was needed; PLE_ROW_BYTES_MAX grows to 170
• --ple-io ram: before mlock, 16 threads fault the table in. mlock alone brings a cold file in one page at a time from one thread; on a ZFS pool that read the 54 GB Q8_0 table at ~0.47 GiB/s (about two minutes), 16 threads at ~2 GiB/s (measured from a cold cache before the rebase onto current main; the container this was re-tested in cannot drop the file cache). The fallback when mlock fails keeps the pages faulted in instead of touching them a second time.

Tests

• dequant_q5_1_test (new, CTest): 4096 synthetic blocks incl. the edge codes 0 and 31 and a negative minimum against ggml's to_float, bit-exact. No model, no GPU.
• dequant_bf16_test knows Q5_1.
• ple_q5_parity now takes a Q5_0, Q5_1 or Q8_0 table and checks both the mapped and the direct reader (single rows and a 16-row batch) against ggml's to_float on the real file:

Q5_1 PLE, mapped reader: 320001536 rows, max_abs 0.000e+00 PASS     (Q5_K_M, shard 1)
Q5_1 PLE, direct reader: 320001536 rows, max_abs 0.000e+00 PASS
Q8_0 PLE, mapped reader: 320001536 rows, max_abs 0.000e+00 PASS     (Swift-1.5 Q4_K_L, shard 2)
Q8_0 PLE, direct reader: 320001536 rows, max_abs 0.000e+00 PASS
Q8_0 PLE, mapped reader: 320001536 rows, max_abs 0.000e+00 PASS     (UD-Q6_K_XL, shard 3)
Q8_0 PLE, direct reader: 320001536 rows, max_abs 0.000e+00 PASS

End to end

The Q5_K_M file (Q5_1 table) on 2x RTX 3090, layer split auto, 262K context, --kv int8 --pcie-frac 0; main refuses it at start:

--ple-io	prompt 2K	prompt 8K	decode
ram (table locked in 3.2 s, warm file cache)	705 tok/s	2116 tok/s	85-87 tok/s
direct (default)	703 tok/s	2117 tok/s	65-74 tok/s

[ksanislo]

I made the same Q8_0 part of this independently before finding this PR, so here is what I measured with it, in case
it helps the review. My change (PleTable Q8_0 rows at 170 B through the existing dequantize_q8_0,
PLE_ROW_BYTES_MAX 170) is equivalent to the Q8_0 part here; I'm not opening a second PR.

End to end with a Q8_0 table. Unsloth's plain Q8_0 (6 shards; experts Q8_0, the table Q8_0 in shard 3), no
--ple-gguf, on 4 x Tesla T4 (sm_75), a Xeon E5-2660 v2 (the AVX older-CPU build, STRATA_ISA_FLOOR=avx), 4-GPU
layer split auto, 262,144 context, resident experts:

• the log names the table: PLE on, table 320001536 rows of .../Qwen3.8-Flash-Next-Q8_0-00003-of-00006.gguf
• correct answers at temperature 0: arithmetic, a trick question ("all but 9" sheep), a Python function, a tool call;
  images through strata-vision; streaming
• 8,495-token prompt / 600 out, 2 runs: 436-440 prompt tok/s, 24.8-24.9 decode tok/s - the same as the same model
  with an IQ4_NL table passed by --ple-gguf (426-434 / 22.3-24.9)

How far each table encoding is from Qwen's original. Qwen/Qwen3.8-Flash-Next (@ de4b8e4) stores the 128
n-gram shards as BF16; the F8_E4M3 shards with one weight_scale that tools/ple_fp8_pack.py reads come from the
separate Qwen/Qwen3.8-Flash-Next-FP8 (@ 236dfdf). With the BF16 table as the reference, 1,536 rows from 24 random
places (range reads of both checkpoints), relative L2 error per row:

Encoding	mean	max
Q8_0 (Unsloth's Q8_0 GGUF, shard 3)	0.53%	0.68%
FP8 E4M3 (ple_fp8_pack.py)	2.66%	3.31%
IQ4_NL (Unsloth's UD-Q4_K_XL table)	7.59%	9.35%

The IQ4_NL figure matches the 8.1% in 505e5e9, which was measured against the FP8 values. So for a model whose GGUF
carries a Q8_0 table, reading it directly is the closest of the readable encodings to the original, not only a way
to start.

One small addition you may want: ple_reader_test --selftest covers 90 and 110 B rows; with 170 added
(selftest(dir, 170)) it passes here too (row_bytes=170: OK).

[anon761]

Superseded by #896: rebased on the current main (this branch conflicted after the upstream changes to ngram.cpp) and carrying the same Q5_1/Q8_0 table support plus these tests.

[Niko1221]

Fixed in 0.1.40: the engine reads a Q8_0 or Q5_1 n-gram table, on the direct and the mapped reader, as one entry each in a new table of PLE formats (it also reads Q4_0, Q5_0, FP8 and BF16 now). --ple-io ram faults the table in on up to 16 threads before mlock (8 GB cold in WSL: median 15 s before, 6 s after). Your commit's authorship is kept. Measured on 4,000 random rows against the checkpoint's BF16 table: Q8_0 0.53%, FP8 2.64%, IQ4_NL 7.60%.