[txiki739]

Problem

On custom (e3727cb) the strata-load tool does not build: src/core/load_main.cpp still reads LoadStats::ok /
::error, which the loader no longer has (it puts the reason in its own message and signals a failure with
seconds < 0).

Fix

load_main follows the loader's failure form. No engine code changes.

gr_parity fails on custom too, for another reason that is yours to decide: since the 0.1.43 merge it is upstream's
test, which checks upstream's fused-read quantization (FusedGrArgs::q8_mixed / q8_cnt, a fused_gr_read_multi
that returns bool), and this tree's fused_gr has neither. This PR leaves it alone (this tree's
ple_reader_test does not build either, also from the merge).

Where this sits (measured today on your new custom)

custom at e3727cb against our engine (your previous custom + this series + the upstream pieces you have merged
too), our configs on a Ryzen 7 5700X + 128 GB DDR4 + RTX 3090 (+ RTX 5060 Ti 16 GB), Linux. custom runs the same
configs without the flags it lacks (--kv-grow, --adapt-decay, --vram-pin-gib / --second-gpu-pin-gib):

six-prompt decode mean, two interleaved runs each	custom e3727cb	our engine (this series)
UD-IQ4_XS, RTX 3090	80.6 tok/s	93.7 (+16%)
UD-Q4_K_XL, RTX 3090	64.4	69.4 (+8%)
UD-IQ4_XS, RTX 3090 + RTX 5060 Ti	103.8	126.1 (+21%)
UD-Q4_K_XL, RTX 3090 + RTX 5060 Ti	78.2	92.0 (+18%)
reading an 18K prompt, both cards (UD-IQ4_XS / UD-Q4_K_XL)	2,110 / 2,010-2,270 tok/s	2,530 / 2,420
the first chat right after READY, both cards (--ple-io ram)	67.8 / 52.8 tok/s	110.9 / 85.8

The per-change measurements and the configs: README: what the changes on top of eddoursul's custom gave,
settings and how to run, examples/r7-5700x.

This series (rebased onto e3727cb)

• load_main: strata-load compiles again #12 load_main: strata-load compiles again
• Linux startup: the expert arena filled with preadv and page-locked layer by layer #13 Linux startup: the expert arena filled with preadv and page-locked layer by layer
• --ple-io ram: READY only once the PLE table is in RAM #14 --ple-io ram: READY only once the PLE table is in RAM
• cpu pool: a native layer's gate/up and down rows as one batch (STRATA_POOL_FUSED=1, opt-in) #16 cpu pool: a native layer's gate/up and down rows as one batch (STRATA_POOL_FUSED=1, opt-in)
• Elastic K/V (--kv-grow): the K/V takes VRAM as the context grows, the expert cache holds the rest #19 Elastic K/V (--kv-grow): the K/V takes VRAM as the context grows, the expert cache holds the rest
• Experts only in VRAM (--vram-pin-gib, --second-gpu-pin-gib) #20 Experts only in VRAM (--vram-pin-gib, --second-gpu-pin-gib)
• adaptive tier: --adapt-decay F (upstream's option), refused outside (0, 1) #21 adaptive tier: --adapt-decay F (upstream's option), refused outside (0, 1)

Closed as already in custom after the 0.1.43 merge: #15 (re-split GGUFs), #17 (upstream's CPU kernels), #18 (upstream fixes).

Tested on custom at e3727cb

Every target that built on custom still builds, and strata-load builds again (custom: gr_parity, ple_reader_test and strata-load fail; with this PR the first two).

[txiki739]

Rebased the series onto your new custom (e3727cb, upstream 0.1.43 merged) and measured the two engines side by side.

Still open, rebased and tested on e3727cb: #12, #13, #14, #16, #19, #20, #21.
Closed, already in custom: #15, #17, #18.

Your new custom against our engine (your previous custom + this series + the upstream pieces you merged too), our
configs on a Ryzen 7 5700X + RTX 3090 (+ RTX 5060 Ti), same configs minus the flags custom lacks:

six-prompt decode mean, two interleaved runs each	custom e3727cb	our engine (this series)
UD-IQ4_XS, RTX 3090	80.6 tok/s	93.7 (+16%)
UD-Q4_K_XL, RTX 3090	64.4	69.4 (+8%)
UD-IQ4_XS, RTX 3090 + RTX 5060 Ti	103.8	126.1 (+21%)
UD-Q4_K_XL, RTX 3090 + RTX 5060 Ti	78.2	92.0 (+18%)
reading an 18K prompt, both cards (UD-IQ4_XS / UD-Q4_K_XL)	2,110 / 2,010-2,270 tok/s	2,530 / 2,420
the first chat right after READY, both cards (--ple-io ram)	67.8 / 52.8 tok/s	110.9 / 85.8

Where the difference comes from (each measured on its own, in the PRs):

• Elastic K/V (--kv-grow): the K/V takes VRAM as the context grows, the expert cache holds the rest #19 elastic K/V: +10% on one card at a 200K context (the K/V no longer takes ~1,000 expert slots from the start);
• Experts only in VRAM (--vram-pin-gib, --second-gpu-pin-gib) #20 experts pinned in VRAM (two GPUs): +6-8%, and the hottest experts survive long prompts;
• --ple-io ram: READY only once the PLE table is in RAM #14 READY waits for the PLE table: the first chat after a start 68 -> 111 tok/s on two cards (the table's load competed with it);
• cpu pool: a native layer's gate/up and down rows as one batch (STRATA_POOL_FUSED=1, opt-in) #16 fused CPU pool: +2-4%; adaptive tier: --adapt-decay F (upstream's option), refused outside (0, 1) #21 --adapt-decay 0.92: +3% on one card;
• Linux startup: the expert arena filled with preadv and page-locked layer by layer #13 the arena load: ready in 20 s instead of 56 (warm), 38-43 s instead of 52-55 (cold).

[txiki739]

One more measurement, to put the whole series in one number: your new custom (e3727cb) with these seven PRs on top,
merged into one branch (txiki739/Strata3090-5060TI edd-custom-plus-prs:
11 commits, no conflicts), against our engine, the same full configs on both sides (custom + the PRs understands
every flag of them), six-prompt decode mean, two interleaved runs each:

	our engine	custom + the seven PRs
UD-IQ4_XS, RTX 3090	94.0 tok/s	93.9 (a second run at 88.0 read its prompt at 615 tok/s: a slow run this machine has now and then with any binary)
UD-Q4_K_XL, RTX 3090	69.6	70.2
UD-IQ4_XS, RTX 3090 + RTX 5060 Ti	127.2	127.5
UD-Q4_K_XL, RTX 3090 + RTX 5060 Ti	91.9	91.6
reading an 18K prompt, both cards (UD-IQ4_XS / UD-Q4_K_XL)	2,531 / 2,413 tok/s	2,539 / 2,421

So the 8-21% between custom and our engine is all in these PRs: with them custom matches it, and we would rather
follow your branch than keep ours. If any of them should be done differently to fit how you want things, tell me and I
will redo it.

[eddoursul]

Thank you: merged into custom. On today's custom gr_parity still stopped at a check for upstream's STRATA_QFUSE read, which the 0.1.43 merge brought in for a kernel this branch does not have; that check is gone now (6845bd1), and every tool target builds.

[txiki739]

Thank you for merging the series, and for the fixes you made on the way (gr_parity, the guards around the pins, the
Windows paths). We are moving this machine onto your custom instead of keeping our own branch: from here we will
send changes to you and follow your branch.

One more thing, with no expectation attached: if any of these changes looks useful for upstream (Niko1221/Strata),
feel free to propose it there yourself, in whatever form suits you. They are built on this branch's engine core, so you
know better than we do how they would map onto upstream's, and you are welcome to take them under your name with a
mention. Nothing needed from your side.