[txiki739]

What

After Hardin22/Strata-DualGPU 11b1f25 (run_fused_native)
by Francesco Albano: every expert's gate/up parts, then every expert's down parts in one phase; the thread that finishes
an expert's last gate/up part quantizes its activations, and that expert's down parts wait for it. No barrier between
the halves and no serial quantization on the host in between; the second GPU's submit still runs when the phase opens.
The same operations and bytes; the two-phase path is unchanged and stays the default.

Measured

+2-4% in every config here (six-prompt decode mean, interleaved runs). UD-IQ4_XS on the RTX 3090: 83.3-84.6 ->
85.2-86.4 tok/s; on top of the elastic K/V of this series 90.8-91.0 -> 92.8-93.0.

Where this sits (measured today on your new custom)

custom at e3727cb against our engine (your previous custom + this series + the upstream pieces you have merged
too), our configs on a Ryzen 7 5700X + 128 GB DDR4 + RTX 3090 (+ RTX 5060 Ti 16 GB), Linux. custom runs the same
configs without the flags it lacks (--kv-grow, --adapt-decay, --vram-pin-gib / --second-gpu-pin-gib):

six-prompt decode mean, two interleaved runs each	custom e3727cb	our engine (this series)
UD-IQ4_XS, RTX 3090	80.6 tok/s	93.7 (+16%)
UD-Q4_K_XL, RTX 3090	64.4	69.4 (+8%)
UD-IQ4_XS, RTX 3090 + RTX 5060 Ti	103.8	126.1 (+21%)
UD-Q4_K_XL, RTX 3090 + RTX 5060 Ti	78.2	92.0 (+18%)
reading an 18K prompt, both cards (UD-IQ4_XS / UD-Q4_K_XL)	2,110 / 2,010-2,270 tok/s	2,530 / 2,420
the first chat right after READY, both cards (--ple-io ram)	67.8 / 52.8 tok/s	110.9 / 85.8

The per-change measurements and the configs: README: what the changes on top of eddoursul's custom gave,
settings and how to run, examples/r7-5700x.

This series (rebased onto e3727cb)

• load_main: strata-load compiles again #12 load_main: strata-load compiles again
• Linux startup: the expert arena filled with preadv and page-locked layer by layer #13 Linux startup: the expert arena filled with preadv and page-locked layer by layer
• --ple-io ram: READY only once the PLE table is in RAM #14 --ple-io ram: READY only once the PLE table is in RAM
• cpu pool: a native layer's gate/up and down rows as one batch (STRATA_POOL_FUSED=1, opt-in) #16 cpu pool: a native layer's gate/up and down rows as one batch (STRATA_POOL_FUSED=1, opt-in)
• Elastic K/V (--kv-grow): the K/V takes VRAM as the context grows, the expert cache holds the rest #19 Elastic K/V (--kv-grow): the K/V takes VRAM as the context grows, the expert cache holds the rest
• Experts only in VRAM (--vram-pin-gib, --second-gpu-pin-gib) #20 Experts only in VRAM (--vram-pin-gib, --second-gpu-pin-gib)
• adaptive tier: --adapt-decay F (upstream's option), refused outside (0, 1) #21 adaptive tier: --adapt-decay F (upstream's option), refused outside (0, 1)

Closed as already in custom after the 0.1.43 merge: #15 (re-split GGUFs), #17 (upstream's CPU kernels), #18 (upstream fixes).

Tested on custom at e3727cb

Smoke runs (UD-IQ4_XS, greedy, a Spanish chat answer and a code prompt, RTX 3090 or both cards): with STRATA_POOL_FUSED=1 78.8 / 85.0 -> 80.8 / 87.8 tok/s (+3%), the same tokens.

[eddoursul]

Thank you: merged into custom; pool_test --stress now runs the native formats through the fused phase too (bitwise the serial runs). On this machine it stays opt-in: time a round against the two-phase path, code / thinking / after a 16K prompt / an edit, IQ3_S -0.8 / -0.5 / -0.3 / -0.7%, UD-Q4_K_XL +2.2 / -0.2 / +0.6 / 0.0% (the pool here reads UD's experts near the RAM's rate, so the barrier is a small share of a layer).

(Measured on a Ryzen 7 9700X, 160 GB DDR5-5600, RTX 3090 + RTX 5070 Ti as the second expert tier, Windows 11; IQ3_S and UD-Q4_K_XL with this branch's configs, fixed text through --spec-follow, 2 rounds in both orders.)