[merbanan]

#463 made a verify window wait for the adaptive tier's previous swaps (cudaEventSynchronize) so that whether a swapped-in expert runs on the GPU or the CPU in a window - they round differently - no longer depends on when its PCIe copy lands. The wait is bounded by the copy volume: on an RTX 2060 SUPER 8 GB over PCIe 3.0 x8 (Q2_0, 2,262 cache slots) it costs ~4.6 ms of a ~50 ms window at every context length, the same as dropping it with STRATA_ADAPT_NOWAIT=1 recovers.

Now a window takes the pending swaps only once they are STRATA_ADAPT_LAG windows old (default 2), and then waits for their copies, which have landed by then. The window that first computes a swapped-in expert on the GPU still depends only on the window count, so greedy output is as reproducible as with #463's wait; a swapped-in expert just counts one window later. STRATA_ADAPT_LAG=1 is #463's behaviour; STRATA_ADAPT_NOWAIT=1 keeps its meaning. Both decode loops (serve and generate).

[midhatn]

I tested this on Windows with a Ryzen 9 7940HS, RTX 4070 Laptop 8 GB and 64 GB DDR5-5600. There is a promising decode result, but I wouldn't call it a task-completion speedup yet.

I adapted ae64f76 onto v0.1.39 (6f32ec0), preserving the newer pin_live / unpin_blobs / source-release handling. Both settings used the same binary. Swift 1.5 IQ3_XXS, 64K allocated context, INT8 KV, 1,040 GPU-cached expert entries, 7 workers plus the caller, kernel uploads, PCIe fraction 0.35, MTP 5 with English drafting and min-p 0.5. Other experimental paths were off.

For the same short coding request, an A-B-B-A run gave:

Run	Admission lag	Decode tok/s	Output tokens, including reasoning	Request wall time	Accepted/offered drafts
A1	1	30.4	1,130	38.94 s	650/953
B1	2	33.2	1,105	35.04 s	674/924
B2	2	32.9	1,428	45.07 s	852/1,148
A2	1	30.5	1,220	41.81 s	718/1,042

Requests used high reasoning, seed 4242, temperature 1, top-p 0.95, top-k 20 and the same 3,072-token output cap. All four generated implementations passed an independent 155-case check, plus generator, invalid-input and non-mutation checks. The surrounding tool-call and prompt-cache checks also passed: 36 requests across these four runs.

The median decode rate rose about 8.5%, while median request time was essentially unchanged: 40.37 vs 40.05 seconds. Draft acceptance and generated lengths also changed, so this doesn't isolate the saved admission wait. It supports further testing on a PCIe 4.0 laptop, but it doesn't test the greedy reproducibility claim or establish a general quality improvement.

[merbanan]

@midhatn you have to good hardware for this patch to matter much.

[star1872]

On the slow hardware this is meant for (PCIe 3.0 x8, DDR3) it is a clear gain. An independent measurement, ported by hand onto 0.1.39 (6f32ec0; the hunks are the same, the context
moved).

PC: Xeon E5-2673 v3 (AVX2), 94 GB DDR3 dual channel, RTX 4060 Ti 16 GB on PCIe 3.0 x8 (engine probe 6.2 GB/s).
Original Q2_0, setup's default config (64K, int8 KV, --spec 4 --spec-min-p 0.5), PR #706's kernel in both arms.
Server, 4 prompts (Chinese essay, code edit, translation, explanation), greedy, 512 tokens, thinking off, 2 passes
each; one binary, STRATA_ADAPT_LAG=1 for #463's behaviour, the arms interleaved:

	mean of the 4 prompts
STRATA_ADAPT_LAG=1 (#463), runs 1 and 6	48.71, 49.60 tok/s
this PR (lag 2), default tier	52.61 tok/s (+7.0%)

In the code-edit windows the time the window sums do not account for (the wait for the swap copies) was 4.4 ms of a
68 ms window with lag 1; with lag 2 it is gone. The gain grows with the tier's swap volume: with
--adapt-every 1 --adapt-swaps 80 --adapt-decay 0.97 lag 1 gives 53.19 and lag 2 gives 55.25 / 54.61 tok/s.

[Niko1221]

In 0.1.40: STRATA_ADAPT_LAG is in. The default is 1, which is exactly 0.1.39; lag 2 is opt-in until we have a KL check. pending_age is reset after pending.clear().