arena: pin by need (STRATA_PIN_BY_NEED=1) and a PCIe share floor (STRATA_PCIE_MIN1=1), +5-7 % prompt read on Windows

bench/results/2026-10-11-community-rtx3090-5600x-arena-pin-by-need/README.md

@@ -0,0 +1,42 @@
+# Pin by need (`STRATA_PIN_BY_NEED=1`) — community report, RTX 3090 + Ryzen 5 5600X (Windows 11)
+
+Branch `pr/pin-by-need` on v0.1.42 (`61b3fb5d`). Results and method only; the change is in the pull request.
+
+**Machine and config.** As in the `--adapt-async` report: RTX 3090 24 GB (also drives the desktop), Ryzen 5 5600X, 2 x 32 GB
+DDR4-3600, Windows 11; Qwen3.8-Flash-Next GSQ-RCO IQ3_S, the default arena, 8,289 expert slots, `--spec 4 --mtp --max-context
+262144 --kv int8 --vram-reserve-mib 1200 --kv-grow --adapt-every 1 --adapt-swaps 80 --adapt-decay 0.97`. Local builds (VS 2022
+17.14, CUDA 12.8, sm_86): `v0142` the tag, `pr2_on` this branch with `STRATA_PIN_BY_NEED=1 STRATA_ARENA_PIN_GIB=29
+STRATA_PCIE_MIN1=1 --pcie-frac 0.6`.
+
+**What it registers.** v0.1.42: `cudaHostRegister of the whole arena FAILED (out of memory); 32 slices pinned (30 GiB)`, layers
+0-31, layers 32-47 without a PCIe share. This branch: `pin by need: 4404 ranges, 28.96 GiB pinned, 92.2 % of the non-resident
+experts (48 of 48 layers have a PCIe share)`, 8,124 of the profile's 24,576 pairs taken as GPU-resident.
+
+## Prompt read and decode after it (prompt read once, decode = mean of 3 seeded answers)
+
+| Prompt tokens | `v0142` prefill tok/s (2 servers) | `pr2_on` prefill tok/s | Decode `v0142` / `pr2_on` |
+|---:|---|---:|---|
+| 55,630 | 2,180 / 2,165 | **2,313 (+6.5 %)** | 95.3 / 96.5 |
+| 116,833 | 2,095 / 2,088 | **2,208 (+5.6 %)** | 97.0 / 98.3 |
+
+Same day, on top of `--adapt-async 1` and other opt-in switches: 63K-token prompt read 2,180 → 2,310 tok/s.
+
+## Cold short prompts (12 prompts x 2 rounds, fresh server per arm and round, paired)
+
+| Arm | Median tok/s | Ratio vs reference | Prompts faster |
+|---|---:|---:|---:|
+| `pr2_on` vs `v0142` (blocking tier) | 99.3 vs 97.2 | 0.999 | 6/12 |
+| on top of `--adapt-async 1` + other opt-in switches: pin by need + `--pcie-frac 0.6` | — | 1.023 | 10/12 |
+| same + `STRATA_PCIE_MIN1=1` (1 round) | — | 1.026 | 10/12 |
+| pin by need alone on top of the same (no PCIe settings) | — | 0.995 | 6/12 |
+
+Decode gains only together with `--adapt-async 1` and a larger PCIe share; the prompt read gains alone.
+
+## Default path
+
+Bit-exact mode (`STRATA_IQ_MT_MIN=1 --pcie-frac 0 --adapt-every 0`), greedy, 6 prompts: on the 5 prompts that v0.1.42
+reproduces between two of its own fresh engines, this branch without the variables gives the same answers byte for byte;
+the 6th differs between two runs of v0.1.42 itself in that mode (`bitexact*.jsonl`). Startup load unchanged (6.7 GiB/s; the
+padded layout reads through the buffered loader).
+
+Raw: `context.jsonl`, `cold.jsonl`, `cold_on_async*.jsonl`, `bitexact*.jsonl`; harness `strata_ab2.py`.

bench/results/2026-10-11-community-rtx3090-5600x-arena-pin-by-need/bitexact.jsonl

@@ -0,0 +1,4 @@
+{"variant": "x_v0142", "engine": 0, "shas": ["27519074f61de9af", "feab63eaed0e3c87", "13083c7bae76321f", "2851571399ae73d5", "b7c60255ff57c1f0", "4fa026f724d6d21f"]}
+{"variant": "x_v0142", "engine": 1, "shas": ["27519074f61de9af", "feab63eaed0e3c87", "13083c7bae76321f", "935c0cbca0b8e848", "b7c60255ff57c1f0", "4fa026f724d6d21f"]}
+{"variant": "x_pr1", "engine": 0, "shas": ["27519074f61de9af", "feab63eaed0e3c87", "13083c7bae76321f", "2851571399ae73d5", "b7c60255ff57c1f0", "4fa026f724d6d21f"]}
+{"variant": "x_pr2", "engine": 0, "shas": ["27519074f61de9af", "feab63eaed0e3c87", "13083c7bae76321f", "4e95b8f3c12c7032", "b7c60255ff57c1f0", "4fa026f724d6d21f"]}

bench/results/2026-10-11-community-rtx3090-5600x-arena-pin-by-need/bitexact_fixed_cache.jsonl

@@ -0,0 +1,4 @@
+{"variant": "y_v0142", "engine": 0, "shas": ["c3688f98a481a45b", "6a807808abdeebf8", "9e346e42ddce5684", "46d8b674ef38e1ea", "4c8597aef38d0f07", "b9c90f31eb5f6303"]}
+{"variant": "y_v0142", "engine": 1, "shas": ["c3688f98a481a45b", "6a807808abdeebf8", "9e346e42ddce5684", "ea52f5b1f95102cc", "4c8597aef38d0f07", "b9c90f31eb5f6303"]}
+{"variant": "y_pr1", "engine": 0, "shas": ["c3688f98a481a45b", "6a807808abdeebf8", "9e346e42ddce5684", "fdbc72441cc0a805", "4c8597aef38d0f07", "b9c90f31eb5f6303"]}
+{"variant": "y_pr2", "engine": 0, "shas": ["c3688f98a481a45b", "6a807808abdeebf8", "9e346e42ddce5684", "46d8b674ef38e1ea", "4c8597aef38d0f07", "b9c90f31eb5f6303"]}

bench/results/2026-10-11-community-rtx3090-5600x-arena-pin-by-need/context.jsonl

@@ -0,0 +1,24 @@
+{"session": "ctx_v0142_64", "sha": "dc33602cc71a7040", "notes": ["strata: K/V grown to 57344 cells (0.76 GiB); the expert cache gave 269 slots for it, 104 hotter experts moved to colder ones' slots (8020 of 8289 slots hold experts)", "strata serve: decode expert cache hit rate: 93.8% (86361 hits / 92096 lookups); 1401 more read by the GPU over PCIe or from another GPU (1.5% of all 93497 routed)"], "prompt_tokens": 55630, "prefill_tps": 2179.7, "gen": 157, "decode_tps": 82.6, "acc": 83, "drafted": 128, "variant": "v0142", "size_k": 64, "rep": 0, "wall_s": 28.0}
+{"session": "ctx_v0142_64", "sha": "addf507444dd2f59", "notes": ["strata serve: decode expert cache hit rate: 97.7% (93347 hits / 95519 lookups); 197 more read by the GPU over PCIe or from another GPU (0.2% of all 95716 routed)", "strata serve: suffix drafts: 1 windows, 1 of 3 drafts accepted"], "prompt_tokens": 55630, "prefill_tps": 164.5, "gen": 139, "decode_tps": 91.9, "acc": 70, "drafted": 135, "variant": "v0142", "size_k": 64, "rep": 1, "wall_s": 2.1}
+{"session": "ctx_v0142_64", "sha": "8d830fd4dcc5ef03", "notes": ["strata serve: decode expert cache hit rate: 98.3% (92600 hits / 94157 lookups); 124 more read by the GPU over PCIe or from another GPU (0.1% of all 94281 routed)"], "prompt_tokens": 55630, "prefill_tps": 154.6, "gen": 152, "decode_tps": 101.8, "acc": 81, "drafted": 127, "variant": "v0142", "size_k": 64, "rep": 2, "wall_s": 2.1}
+{"session": "ctx_v0142_64", "sha": "2fd40ee569e49c46", "notes": ["strata: K/V grown to 122880 cells (1.57 GiB); the expert cache gave 432 slots for it, 182 hotter experts moved to colder ones' slots (7588 of 8289 slots hold experts)", "strata serve: decode expert cache hit rate: 97.1% (105047 hits / 108157 lookups); 296 more read by the GPU over PCIe or from another GPU (0.3% of all 108453 routed)"], "prompt_tokens": 116833, "prefill_tps": 2094.6, "gen": 184, "decode_tps": 100.2, "acc": 101, "drafted": 149, "variant": "v0142", "size_k": 128, "rep": 0, "wall_s": 58.3}
+{"session": "ctx_v0142_64", "sha": "cf80c246816a95eb", "notes": ["strata serve: decode expert cache hit rate: 98.2% (90131 hits / 91758 lookups); 139 more read by the GPU over PCIe or from another GPU (0.2% of all 91897 routed)", "strata serve: suffix drafts: 1 windows, 1 of 2 drafts accepted"], "prompt_tokens": 116833, "prefill_tps": 151.3, "gen": 147, "decode_tps": 94.1, "acc": 75, "drafted": 121, "variant": "v0142", "size_k": 128, "rep": 1, "wall_s": 2.2}
+{"session": "ctx_v0142_64", "sha": "f5b9a378a44be80c", "notes": ["strata serve: decode expert cache hit rate: 98.2% (90132 hits / 91804 lookups); 153 more read by the GPU over PCIe or from another GPU (0.2% of all 91957 routed)"], "prompt_tokens": 116833, "prefill_tps": 155.9, "gen": 146, "decode_tps": 98.0, "acc": 81, "drafted": 128, "variant": "v0142", "size_k": 128, "rep": 2, "wall_s": 2.2}
+{"session": "ctx_pr2_on_64", "sha": "bd9159551143f58a", "notes": ["strata: K/V grown to 57344 cells (0.76 GiB); the expert cache gave 269 slots for it, 127 hotter experts moved to colder ones' slots (8020 of 8289 slots hold experts)", "strata serve: decode expert cache hit rate: 94.5% (88795 hits / 93948 lookups); 1860 more read by the GPU over PCIe or from another GPU (1.9% of all 95808 routed)", "strata serve: suffix drafts: 1 windows, 0 of 3 drafts accepted"], "prompt_tokens": 55632, "prefill_tps": 2313.4, "gen": 157, "decode_tps": 84.4, "acc": 82, "drafted": 132, "variant": "pr2_on", "size_k": 64, "rep": 0, "wall_s": 26.5}
+{"session": "ctx_pr2_on_64", "sha": "72de14399ee8bb0c", "notes": ["strata serve: decode expert cache hit rate: 97.9% (78697 hits / 80345 lookups); 341 more read by the GPU over PCIe or from another GPU (0.4% of all 80686 routed)", "strata serve: suffix drafts: 2 windows, 2 of 6 drafts accepted"], "prompt_tokens": 55632, "prefill_tps": 161.4, "gen": 132, "decode_tps": 104.8, "acc": 76, "drafted": 113, "variant": "pr2_on", "size_k": 64, "rep": 1, "wall_s": 1.9}
+{"session": "ctx_pr2_on_64", "sha": "33ea903aa671dd18", "notes": ["strata serve: decode expert cache hit rate: 98.2% (96489 hits / 98285 lookups); 430 more read by the GPU over PCIe or from another GPU (0.4% of all 98715 routed)"], "prompt_tokens": 55632, "prefill_tps": 170.3, "gen": 156, "decode_tps": 100.3, "acc": 86, "drafted": 137, "variant": "pr2_on", "size_k": 64, "rep": 2, "wall_s": 2.2}
+{"session": "ctx_pr2_on_64", "sha": "686a3421945a8b06", "notes": ["strata: K/V grown to 122880 cells (1.57 GiB); the expert cache gave 432 slots for it, 199 hotter experts moved to colder ones' slots (7588 of 8289 slots hold experts)", "strata serve: decode expert cache hit rate: 97.3% (88138 hits / 90538 lookups); 576 more read by the GPU over PCIe or from another GPU (0.6% of all 91114 routed)"], "prompt_tokens": 116835, "prefill_tps": 2208.3, "gen": 145, "decode_tps": 95.0, "acc": 81, "drafted": 129, "variant": "pr2_on", "size_k": 128, "rep": 0, "wall_s": 55.1}
+{"session": "ctx_pr2_on_64", "sha": "7963d83bcdf2f406", "notes": ["strata serve: decode expert cache hit rate: 98.0% (105985 hits / 108110 lookups); 504 more read by the GPU over PCIe or from another GPU (0.5% of all 108614 routed)"], "prompt_tokens": 116835, "prefill_tps": 150.6, "gen": 163, "decode_tps": 96.6, "acc": 92, "drafted": 158, "variant": "pr2_on", "size_k": 128, "rep": 1, "wall_s": 2.4}
+{"session": "ctx_pr2_on_64", "sha": "16291bcce37326a7", "notes": ["strata serve: decode expert cache hit rate: 98.7% (91148 hits / 92347 lookups); 167 more read by the GPU over PCIe or from another GPU (0.2% of all 92514 routed)"], "prompt_tokens": 116835, "prefill_tps": 146.8, "gen": 152, "decode_tps": 103.4, "acc": 86, "drafted": 128, "variant": "pr2_on", "size_k": 128, "rep": 2, "wall_s": 2.1}
+{"session": "ctx_pr1_on_64", "sha": "4077b52877022011", "notes": ["strata: K/V grown to 57344 cells (0.76 GiB); the expert cache gave 269 slots for it, 110 hotter experts moved to colder ones' slots (8020 of 8289 slots hold experts)", "strata serve: decode expert cache hit rate: 93.1% (89924 hits / 96640 lookups); 1712 more read by the GPU over PCIe or from another GPU (1.7% of all 98352 routed)", "strata serve: asynchronous adaptive tier: 95 rounds, 4677 experts swapped in, 36.1 ms per round (start to flip, between windows) (arena, copy-in only, batch 0, wait 1, pick inline 0.12 ms/round)"], "prompt_tokens": 55632, "prefill_tps": 2166.9, "gen": 154, "decode_tps": 86.0, "acc": 81, "drafted": 139, "variant": "pr1_on", "size_k": 64, "rep": 0, "wall_s": 28.1}
+{"session": "ctx_pr1_on_64", "sha": "87c1110ca570330a", "notes": ["strata serve: decode expert cache hit rate: 98.0% (90063 hits / 91881 lookups); 177 more read by the GPU over PCIe or from another GPU (0.2% of all 92058 routed)", "strata serve: asynchronous adaptive tier: 163 rounds, 5461 experts swapped in, 33.0 ms per round (start to flip, between windows) (arena, copy-in only, batch 0, wait 1, pick inline 0.11 ms/round)", "strata serve: suffix drafts: 1 windows, 1 of 3 drafts accepted"], "prompt_tokens": 55632, "prefill_tps": 166.2, "gen": 148, "decode_tps": 100.9, "acc": 83, "drafted": 128, "variant": "pr1_on", "size_k": 64, "rep": 1, "wall_s": 2.1}
+{"session": "ctx_pr1_on_64", "sha": "a12ab4060e0f25f2", "notes": ["strata serve: decode expert cache hit rate: 98.3% (81884 hits / 83275 lookups); 117 more read by the GPU over PCIe or from another GPU (0.1% of all 83392 routed)", "strata serve: asynchronous adaptive tier: 227 rounds, 5946 experts swapped in, 31.8 ms per round (start to flip, between windows) (arena, copy-in only, batch 0, wait 1, pick inline 0.10 ms/round)"], "prompt_tokens": 55632, "prefill_tps": 141.6, "gen": 139, "decode_tps": 104.9, "acc": 75, "drafted": 113, "variant": "pr1_on", "size_k": 64, "rep": 2, "wall_s": 1.9}
+{"session": "ctx_pr1_on_64", "sha": "a3a5ee334c85c2de", "notes": ["strata: K/V grown to 122880 cells (1.57 GiB); the expert cache gave 432 slots for it, 199 hotter experts moved to colder ones' slots (7588 of 8289 slots hold experts)", "strata serve: decode expert cache hit rate: 96.8% (83154 hits / 85916 lookups); 250 more read by the GPU over PCIe or from another GPU (0.3% of all 86166 routed)", "strata serve: asynchronous adaptive tier: 294 rounds, 7075 experts swapped in, 31.2 ms per round (start to flip, between windows) (arena, copy-in only, batch 0, wait 1, pick inline 0.10 ms/round)"], "prompt_tokens": 116831, "prefill_tps": 2088.5, "gen": 139, "decode_tps": 95.9, "acc": 74, "drafted": 117, "variant": "pr1_on", "size_k": 128, "rep": 0, "wall_s": 58.0}
+{"session": "ctx_pr1_on_64", "sha": "36d42325cea1a1cb", "notes": ["strata serve: decode expert cache hit rate: 98.3% (97014 hits / 98736 lookups); 165 more read by the GPU over PCIe or from another GPU (0.2% of all 98901 routed)", "strata serve: asynchronous adaptive tier: 366 rounds, 7698 experts swapped in, 30.7 ms per round (start to flip, between windows) (arena, copy-in only, batch 0, wait 1, pick inline 0.10 ms/round)", "strata serve: suffix drafts: 1 windows, 2 of 3 drafts accepted"], "prompt_tokens": 116831, "prefill_tps": 154.9, "gen": 152, "decode_tps": 96.5, "acc": 83, "drafted": 138, "variant": "pr1_on", "size_k": 128, "rep": 1, "wall_s": 2.2}
+{"session": "ctx_pr1_on_64", "sha": "a41fac938ada325d", "notes": ["strata serve: decode expert cache hit rate: 98.2% (96554 hits / 98341 lookups); 167 more read by the GPU over PCIe or from another GPU (0.2% of all 98508 routed)", "strata serve: asynchronous adaptive tier: 440 rounds, 8265 experts swapped in, 30.5 ms per round (start to flip, between windows) (arena, copy-in only, batch 0, wait 1, pick inline 0.09 ms/round)"], "prompt_tokens": 116831, "prefill_tps": 152.4, "gen": 153, "decode_tps": 95.8, "acc": 81, "drafted": 135, "variant": "pr1_on", "size_k": 128, "rep": 2, "wall_s": 2.3}
+{"session": "ctx_v0142_64", "sha": "9f5c2ef4692fd88e", "notes": ["strata: K/V grown to 57344 cells (0.76 GiB); the expert cache gave 269 slots for it, 104 hotter experts moved to colder ones' slots (8020 of 8289 slots hold experts)", "strata serve: decode expert cache hit rate: 94.1% (81780 hits / 86934 lookups); 1231 more read by the GPU over PCIe or from another GPU (1.4% of all 88165 routed)"], "prompt_tokens": 55629, "prefill_tps": 2164.7, "gen": 140, "decode_tps": 85.2, "acc": 79, "drafted": 128, "variant": "v0142", "size_k": 64, "rep": 0, "wall_s": 27.9}
+{"session": "ctx_v0142_64", "sha": "5c0380723480c433", "notes": ["strata serve: decode expert cache hit rate: 97.9% (73074 hits / 74624 lookups); 111 more read by the GPU over PCIe or from another GPU (0.1% of all 74735 routed)"], "prompt_tokens": 55629, "prefill_tps": 170.4, "gen": 124, "decode_tps": 105.4, "acc": 68, "drafted": 103, "variant": "v0142", "size_k": 64, "rep": 1, "wall_s": 1.8}
+{"session": "ctx_v0142_64", "sha": "0f39f8b3760336fc", "notes": ["strata serve: decode expert cache hit rate: 98.2% (70239 hits / 71507 lookups); 93 more read by the GPU over PCIe or from another GPU (0.1% of all 71600 routed)", "strata serve: suffix drafts: 1 windows, 2 of 3 drafts accepted"], "prompt_tokens": 55629, "prefill_tps": 173.1, "gen": 115, "decode_tps": 104.7, "acc": 63, "drafted": 100, "variant": "v0142", "size_k": 64, "rep": 2, "wall_s": 1.7}
+{"session": "ctx_v0142_64", "sha": "3e26b7eecd267952", "notes": ["strata: K/V grown to 122880 cells (1.57 GiB); the expert cache gave 432 slots for it, 203 hotter experts moved to colder ones' slots (7588 of 8289 slots hold experts)", "strata serve: decode expert cache hit rate: 96.5% (52515 hits / 54398 lookups); 176 more read by the GPU over PCIe or from another GPU (0.3% of all 54574 routed)"], "prompt_tokens": 116834, "prefill_tps": 2087.5, "gen": 93, "decode_tps": 100.1, "acc": 53, "drafted": 77, "variant": "v0142", "size_k": 128, "rep": 0, "wall_s": 57.5}
+{"session": "ctx_v0142_64", "sha": "76d91bdeb91bb59a", "notes": ["strata serve: decode expert cache hit rate: 97.8% (72388 hits / 74042 lookups); 159 more read by the GPU over PCIe or from another GPU (0.2% of all 74201 routed)"], "prompt_tokens": 116834, "prefill_tps": 157.9, "gen": 127, "decode_tps": 101.6, "acc": 69, "drafted": 99, "variant": "v0142", "size_k": 128, "rep": 1, "wall_s": 1.9}
+{"session": "ctx_v0142_64", "sha": "14c0bb7dbe3686b3", "notes": ["strata serve: decode expert cache hit rate: 97.9% (108506 hits / 110827 lookups); 174 more read by the GPU over PCIe or from another GPU (0.2% of all 111001 routed)"], "prompt_tokens": 116834, "prefill_tps": 153.6, "gen": 166, "decode_tps": 88.1, "acc": 84, "drafted": 152, "variant": "v0142", "size_k": 128, "rep": 2, "wall_s": 2.6}