Repository navigation
[Benchmark] With Deepseek Harness Prefix cache never hits: Reused = 0 on 12/12 requests, long-context prefill consumes 73% of wall time #3493
Description
this is log.
| Time | Status | Prompt | Reused | Output | Prefill t/s | Decode t/s | VRAM hit rate | Duration |
|---|---|---|---|---|---|---|---|---|
| 02:20:28 | Done | 35,963 | 0 | 80 | 1,491 | 41.4 | 55.8% +10.8% PCIe | 26.1 s |
| 02:19:11 | Done | 100,582 | 0 | 459 | 1,471 | 53.1 | 69.1% +7.9% PCIe | 77.0 s |
| 02:18:41 | Done | 35,516 | 0 | 265 | 1,474 | 45.6 | 66.7% +8.2% PCIe | 29.9 s |
| 02:16:58 | Done | 98,206 | 0 | 2,316 | 1,542 | 58.6 | 82.9% +4.0% PCIe | 103.2 s |
| 02:16:21 | Done | 35,341 | 0 | 664 | 1,470 | 60.1 | 81.2% +4.3% PCIe | 35.1 s |
| 02:15:09 | Done | 97,673 | 0 | 460 | 1,530 | 55.6 | 71.5% +7.4% PCIe | 72.2 s |
| 02:14:33 | Done | 34,697 | 0 | 482 | 1,446 | 54.2 | 75.4% +5.8% PCIe | 32.9 s |
| 02:13:02 | Done | 95,967 | 0 | 1,538 | 1,519 | 55.9 | 79.4% +5.0% PCIe | 90.8 s |
| 02:11:03 | Done | 34,404 | 0 | 3,000 | 1,435 | 66.5 | 84.7% +3.2% PCIe | 69.1 s |
| 02:09:54 | Done | 94,214 | 0 | 257 | 1,486 | 52.4 | 62.9% +10.2% PCIe | 68.3 s |
| 02:09:29 | Done | 33,545 | 0 | 85 | 1,458 | 38.8 | 53.8% +12.0% PCIe | 25.2 s |
| 02:07:50 | Done | 92,123 | 0 | 2,029 | 1,458 | 57.4 | 81.3% +4.3% PCIe | 98.5 s |
Activity
github-actions commentedon Oct 20, 2026
Thanks for the issue. To keep the tracker readable, every item needs an area prefix in the title and the parts of the issue form filled in. This one is missing:
Edit the issue (title and description) and this note goes away on its own. If it is still missing parts in 5 days it will be closed, which is not a rejection: fix it and comment
/reopen.Details: CONTRIBUTING.md
[-][Bug] With Deepseek Harness Prefix cache never hits: Reused = 0 on 12/12 requests, long-context prefill consumes 73% of wall time[/-][+][Benchmark] With Deepseek Harness Prefix cache never hits: Reused = 0 on 12/12 requests, long-context prefill consumes 73% of wall time[/+]github-actions commentedon Oct 20, 2026
Thanks, this follows the format now.
shiftonetothree commentedon Oct 20, 2026
--conversation-cache-mib 8192 --conversation-cache-slots 4solve problenm, but how can i save this config.Niko1221 commentedon Oct 22, 2026
Thanks, and good to hear
--conversation-cache-mib 8192 --conversation-cache-slots 4fixed it. To keep them, add the two flags to the model's start options: in the app under the model's Expert settings, or in the args list of the model's config file in the Terminal. We're also changing the defaults so the cache is sized for agent use by itself, so this won't be needed in a future release.serve: the conversation cache is sized from the free RAM when --conve…
tests: setup no longer tips about the conversation cache (the engine …
Merge f-023-tune: Work profile CPU workers ceil(0.8 x physical cores)…
Merge upstream v0.2.3 into memory-core: the fork rides 0.2.3 (engine …
serve: --conversation-cache-mib がなければ、モデルを載せたあとの空き RAM から会話キャッシュの大きさを…
Niko1221 commentedon Oct 24, 2026
This is fixed in Strata 0.2.3. Please update to the latest release (in the Terminal:
strata update; in the app: download the newest installer from the releases page). If it still happens on the latest version, reply here and we will reopen it. Thank you for the report.Merge upstream/main (v0.2.4.2) into cache-arena
Results 11: upstream v0.2.7 merged, measured against the old main; th…
serve: disk-backed conversation cache in generate.cpp (1aa35f0, adapted)
serve: --conversation-cache-mib を省くと、会話のキャッシュを RAM の空きから決める（upstream f…
poly: --conversation-cache-mib auto, sized from the free RAM