[Krypto-Whitehat]

Issue: Resolves #2264

Summary

--conversation-cache-park-gain N makes parking rate-limited by growth, opt-in and default 0 = park every turn, byte-identical to today. A conversation that advanced fewer than N tokens since the copy it would supersede keeps that copy: it still holds this conversation's prefix (a conversation only appends), and re-parking writes the whole snapshot only for the next turn to drop it again.

The report behind this measured 1,418 successful parks for one long single conversation, 2,251.9 GB of host copy written and 105.8 GB (4.7 %) ever read back, with 425 superseded copies dropped and 1,548 s of copy time on the request path. With the gate on it measured 3 park events and 3.26 GB (-66.7 %), the remaining copy being the conversation's first park.

The default stays off because the measurement is from one rig. Deciding whether it should be on by default is the maintainer's call, and nothing here forecloses it.

What changed

• include/strata/core/conversation_cache.hpp: one new const accessor, parked_prefix_tokens(ids, images, checkpoints, cvec). It reports, in tokens, the longest parked entry the outgoing state supersedes - exactly the entries drop_superseded() would remove, found the same way drop_superseded() finds them (deepest checkpoint the outgoing chain still holds), so the gate can never name a different entry than the one a park would drop.
• src/program/generate.cpp, the park decision (line 10445 at a0116b27): the existing condition gains one term, park_worth. With the default 0 that term is constantly true and the line behaves as before.
• src/program/generate.cpp: the option, the --conversation-cache-park-gain N parse (alongside the other --conversation-cache-* flags, same INT32_MAX bound as --conversation-cache-slots), and the help text.
• src/core/conversation_cache_test.cpp: seven checks for the accessor - an empty cache, parking a conversation, the entry the chain supersedes, another conversation sharing only the root, the other steering mode, an appended tail, and an entry without checkpoints.

Left alone on purpose: the default (0, so no behaviour change without the flag), the copy path itself, drop_superseded() and the K/V retention, the physical-RAM admission gate, and every log line. This is one flag, one accessor and one term in one condition.

Extra Notes

• Checked by build: cmake -DSTRATA_ENABLE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89 with nvcc 13.4.92 and MSVC 14.44. The strata target compiles and links with 0 errors (Windows CUDA, sm_89, so the Linux-only paths of this file are not what that compiles). conversation_cache_test builds and passes: 4,226 checks, exit 0.
• The unit test earned its keep: it failed the first assertion because I had expected the entry's live length (6) where the accessor reports the deepest checkpoint (5). drop_superseded() keys on the checkpoint, so 5 is the right answer and the expectation was wrong.
• Not tested here: HIP and SYCL (no toolchain on this machine), and no decode run on this box - there is no model and no matching rig. Every tok/s number in the issue comes from the reporter's own A/B on an RTX 5090 Laptop, not from a run of this patch. What is verified here is that the change compiles and that the accessor names exactly what a park would supersede.
• The gate is a policy decision with a real trade-off the maintainer should weigh: a conversation that rewrites (edits an earlier turn) is not a prefix of its parked copy, so with the gate on, such a conversation keeps an entry it cannot reuse until it grows past the threshold. That is the same loss the cache already accepts for drop_superseded, and it is why this ships opt-in.

[Niko1221]

Thank you, this is in for 0.1.45 as an opt-in with your authorship; I added the docs line. On our RTX 3060 with a 12.5K-token conversation whose client trims the reply each turn, turns 2-8 went from 6.06 s to 5.07 s with --conversation-cache-park-gain 500 (3 of 4 rounds faster), same output.

[Niko1221]

Thanks for the pull request.

Your change is already in main for 0.2.0-beta, with your credit, so there is nothing left for you to do here. We're closing it because we're resetting the tracker for 0.2.0-beta: there are more than 1,300 open issues and pull requests, and we're closing everything that isn't being worked on right now.

If part of your work did not make it in and you want it considered, open a new pull request on top of main that follows CONTRIBUTING.md, or comment /reopen after fixing the title area prefix and the template.