Repository navigation
[Engine] test: the byte-identity half of the reuse contract - #3973
Open
wannagoxuf wants to merge 1 commit into
Open
wannagoxuf wants to merge 1 commit into
wannagoxuf wants to merge 1 commit into
Conversation
The snapshot API contract says a nonzero unchanged_tokens is valid only for storage retained from an image actually restored into this session, bounded by every subsequent rewrite, and that "equal token IDs alone do not establish that its K/V bytes are unchanged". The bookkeeping half of that (which bytes are claimed unchanged, and where the claim stops) is covered in conversation_cache_test.cpp. This adds the half that was not: whether the bytes really are unchanged. - a tail rewrite leaves the prefix bytes alone, and changes the tail bytes - a different prefix gives different bytes (the negative control: without it the first case would prove nothing) - the claim stops at the first rewrite, and a rewrite from the start drops the donation entirely - a capture with a nonzero unchanged prefix keeps the donated bytes, not the device's, and past the claimed prefix it takes the device's bytes The engine-level half of the question - two independent computations of the same token prefix through different chunkings - needs a real model and is measured with STRATA_STATE_HASH=1, not here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The snapshot API contract (
include/strata/core/conversation_snapshot.hpp) says:The bookkeeping half of that - which bytes a capture claims are unchanged, and where the claim stops - is
covered in
conversation_cache_test.cpp(retain/limit_reuse/take_reuse). The half that is notcovered is the byte half: whether the bytes really are unchanged.
That question is live in the two pull requests that touch the reuse path (#3711 and the harvest discussed in
#3321), and it is decidable in the shapes the cache actually produces.
Approach
A new test (
src/core/conversation_reuse_identity_test.cpp, 21 checks) that covers:unchanged_tokensclaim rests on: the bytes before the first rewrite are still the ones the session holds.nothing, since identical bytes could just mean the comparison is too coarse to see a difference.
limit_reusecuts it at the first rewrite, a rewrite from the start drops the wholedonation, and a later continuation cannot widen a claim that was already cut.
unchanged_tokensthe capture keeps the donated prefixbytes and not the device's, and past the claimed prefix it takes the device's bytes.
The engine-level half - two independent computations of the same token prefix through different chunkings -
needs a real model and is not decided here. It is measured with
STRATA_STATE_HASH=1(the engine'sper-request fingerprint of the session's K/V) and the result is in #3321: on 4x4090 with
Qwen3.8-Flash-Next IQ3_XXS, the same prompt through
--prefill 4096and--prefill 1024gives differentmain-K/V fingerprints (
2b0d5053ac251873vs42e1a97e3930a0fe), while the same chunking twice gives thesame fingerprint. So equal token IDs do not, in fact, establish equal bytes across a chunking boundary - the
contract's caution is observable, not theoretical.
Validation
Passed:
origin/main(7dd5cf1):cmake -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_CUDA=ON -DSTRATA_BUILD_CONVERSATION_TESTS=ON+cmake --build . -j 4(CUDA 12.8, driver 580.178.04, 4x RTX 4090).ctest -R conversation --output-on-failure -j 2: 8/8 passed, including the newconversation_reuse_identity_test(21 checks).add_testregistration.Not run:
test uses only
conversation_kv_save/conversation_kv_verify/ConversationCache, none of which arebackend-specific, but that is an argument, not a run.
STRATA_STATE_HASH=1runs) is on the CUDA engine only.Failed: nothing.
Limits
not decide whether two independent computations of the same prefix agree - that needs a model, and the
answer there is no (see the measurement above).
different prefix differs; it cannot show that a real model's prefix bytes are stable across chunkings.
Related