Skip to content

[Engine] test: the byte-identity half of the reuse contract - #3973

Open
wannagoxuf wants to merge 1 commit into
Niko1221:mainfrom
wannagoxuf:feature/reuse-identity-test
Open

wannagoxuf wants to merge 1 commit into
Niko1221:mainfrom
wannagoxuf:feature/reuse-identity-test

Conversation

@wannagoxuf

Copy link
Copy Markdown
Contributor

Problem

The snapshot API contract (include/strata/core/conversation_snapshot.hpp) says:

A nonzero unchanged_tokens is valid only for storage retained from an image actually restored into this
session, bounded by every subsequent rewrite. Equal token IDs alone do not establish that its K/V bytes
are unchanged.

The bookkeeping half of that - which bytes a capture claims are unchanged, and where the claim stops - is
covered in conversation_cache_test.cpp (retain / limit_reuse / take_reuse). The half that is not
covered is the byte half: whether the bytes really are unchanged.

That question is live in the two pull requests that touch the reuse path (#3711 and the harvest discussed in
#3321), and it is decidable in the shapes the cache actually produces.

Approach

A new test (src/core/conversation_reuse_identity_test.cpp, 21 checks) that covers:

  • a tail rewrite leaves the prefix bytes alone, and changes the tail bytes. This is what the
    unchanged_tokens claim rests on: the bytes before the first rewrite are still the ones the session holds.
  • a different prefix gives different bytes - the negative control. Without it the first case would prove
    nothing, since identical bytes could just mean the comparison is too coarse to see a difference.
  • the claim's bound: limit_reuse cuts it at the first rewrite, a rewrite from the start drops the whole
    donation, and a later continuation cannot widen a claim that was already cut.
  • the claim survives the capture: with a nonzero unchanged_tokens the capture keeps the donated prefix
    bytes and not the device's, and past the claimed prefix it takes the device's bytes.

The engine-level half - two independent computations of the same token prefix through different chunkings -
needs a real model and is not decided here. It is measured with STRATA_STATE_HASH=1 (the engine's
per-request fingerprint of the session's K/V) and the result is in #3321: on 4x4090 with
Qwen3.8-Flash-Next IQ3_XXS, the same prompt through --prefill 4096 and --prefill 1024 gives different
main-K/V fingerprints (2b0d5053ac251873 vs 42e1a97e3930a0fe), while the same chunking twice gives the
same fingerprint. So equal token IDs do not, in fact, establish equal bytes across a chunking boundary - the
contract's caution is observable, not theoretical.

Validation

Passed:

  • Build on origin/main (7dd5cf1): cmake -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_CUDA=ON -DSTRATA_BUILD_CONVERSATION_TESTS=ON + cmake --build . -j 4 (CUDA 12.8, driver 580.178.04, 4x RTX 4090).
  • ctest -R conversation --output-on-failure -j 2: 8/8 passed, including the new
    conversation_reuse_identity_test (21 checks).
  • No production code is touched: the diff is one new test file and its add_test registration.

Not run:

  • The HIP and SYCL backends are not built here (no toolchain), so the test is not exercised on them. The
    test uses only conversation_kv_save / conversation_kv_verify / ConversationCache, none of which are
    backend-specific, but that is an argument, not a run.
  • The engine-level identity measurement (the STRATA_STATE_HASH=1 runs) is on the CUDA engine only.

Failed: nothing.

Limits

  • This test decides the bookkeeping and the byte-level properties of the shapes the cache produces. It does
    not decide whether two independent computations of the same prefix agree - that needs a model, and the
    answer there is no (see the measurement above).
  • The fixture writes the K/V bytes itself, so it can show that a rewrite leaves a prefix alone and that a
    different prefix differs; it cannot show that a real model's prefix bytes are stable across chunkings.

Related

The snapshot API contract says a nonzero unchanged_tokens is valid only for storage
retained from an image actually restored into this session, bounded by every
subsequent rewrite, and that "equal token IDs alone do not establish that its K/V
bytes are unchanged".

The bookkeeping half of that (which bytes are claimed unchanged, and where the claim
stops) is covered in conversation_cache_test.cpp.  This adds the half that was not:
whether the bytes really are unchanged.

- a tail rewrite leaves the prefix bytes alone, and changes the tail bytes
- a different prefix gives different bytes (the negative control: without it the
  first case would prove nothing)
- the claim stops at the first rewrite, and a rewrite from the start drops the
  donation entirely
- a capture with a nonzero unchanged prefix keeps the donated bytes, not the
  device's, and past the claimed prefix it takes the device's bytes

The engine-level half of the question - two independent computations of the same
token prefix through different chunkings - needs a real model and is measured with
STRATA_STATE_HASH=1, not here.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant