Repository navigation
fix(code-review): key coverage-critic cache on the diff too - #193
Open
mikeangstadt wants to merge 11 commits into
Open
mikeangstadt wants to merge 11 commits into
mikeangstadt wants to merge 11 commits into
Conversation
- ISS-8961: the signals/ namespace key was (diff_tip, taxonomy_hash, prompt_hash), and diff_tip is a ref name -- literally "HEAD" for a local branch review -- so no component varied with the diff. The cache dir is ~/.claude/cr-cache-global-repo-<worktree basename>, so two reviews from one pooled directory shared directory and key, and the second was served the first's extracted signals. - Add signal_input_hash: SHA-256 over the canonical serialization of the agent input bundle extract-signals-prepare builds, which is exactly what the extraction agent reads. Key is now (diff_tip, input_hash, taxonomy_hash, prompt_hash). - Build the bundle before computing the key rather than only on a miss. - A bundle with no changed files has no diff identity: emit an empty cache_key plus cache_bypass_reason "no-changed-files", which suppresses the read and (via consolidate's empty-key guard) the write. - Correct the --diff-tip help, which claimed the flag was a SHA. Testing: pytest plugins/ (3 pre-existing failures unrelated to this change: TestOverrideCache/TestPR114ReviewFixes fail on clean main because their override fixture is dated 2026-05-29 and the overrides TTL is 90 days). Counterfactual: reverting code_review_helpers.py turns TestPooledWorktreeSignalCacheIsolation red with lane 2's manifest reading status "cache_hit". ruff check; pyright. Risks: cached extractions written under the old key scheme are misses after this lands, so the first review in each cache directory re-runs one Haiku extraction.
- Close a false green in the pooled-worktree regression test: it varied the changed PATH as well as the content, so a key hashing only the file list still passed. Add a lane pair holding path, status, and line counts identical and varying only the excerpt -- the shape two reviews of one branch actually take as it is amended. - Delegate signal_input_hash to _stable_json_hash, the module's existing owner of the deterministic cache-key JSON rule, instead of a byte-identical second copy of it. - Pin the bundle's run-to-run stability: a run-varying field added to _build_signal_input would make every review a permanent miss, and the golden harness cannot catch it because it normalizes cache_key. - Emit input_hash in the manifest beside taxonomy_hash and prompt_hash, so the one component that actually varies is visible when diagnosing a key move. Normalized in the golden harness like its siblings. - State the real reason the no-changed-files bundle bypasses the cache. It is not that such a bundle cannot be keyed -- it is the shape a degraded parse-diff takes, and caching it would persist a degraded extraction for the namespace TTL. Name the cost too: a genuinely empty review now re-dispatches its extraction every run. Testing: uv run pytest plugins/ -- 2113 passed, 3 pre-existing failures (override-TTL fixture dated 2026-05-29 vs a 90-day TTL, red on main). ruff check; uv run pyright 0 errors. Counterfactual: restoring origin/main's helper turns 13 tests red, including both pooled-lane tests. Mutation: narrowing the key to the file list alone is now killed by test_same_file_different_content_is_not_served_a_stale_hit. Risks: manifest gains an additive input_hash field; no consumer reads it, and the golden harness normalizes it.
Mechanics repair, not an expectation change. The behaviour under test is unchanged and still asserted: a valid, within-TTL override short-circuits verification, routes through consolidate as RE_ASSERTED, and lands in the by-reviewer stats. No assertion was weakened, widened, skipped, or deleted. Three tests hardcoded asserted_at "2026-05-29T22:00:00+00:00" while _override_is_expired sweeps the overrides namespace against the real wall clock with a 90-day TTL. The fixture was therefore a time bomb: it went red on 2026-08-27 with no code change, and the suite has been red on main ever since (main last ran CI 2026-08-17, green). A test that fails on a calendar date rather than a behaviour change was never pinning what it claimed to. Both sides of the comparison are now literals -- the fixture timestamp and the clock -- so the gap stays a constant 3 days forever. The TTL gate itself is untouched, and the tests whose subject IS the TTL keep offsetting from the real clock on purpose. Testing: uv run pytest plugins/ -- 2116 passed, 0 failed (first green run since 2026-08-27). Mutation: forcing _override_is_expired to return True turns all three repaired tests red, so the pin did not neuter the gate they depend on. ruff check; uv run pyright 0 errors. Risks: none identified. Test-only change.
ISS-9674, the coverage-critic twin of ISS-8961. - coverage_critic_cache_key was (coverage_plan_initial_hash, signals_hash, diff_tip, prompt_hash, available_reviewers_hash). _build_coverage_critic_input returns (main_input, diff_summary) where diff_summary is literally _build_signal_input(diff_data, None) -- the bundle the critic reads -- and it was in no key component. diff_tip is a ref name, never a commit id, so nothing varied with the diff. - It did not misfire only because signals_hash hashes extract_signals.json, which carries a wall-clock generated_at, so the key changed every run and the namespace never hit. That made the obvious optimization -- strip generated_at so the cache finally hits -- a change that would serve one review's coverage plan to another review from the same pooled worktree, with no test failing. - Add diff_summary_hash to the key; build the input before the key rather than after; report the hash in the manifest beside its siblings. - Correct the docstring's false "All five are content-addressed" claim and the SCHEMA.md row (wrong path shape, 3 of 5 components listed). - Reseed the cache-hit test from the key prepare reports instead of re-deriving it through the builder under test. Testing: uv run pytest plugins/ -- 2119 passed, 0 failed. Counterfactual: dropping diff_summary_hash from the payload turns 3 tests red, including test_pooled_lane_with_a_different_diff_misses_prior_cache, which holds plan, signals, prompt, roster and diff tip constant and varies only the diff. ruff check; uv run pyright 0 errors. Risks: one-time invalidation of the coverage_critic namespace, which in practice never hit anyway.
Collaborator
|
Beginning a code-review. Please allow time for me to finish before merging the PR |
shafty023
reviewed
Sep 7, 2026
shafty023
reviewed
Sep 7, 2026
Collaborator
|
Beginning a code-review. Please allow time for me to finish before merging the PR |
shafty023
reviewed
Oct 19, 2026
main's #321 already keys the signals cache on the agent input bundle and drops diff_tip from the key; this branch keeps that key shape and retains its additive pieces (input_hash in the manifest, the no-changed-files cache bypass). The override-clock pin is dropped in favour of main's _iso_days_ago fixtures.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…674-coverage-critic-cache-key main's #321 already keys the coverage_critic cache on the diff summary (without diff_tip), passes the summary to the critic via extra_inputs, and seeds a prior entry in its changed-diff regression. This branch keeps that shape and retains diff_summary_hash in the manifest.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mikeangstadt
added a commit
that referenced
this pull request
Nov 3, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Review request: Lint, Type Check and Tests pass on the head, there are 0 unresolved threads, and the branch merges cleanly with |
…-critic-cache-key # Conflicts: # CHANGELOG.md
This was referenced Nov 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #192 (base is
fix/iss-8961-review-signal-cache-key, notmain). Retarget tomainafter #192 merges. ISS-9674.Summary
coverage_critic/namespace, and it was safe only by accident.coverage_critic_cache_keywas(coverage_plan_initial_hash, signals_hash, diff_tip, prompt_hash, available_reviewers_hash)._build_coverage_critic_inputreturns(main_input, diff_summary)wherediff_summaryis literally_build_signal_input(diff_data, intent_summary=None)— the bundle the critic reads — and it appeared in no key component. As in fix(code-review): key signal cache on the diff, not the worktree #192,diff_tipis a ref name and never a commit id, so nothing in the tuple varied with the diff, while the cache directory is the pooled worktree's basename.signals_hashhashesextract_signals.json, which carries a wall-clockgenerated_at. The key therefore changed every run and the cache never hit. Anyone looking at a namespace with a 0% hit rate would find that timestamp, remove it, measure a real speedup, and ship a critic that serves one review's coverage plan to a different review — with no test failing. That is the whole reason this is a separate, filed ticket rather than a footnote.diff_summary_hash(signal_input_hashover that bundle), andcmd_coverage_critic_preparebuilds the agent input before computing the key rather than after. The manifest reportsdiff_summary_hashbeside its sibling component hashes.diff_tipis not), and theSCHEMA.mdcoverage-critic row, which documented the path as<diff_tip>.jsonwhen the code writes<key>.jsonand listed three of the five components. These are the two doc errors fix(code-review): key signal cache on the diff, not the worktree #192 deliberately left on the adjacent table row to keep that PR to one subject.Breaking changes
None — additive. One-time invalidation of the
coverage_critic/namespace, which in practice never hit.coverage_critic_cache_key()gains a positional argument; it has no callers outside this module and its tests.UI Feature Flag
[ui-flag:N/A]— tooling-only change in a Python helper; no UI surface in this repo.Test plan
test_pooled_lane_with_a_different_diff_misses_prior_cacheis the regression test: one shared cache directory, and plan_initial, signals, prompt, roster, anddiff_tipall held constant so that only the diff varies — which is precisely what the old key could not see.diff_summary_hashback out of the key payload turns 3 tests red, including that one. Restored: green.test_cache_hit_serves_directlyno longer re-derives the key to seed its fixture; it prepares once and seeds under the key the manifest reports — the false-green shape caught in fix(code-review): key signal cache on the diff, not the worktree #192's review.diff_summary_hashis normalized in the golden harness like its sibling hashes: it is fixture-coupled rather than wall-clock, so pinning it in goldens would force a hand-updated digest on every fixture diff edit. Composition is pinned by the unit test instead.ruff checkclean;uv run pyright0 errors;uv run pytest plugins/2119 passed, 0 failed.