Repository navigation
Tracking: graph evidence — prism task-shaped ops on the Shale PR card #17
Description
Umbrella for integrating prism's task-shaped operations into the Shale card. Supersedes the scope sketch in RFC #5 with a measured foundation: the ops now exist, are released (prism ≥ v0.19.4 / grove ≥ v0.17.5), and carry a controlled study with independent compiler oracles behind them.
Thesis
Shale today renders testimony (agent intent + session evidence). Prism's deterministic ops upgrade the card to proof: "the graph confirms 38/38 required sites are touched" instead of "the agent says it updated the callers."
Measured basis (paper §5.5, 162-run compile experiment on jackson-databind): applying each agent answer as edits and compiling, every answer that missed a change-site failed to compile and every complete answer compiled. A missed site on this card is a pre-merge build-break catch — deterministic, reproducible, zero LLM cost.
Why this positions Shale
- No API key. Every LLM review bot needs a paid key and ships code to a vendor. This runs a pinned engine inside the user's CI: one workflow file, no secrets, code never leaves the runner.
- Deterministic where competitors are probabilistic. Same PR → same card, bit-for-bit (engine builds are hash-verified deterministic as of grove v0.17.x). An LLM reviewer's comment is an opinion; "3 sites missing, completeness: closed" is a gateable fact.
- The Codecov loop. The card on public PRs is the distribution channel — every adopting repo advertises to every contributor. No sales motion required.
- Citable evidence. The card's "why trust this" links to published benchmarks (https://provasign.dev/benchmarks/) and the study. No review bot has that.
Delivery order (each independently shippable)
- Graph evidence 1/8: detect change-impact triggers in the PR diff #8 — trigger detection in the PR diff (foundation)
- Graph evidence 2/8: CI plumbing — prism install + .grove cache in shale-action #9 — CI plumbing: pinned prism + .grove cache
- Graph evidence 3/8: card section — change-set completeness (change-impact) #10 — change-set completeness section (the wedge)
- Graph evidence 4/8: card section — test exposure (untested-surface) #11 — test-exposure section (untested-surface)
- Graph evidence 5/8: card section — contract check (missing-implementations) #12 — contract-check section (missing-implementations)
- Graph evidence 6/8: card section — strandings (dead-code delta) #13 — strandings section (dead-code delta)
- Graph evidence 7/8: rename verification (rename-plan) #14 — rename verification (rename-plan)
- Graph evidence 8/8: merge-gate mode (opt-in required check) #15 — merge-gate mode (opt-in, ships last)
- Graph evidence: language rollout + confidence framing (TS/Python phase 2) #16 — TS/Python rollout with confidence framing
Phase 1 = #8 + #9 + #10 on Go/Java. That alone is a launchable story: "agents write the code; Shale proves the change-set is complete."
Ground rules (carried from the study, non-negotiable)
- Relay discipline: engine output renders as-is; no LLM between engine and card (measured: re-processing drops real sites and adds spurious ones).
- Honesty surfaces: completeness field, caveats, and bucket distinctions (missing vs unverifiable vs ambiguous) always render;
project-localis never a green checkmark; TS renders advisory until its measured precision supports more. - No nagging: zero triggers → zero graph sections. The card earns attention by being right and silent otherwise.
Activity