Skip to content

Provenance tier labels on agent claims #404

Closed

Description

@currentbits

From the high-stakes verticals research (Harvey's source tiers): label agent statements by origin — repo file, shared-memory entry, GitHub issue, or MODEL PRIOR KNOWLEDGE. The first three are addressable via the memory server and worktree; the 'this came from the model's head, not your repo' tag is the valuable part. Attacks the 'fluent summary beats skim review' failure mode. Pairs with #239 (cited recaps).

Activity

  1. added
    enhancementNew feature or request
    and removed on Aug 17, 2026
  2. currentbits commented on Aug 18, 2026

    @currentbits
    OwnerAuthor

    Implemented in the worktree for thread b9432c68.

    What shipped

    • src/provenance.ts: pure per-message classifier. Each assistant message is labeled by origin using (a) the tool calls in its own turn and (b) citations in its text:
      • repo — Read/Grep/Glob/Edit/Write paths, git show/log/diff/blame, backticked file paths
      • memory — shared-memory MCP tool calls (memory_search/get/..., bare or mcp-prefixed)
      • issue — gh issue/pr commands, github.com issue URLs, #N refs
      • a substantive message (≥240 chars) with none of these gets the model prior knowledge tag — the 'this came from the model's head, not your repo' case. Short chatter is never tagged.
    • ThreadView: a ProvenanceStrip under each assistant message renders one chip per tier (full ref list on hover); prior tag is amber/dashed.
    • Tests: test/provenance.test.ts (12 cases: tiers, turn boundary, truncated-input fallback, chatter threshold) + render tests in test/threadView.test.tsx. Renderer suite 1172/1172 green, tsc --noEmit clean.

    Deliberate scope cuts: chips are labels, not click-throughs, and the repo chip asserts citation, not file-existence verification against the worktree. Both are follow-ups; the review itinerary in .solenta/review-itinerary.json lists the known false-positive (turn-scoped evidence tags a long final summary whose reads happened an earlier turn).

  3. added 2 commits that reference this issue on Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions