Skip to content

Cross-model review: agent PRs reviewed by a different provider than the author #259

Closed

Description

@currentbits

Inspired by Greptile's Model Inversion (models catch more bugs in other models' code) and their v5 swarm review (parallel single-hypothesis reviewers, 52% -> 66% addressed-comment rate).

Add an optional review stage to the PR flow: a thread's PR gets reviewed by a different provider than the one that wrote it (claude authors, codex reviews), optionally fanned out as parallel single-hypothesis review passes using the existing best-of-N infrastructure. Review findings route back through #161.

Activity

  1. currentbits commented on Aug 17, 2026

    @currentbits
    OwnerAuthor

    Research note (Cognition's 'Multi-Agents: What's Actually Working', Apr 2026, measured on Devin Review): the reviewer should share NO context with the coder. Give the reviewing agent the diff only; context rot makes the coder blind to its own bugs, and a reviewer anchored on the same spec misses the same things. Second finding: the coder must be the filter on review findings to prevent fix-loops. So: clean-context review thread type (fork zero context), findings routed back to the author thread for filtering.

  2. currentbits commented on Aug 18, 2026

    @currentbits
    OwnerAuthor

    Research note (Anthropic multi-agent patterns): the epistemics experiment validates cross-checking — newer models detected lies only because scout reports OVERLAPPED, letting contradictions surface. Cross-provider review works the same way: overlap is what makes contradictions visible.

  3. currentbits commented on Aug 19, 2026

    @currentbits
    OwnerAuthor

    The steal from Claude Code Desktop's self-review is the rubric, not the button.

    In the diff view there is a Review code action, and the docs state its scope explicitly (docs):

    "The review focuses on high-signal issues: compile errors, definite logic errors, security vulnerabilities, and obvious bugs. It does not flag style, formatting, pre-existing issues, or anything a linter would catch."

    That negative list is the whole feature. An unconstrained "review this diff" prompt returns twelve nits and one real bug, and the user learns to skip the panel — the failure mode that kills every review surface. Whatever provider Solenta routes the cross-model review to, the prompt needs those four exclusions in it verbatim, and the results need to land as comments in the diff rather than as a prose turn (which is how #162's comment thread becomes bidirectional: agent comments and user comments in the same gutter).

    Codex frames the same idea as severity tiers instead of exclusions — it surfaces P0/P1 by default with the threshold configurable via review guidelines in AGENTS.md (26.415 writeup). Repo-configurable review scope pairs well with #378 (repo-versioned agent definitions).

  4. added and removed on Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions