feat(pr-review): suppress findings the team keeps rejecting

[Makisuo]

Stacked on #1014 (stack: #1001 → #1014 → this).

What

The review now learns from how a team reacted to its earlier findings. Before a review is stored and posted, each new finding is compared with past findings people voted on. If it closely resembles several that were rejected, it is dropped.

• Embeddings. FindingEmbedder is a port in packages/backend. The model behind it is wired in apps/ai/src/platform/Llm.ts: OpenRouter's OpenAI-compatible /embeddings endpoint, openai/text-embedding-3-small by default, overridable with MAPLE_EMBEDDING_MODEL. The layer has its own OpenAiClient, so the Workers AI one is not replaced. Without an OpenRouter key there is no embedder and the filter is off.
• Storage. New table pr_review_finding_embeddings (finding_id, org, repository, model, real[]). It is kept separate from pr_review_findings so the lifecycle reads do not carry vectors. Vectors from different models are never compared.
• Labels. A dismissed finding counts as negative. Otherwise a finding with more 👎 than 👍 is negative, and one with more 👍 than 👎, or one a later push fixed, is positive. Anything else does not vote.
• Rule. A finding is suppressed when its cosine similarity is at least 0.5 to three or more negatives and to fewer than three positives. Ambiguous findings are posted. The 2,000 newest voting findings are compared in memory.
• Never suppressed: anything in the security category or with critical severity.
• Scope. The whole organization by default. The new repository setting feedbackScope (with a select in the settings dialog) narrows it to one repository or turns the filter off. Vectors are still stored while it is off, so turning it back on has history to work with.
• Failure. If there is no embedder, or the embedding call or the read of past votes fails, every finding is posted. The filter can only ever remove findings.
• Telemetry. maple.pr_review.suppressed on the submitReview span. maple.pr_review.feedback_filter (applied, off, no_embedder, embed_failed, read_failed), feedback_scope and feedback_examples are on the child span. Suppressed findings are also logged with their path and title.

Threshold

I measured it against the real endpoint with ten sample findings. The same kind of comment about different code scored 0.52 to 0.68, and different kinds scored at most 0.44. A 0.85 bar would never have matched. This is a small sample, so FEEDBACK_SIMILARITY is a single constant and should be re-checked against real reactions once they build up.

Migration

packages/db/drizzle/20260923221619_pr_review_finding_embeddings: one CREATE TABLE plus an index on (org_id, model). The prd deploy applies it.

Tests

• feedback.test.ts: labels, cosine similarity, the vote rule, security and critical exemptions.
• PrReviewService.test.ts (PGlite, fake embedder): a finding like three downvoted or dismissed ones is suppressed while security and critical ones are posted; enough positive votes keep a finding; repository scope ignores votes from other repositories; off posts everything; vectors from another model are ignored; an embedder failure or a missing embedder posts everything and stores no vectors.
• bun run --cwd packages/backend test src/services/pr-review passes (57 tests). The packages/db tests pass. tsc passes for backend, ai and web. oxlint is clean.
• I also ran layerFindingEmbedder against the live endpoint and got 1536-dimension vectors.

Not in this PR

• The span attribute is not asserted in tests.
• The settings dialog change has not been opened in a browser.
• Suppressed findings are left out of the summary comment entirely. Listing them in a collapsed section is an option if teams want to audit the filter.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

Summary by CodeRabbit

• New Features
  • PR reviews can learn from prior feedback and suppress similar findings that were dismissed or downvoted. Security and critical findings are always retained.
  • Choose whether feedback applies across the organization, only within a repository, or not at all.
  • When feedback filtering isn’t available, findings continue to post as usual.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: ee1c9f15-11e4-41d3-9c52-839b51947ee1

📥 Commits

Reviewing files that changed from the base of the PR and between 0b2b09d and 9c81821.

📒 Files selected for processing (13)

• apps/ai/src/chat/turn-runner.ts
• apps/ai/src/platform/Llm.ts
• apps/web/src/components/integrations/pr-review-settings.tsx
• docs/pr-review-agent-plan.md
• packages/backend/src/services/pr-review/FindingEmbedder.ts
• packages/backend/src/services/pr-review/PrReviewService.test.ts
• packages/backend/src/services/pr-review/PrReviewService.ts
• packages/backend/src/services/pr-review/feedback.test.ts
• packages/backend/src/services/pr-review/feedback.ts
• packages/db/drizzle/20260923221619_pr_review_finding_embeddings/migration.sql
• packages/db/drizzle/20260923221619_pr_review_finding_embeddings/snapshot.json
• packages/db/src/schema/vcs.ts
• packages/domain/src/http/pr-review.ts

 _________________________________________________________________________________________________________________________________
< For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled. - Richard Feynman >
 ---------------------------------------------------------------------------------------------------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}