Skip to content

fix(server): skip CodeRabbit review acknowledgement wakes - #32300

Open
voltcrash wants to merge 5 commits into
pingdotgg:mainfrom
voltcrash:t3/ignore-review-progress-ack
Open

voltcrash wants to merge 5 commits into
pingdotgg:mainfrom
voltcrash:t3/ignore-review-progress-ack

Conversation

@voltcrash

Copy link
Copy Markdown
Contributor

Problem

CodeRabbit's reply to a review request says “Review triggered.” before editing the same comment with the result. PR watches currently wake the agent for both, although the acknowledgement has nothing to act on. Fixes #32275.

Change

Skip only coderabbitai[bot] replies whose first 1,000 characters contain CodeRabbit's reply marker and an “Action(s) performed” section starting with “Review triggered.” or “Full review triggered.” Still advance the watermark, so progress is consumed without spending a comment wake. Outcome edits, other replies, and other authors keep their existing behavior.

Scope and approval

This qualifies for CONTRIBUTING.md's very small, focused obvious-bug exception: the existing watch reports a queued-review acknowledgement as actionable feedback. The fix only filters that acknowledgement in the server evaluator; it adds no settings, contracts, client changes, or provider-specific behavior. The linked issue has not been maintainer-triaged. No changes to edit debouncing, wake labels, snippet rendering, or check recovery.

Verification

  • The two focused CodeRabbit regressions fail on upstream main because the acknowledgement appears in remarks. With the fix, all 23 tests in vp test run apps/server/src/orchestration-v2/pullRequestWatch.test.ts pass. Coverage includes both acknowledgement variants, finished/rate-limited outcome edits delivered once, mixed comments, same-time arrivals, other authors, other replies, and the 1,000-character bound.
  • vp run check:changed passed: format, lint, both unused-code scans, generated GraphQL, three affected package typechecks, and 25 tests across the evaluator and agent-context budget files.
  • Replayed the actual CodeRabbit reply's edit history through the evaluator using Node, without starting a server:
                         upstream main      fixed
Review triggered.        remarks, wakes=1   no change, wakes=0
same acknowledgement     no change          no change
Review finished. edit    remarks, wakes=2   remarks, wakes=1
same outcome             no change          no change

Both versions advance the watermark to 00:24:57Z for the acknowledgement and 00:29:47Z for the outcome on 2026-11-07. The reactor already persists watermark-only changes.

Limitations: no live remote watch or paid provider turn was started. The filter intentionally depends on CodeRabbit's observed reply format; unrecognised formats retain the existing wake behavior. Wake wording is unchanged.

Reviewed with review-loop: 1 round with GPT-6.1 Sol on xhigh. Reviewer verdict: approved.

Created with GPT-6.1 Sol (xhigh) in Codex through T3 Code.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:S 10-29 changed lines (additions + deletions). labels Nov 7, 2026
@macroscopeapp

macroscopeapp Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 3c5e9d5

Macroscope's review found this PR approvable — PR watching skips CodeRabbit’s review-trigger acknowledgements while continuing to track comments and report other events. The narrowly targeted filter reduces unnecessary agent wakes without introducing an obvious material risk.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • Resume reviews
  • Trigger review

No actionable comments were generated in the recent review.

Walkthrough

The PR watch evaluator excludes qualifying CodeRabbit review acknowledgements from reported remarks while retaining fresh-remark tracking. Tests cover watermark advancement, later outcome edits, and other comments that remain actionable.

Changes: CodeRabbit review acknowledgements

Layer / File(s) Summary
Detect and filter acknowledgements
apps/server/src/orchestration-v2/pullRequestWatch.ts
The evaluator recognizes matching CodeRabbit bot comments and excludes fresh acknowledgements from reported remarks. Fresh comments still determine cursor tracking.
Test acknowledgement and outcome handling
apps/server/src/orchestration-v2/pullRequestWatch.test.ts
Tests cover watermark advancement without a wake, later outcome edits, comments with the same timestamp, and replies that remain actionable.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Low

Possibly related issues

Suggested reviewers: saphid

Merge Risk: Minimal · up to 769d3

No merge-blocking risk was established for the acknowledgement filter.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: preventing CodeRabbit review acknowledgement comments from waking the server watch.
Description check Passed The description includes complete Problem, Change, Scope and approval, and Verification sections. It explains the expected behavior, implementation scope, focused tests, broader checks, replay results…
Linked Issues check Passed Issue #32275 is open and directly linked. evaluatePullRequestWatch filters only coderabbitai review acknowledgements that match the specified CodeRabbit format within the first 1,000 characters. T…
Out of Scope Changes check Passed The changes are limited to the pull-request watch evaluator and focused regression tests. The predicate and tests directly support issue #32275. No unrelated settings, contracts, client changes, provi…
Full details: Description check

Explanation

The description includes complete Problem, Change, Scope and approval, and Verification sections. It explains the expected behavior, implementation scope, focused tests, broader checks, replay results, and limitations.

Full details: Linked Issues check

Explanation

Issue #32275 is open and directly linked. evaluatePullRequestWatch filters only coderabbitai review acknowledgements that match the specified CodeRabbit format within the first 1,000 characters. The evaluator still advances the watermark. Outcome edits, other replies, other authors, and non-matching formats remain actionable. Focused tests cover both trigger variants, outcome delivery, bounds, and non-qualifying comments.

Full details: Out of Scope Changes check

Explanation

The changes are limited to the pull-request watch evaluator and focused regression tests. The predicate and tests directly support issue #32275. No unrelated settings, contracts, client changes, provider behavior, or other watch behavior changes are shown.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Usage-based review receipt

Note

This review was completed with usage-based billing: files reviewed beyond your plan's included limits are billed at $0.25/file. View usage-based billing.

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 7ec6c15 and 769d347.

Files selected for processing: 2

  • apps/server/src/orchestration-v2/pullRequestWatch.test.ts
  • apps/server/src/orchestration-v2/pullRequestWatch.ts

Limit details: You’ve used all 10 included reviews currently available.

@juliusmarminge juliusmarminge added bugfix Fix PR opened by an issue lead; enters the PR loop pr:gate-failed A land gate failed labels Nov 7, 2026
@voltcrash
voltcrash force-pushed the t3/ignore-review-progress-ack branch from 3c5e9d5 to 6db649a Compare November 7, 2026 12:37
AI-assisted; verified with the tests and tools listed above.
AI-assisted; verified with the tests and tools listed above.
GitHub's GraphQL API and GitLab name the bot "coderabbitai".

AI-assisted; verified with the tests and tools listed above.

Copy link
Copy Markdown
Member

Evidence

Real Chromium web client, 1280×800, light theme; real server and replay agent, fresh isolated state per commit. GitHub API responses are synthetic; no live forge writes. Normal two-minute watch sweeps. The initial check-success wake precedes the reproduction on both sides.

Create a synthetic project and replay thread, enable its pull-request watch, and let the startup check-success wake finish. Supply a matching coderabbitai “Review triggered.” acknowledgement, wait for a normal watch sweep, and capture the thread. Edit the same comment to “Review complete. No actionable findings.”, sweep and capture again. Leave the comment unchanged through another observed sweep and capture the final state. Run the identical sequence on both commits.

Scenario: CodeRabbit acknowledgement stays idle; edited outcome wakes once

Proven: the scenario fails on base b3fdf2b10f and passes on head 769d3470ca.

Base b3fdf2b10f Head 769d3470ca
Result ❌ failed
ASSERTION_FAILED: expected {"ackWakes":1,"ackTurns":1,"outcomeWakes":2,"outcomeTurns":2,"unchangedWakes":2,"unchangedTurns":2,"outcomeDelivered":1,"unchangedDelivered":1} to equal {"ackWakes":0,"ackTurns":0,"outcomeWakes":1,"outcomeTurns":1,"unchangedWakes":1,"unchangedTurns":1,"outcomeDelivered":1,"unchangedDelivered":1}
✅ passed
watch started Base b3fdf2b10f: watch started Head 769d3470ca: watch started
acknowledgement Base b3fdf2b10f: acknowledgement Head 769d3470ca: acknowledgement
finished outcome Base b3fdf2b10f: finished outcome Head 769d3470ca: finished outcome
unchanged sweep Base b3fdf2b10f: unchanged sweep Head 769d3470ca: unchanged sweep
failure Base b3fdf2b10f: failure
Before recording After recording
Full sequence Full sequence

Scenario, Synthetic forge driver, Runner config patch, Run instructions, Before runtime transcript, After runtime transcript.

AI-assisted; verified with the tests and tools listed above.

Not shown:

  • A live @coderabbitai review request and the CodeRabbit service were not exercised: forge writes were excluded from this task, so matching acknowledgement and edit responses were supplied by a synthetic GitHub endpoint.
  • The rate-limited outcome, Full review triggered wording, and [bot]-suffixed author variant were not captured; this proof covers coderabbitai, Review triggered., and a finished outcome.

@coderabbitai

coderabbitai Bot commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

@juliusmarminge Thanks for the evidence. The reported head run shows no wake for the acknowledgement, one wake for the outcome edit, and no additional wake on the unchanged sweep.

The run used a synthetic GitHub endpoint. It did not exercise a live CodeRabbit request or capture the other variants you listed.


Learnings used

Learnt from: eimexdev
Repo: pingdotgg/t3code PR: 15427
File: apps/server/src/orchestration-v2/PullRequestWatchReactor.ts:136-139
Timestamp: 2026-10-04T02:34:08.441Z
Learning: In the pull-request activity producers under apps/server/src/pullRequest, initial review-thread comments already appear in PullRequestActivity.comments. GitHub merges reviewThreadConversation output; Bitbucket builds both lists from the same kept comment entries; GitLab notes include discussion comments; Forgejo builds both lists from its inline records; Azure returns no reviewThreads. In apps/server/src/orchestration-v2/PullRequestWatchReactor.ts, readRemarks can therefore seed its comment map from activity.comments and add continuation-page comments without separately merging initial thread.comments.

You are interacting with an AI system.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Fix PR opened by an issue lead; enters the PR loop pr:gate-failed A land gate failed size:S 10-29 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: PR watch wakes the agent for CodeRabbit's "Review triggered" acknowledgement, which has nothing to act on

2 participants