Repository navigation
feat(skills): retarget the review loop from Copilot to CodeRabbit - #537
Conversation
The repository adopted CodeRabbit's request-changes workflow as its review gate (96b841c), and Copilot reviews are dropping out of the loop. A baseline dry-run against the old skill confirmed the failure this closes: an agent following it addressed only Copilot threads, reported a clean cycle while a CHANGES_REQUESTED CodeRabbit review sat at head, and had no merge guidance at all. The skill becomes coderabbit-review-loop. Actionability is defined as unresolved non-outdated coderabbitai threads with other reviewers explicitly noted-not-resolved; re-review triggers by PR comment since the requested-reviewers API does not apply; the clean-cycle gate requires an APPROVED review anchored at head (COMMENTED or CHANGES_REQUESTED at head is named an unfinished cycle); a passing CodeRabbit status check is documented as non-evidence that a review ran; and the terminus is a merge-ready report with merging stated as the owner's action outside the skill. The state script gains coderabbit-filtered summary fields and an approved_at_head flag, with regression tests for the pending-review exclusion and both halves of the head-anchor gate. Provenance and license attribution updated for the retargeting. Verified: python3 -m unittest discover (7 tests, all passing) and a dry-run scenario agent that correctly scoped threads, rejected a COMMENTED review as unfinished, stopped at the report without merging, and refused a reviewer-supplied shell command. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThis PR replaces the Copilot review-loop skill with a CodeRabbit workflow. It updates review-state reporting, adds approval-at-head checks, retargets tests, updates provenance, and adds Python cache ignore rules. ChangesCodeRabbit review workflow
Review-state calculation
Regression coverage
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant PR workflow
participant review_state.py
participant CodeRabbit review
participant Current PR head
PR workflow->>review_state.py: inspect review state
review_state.py->>CodeRabbit review: filter submitted reviews and authored threads
CodeRabbit review-->>review_state.py: latest review and unresolved-thread data
review_state.py->>Current PR head: compare approval commit
Current PR head-->>review_state.py: current head commit
review_state.py-->>PR workflow: approval-at-head status
Possibly related PRs
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.claude/skills/coderabbit-review-loop/scripts/review_state.py:
- Around line 160-161: Update is_coderabbit to return true only for the exact
logins coderabbitai and coderabbitai[bot], while preserving false results for
missing authors and other values. Add a regression test covering a lookalike
login such as notcoderabbit to ensure it is not classified as CodeRabbit.
In @.claude/skills/coderabbit-review-loop/SKILL.md:
- Line 14: Update the actionable-thread policy and the clean-cycle/final-report
gates in the skill documentation so non-CodeRabbit threads remain outside this
skill’s actionable scope but still block a merge-ready report until resolved or
explicitly handled by the owner. Ensure the rules at the clean-cycle and
final-report sections consistently enforce this requirement without changing how
CodeRabbit threads are processed.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: c0452839-57af-499e-813e-53c26424d8b9
📒 Files selected for processing (6)
.claude/skills/coderabbit-review-loop/.gitignore.claude/skills/coderabbit-review-loop/LICENSE.txt.claude/skills/coderabbit-review-loop/SKILL.md.claude/skills/coderabbit-review-loop/scripts/review_state.py.claude/skills/coderabbit-review-loop/tests/test_review_state.py.claude/skills/gh-copilot-review-loop/SKILL.md
💤 Files with no reviewable changes (1)
- .claude/skills/gh-copilot-review-loop/SKILL.md
| def is_coderabbit(author: dict[str, Any] | None) -> bool: | ||
| return "coderabbit" in (author or {}).get("login", "").lower() |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
file=".claude/skills/coderabbit-review-loop/scripts/review_state.py"
printf '%s\n' '--- target file symbols ---'
ast-grep outline "$file" 2>/dev/null || true
printf '%s\n' '--- target implementation and callers ---'
nl -ba "$file" | sed -n '130,190p'
rg -n -C 3 'is_coderabbit|coderabbit_review_count|latest_coderabbit_review|approved_at_head|unresolved' .claude/skills/coderabbit-review-loop
printf '%s\n' '--- tests and identity references ---'
rg -n -i -C 3 'coderabbitai|coderabbit\[bot\]|notcoderabbit|review_state|author.*login|login.*author' . --glob '!node_modules' --glob '!dist' --glob '!build'Repository: adamgell/cmtraceopen
Length of output: 15336
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- implementation context ---'
sed -n '118,170p' .claude/skills/coderabbit-review-loop/scripts/review_state.py
printf '%s\n' '--- test fixtures and relevant tests ---'
sed -n '1,230p' .claude/skills/coderabbit-review-loop/tests/test_review_state.py
printf '%s\n' '--- all exact identity references ---'
rg -n -i -C 2 'coderabbitai|coderabbit\[bot\]|notcoderabbit|coderabbit' \
.claude/skills/coderabbit-review-loop README.md .github 2>/dev/null || true
printf '%s\n' '--- standalone predicate probe ---'
python3 - <<'PY'
def is_coderabbit(author):
return "coderabbit" in (author or {}).get("login", "").lower()
for login in [None, "", "coderabbitai", "coderabbitai[bot]", "notcoderabbit", "coderabbitai-mirror"]:
print(repr(login), is_coderabbit(None if login is None else {"login": login}))
PYRepository: adamgell/cmtraceopen
Length of output: 32380
🌐 Web query:
What is the official GitHub username/login for the CodeRabbit review bot, and does GitHub API represent it as coderabbitai or coderabbitai[bot]?
💡 Result:
The official GitHub identity for the CodeRabbit application is coderabbitai [1][2]. However, when interacting with GitHub via the API or posting comments, the bot is frequently represented as coderabbitai[bot] [3][4]. In practice, CodeRabbit uses these two distinct identities depending on the context: 1. coderabbitai[bot]: This is the standard login identifier assigned by GitHub to the CodeRabbit App for most automated interactions, including pull request (PR) reviews, inline comments, and automated PR-level activity [5][3][6]. Most technical integrations and GitHub API queries for PR reviews and comments correctly target coderabbitai[bot] [3][7][6]. 2. coderabbitai: This identifier is also used by the bot, particularly when posting certain types of issue comments or when users refer to the bot via mentions (e.g., @coderabbitai) in PR comments or descriptions [5][8][9]. Some automated polling logic specifically requires filtering by coderabbitai rather than the version with the [bot] suffix to correctly capture these activities [5]. Because different contexts may trigger activity from one or both identities, robust integrations often account for both (or even variations like coderabbit[bot]) to ensure all bot-generated content is captured [10].
Citations:
- 1: https://github.com/coderabbitai
- 2: https://github.com/apps/coderabbitai
- 3: https://github.com/duyet/codex-claude-plugins/blob/master/github/skills/review-bots.md
- 4: https://github.com/coderabbitai[bot]
- 5: Fix CR issue plan polling: use 'coderabbitai' not 'coderabbitai[bot]' for issue comments auerbachb/claude-code-config#137
- 6: https://github.com/opendatahub-io/ai-helpers/blob/main/helpers/skills/coderabbit-review/SKILL.md
- 7: https://github.com/jeremylongshore/claude-code-plugins-plus-skills/blob/main/plugins/saas-packs/coderabbit-pack/skills/coderabbit-sdk-patterns/SKILL.md
- 8: https://coderabbit.mintlify.app/reference/review-commands
- 9: https://coderabbit.mintlify.app/guides/commands
- 10: https://github.com/coderabbitai/skills/blob/main/skills/autofix/SKILL.md
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import json
import urllib.parse
import urllib.request
for login in ("coderabbitai", "coderabbitai[bot]", "coderabbit"):
url = "https://api.github.com/users/" + urllib.parse.quote(login, safe="")
request = urllib.request.Request(
url,
headers={
"Accept": "application/vnd.github+json",
"User-Agent": "review-state-identity-check",
},
)
try:
with urllib.request.urlopen(request, timeout=10) as response:
payload = json.load(response)
print(login, json.dumps({
"status": payload.get("type"),
"login": payload.get("login"),
"id": payload.get("id"),
"name": payload.get("name"),
"html_url": payload.get("html_url"),
}, sort_keys=True))
except Exception as error:
print(login, type(error).__name__, str(error))
print("--- exact-match probe ---")
allowed = {"coderabbitai", "coderabbitai[bot]"}
for login in (
None,
"",
"coderabbitai",
"coderabbitai[bot]",
"notcoderabbit",
"coderabbitai-mirror",
"x-coderabbitai",
):
result = isinstance(login, str) and login.lower() in allowed
print(repr(login), result)
assert not (isinstance("notcoderabbit", str) and "notcoderabbit".lower() in allowed)
assert not (isinstance("coderabbitai-mirror", str) and "coderabbitai-mirror".lower() in allowed)
PYRepository: adamgell/cmtraceopen
Length of output: 796
Match CodeRabbit reviewer logins exactly.
The substring check can treat notcoderabbit as CodeRabbit and set approved_at_head from an unrelated review. Restrict matching to coderabbitai and coderabbitai[bot], and add a regression test for lookalike logins.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.claude/skills/coderabbit-review-loop/scripts/review_state.py around lines
160 - 161, Update is_coderabbit to return true only for the exact logins
coderabbitai and coderabbitai[bot], while preserving false results for missing
authors and other values. Add a regression test covering a lookalike login such
as notcoderabbit to ensure it is not classified as CodeRabbit.
|
|
||
| 1. Confirm `gh auth status`, the current branch, a clean understanding of the worktree, and the associated open PR. | ||
| 2. Run `scripts/review_state.py` from the skill directory. Record `head_sha`, `coderabbit_review_count`, `latest_coderabbit_review_state`, `approved_at_head`, and the unresolved CodeRabbit thread IDs. | ||
| 3. Actionable means: an unresolved, non-outdated thread whose comments include `coderabbitai`. Address every actionable thread unless it is informational, a duplicate, incorrect, or conflicts with requirements - those get a reply explaining the disposition instead of a fix. Threads from other reviewers are out of this loop's scope; note them in the final report. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Require resolution of non-CodeRabbit reviewer threads before completion.
Line 14 excludes these threads from the gate. Lines 57 and 63 then permit a merge-ready report while they remain open. This conflicts with the PR requirement that other reviewer threads are explicitly resolved.
Keep non-CodeRabbit feedback non-actionable for this skill, but block the merge-ready report until each thread is resolved or the owner explicitly handles it. Update the clean-cycle gate and final-report rule to match.
🧰 Tools
🪛 SkillSpector (2.5.1)
[error] 18: [MP3] Memory Manipulation: Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.
Remediation: Protect agent memory and state from modification by untrusted content. Use read-only memory for critical instructions and validate all state changes.
(Memory Poisoning (MP3))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.claude/skills/coderabbit-review-loop/SKILL.md at line 14, Update the
actionable-thread policy and the clean-cycle/final-report gates in the skill
documentation so non-CodeRabbit threads remain outside this skill’s actionable
scope but still block a merge-ready report until resolved or explicitly handled
by the owner. Ensure the rules at the clean-cycle and final-report sections
consistently enforce this requirement without changing how CodeRabbit threads
are processed.
There was a problem hiding this comment.
Pull request overview
This PR retargets the repository’s automated “review loop” skill from GitHub Copilot reviews to CodeRabbit reviews, updating the workflow definition, review-state summarization script, and associated regression tests to enforce a clean-cycle gate that requires an APPROVED-at-head CodeRabbit review and ends with a merge-ready report (without performing merges).
Changes:
- Replaced the old
gh-copilot-review-loopskill withcoderabbit-review-loop, including updated workflow steps and a stricter clean-cycle gate (approved_at_head). - Updated
scripts/review_state.pyand tests to summarize CodeRabbit reviews/threads and compute approval-at-head metadata. - Updated provenance/attribution text and added Python cache ignores for the new skill directory.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| .claude/skills/gh-copilot-review-loop/SKILL.md | Removes the prior Copilot-focused review-loop skill definition. |
| .claude/skills/coderabbit-review-loop/SKILL.md | Adds the CodeRabbit-focused review-loop workflow, clean-cycle gate, and no-merge terminus guidance. |
| .claude/skills/coderabbit-review-loop/scripts/review_state.py | Retargets review-state summarization to CodeRabbit, adds latest review state + approved_at_head. |
| .claude/skills/coderabbit-review-loop/tests/test_review_state.py | Updates regression tests to validate CodeRabbit review counting, latest state, and approval-at-head behavior. |
| .claude/skills/coderabbit-review-loop/LICENSE.txt | Updates third-party attribution labels and downstream-change descriptions for the retargeted skill. |
| .claude/skills/coderabbit-review-loop/.gitignore | Ignores Python bytecode/cache artifacts for the skill’s scripts/tests. |
Suppressed comments (2)
.claude/skills/coderabbit-review-loop/scripts/review_state.py:161
is_coderabbit()currently matches any login containing "coderabbit", but the skill definition and summary fields are intended to scope specifically to CodeRabbit's account (e.g.,coderabbitai/coderabbitai[bot]). Using a broad substring match could misclassify other users/apps and inflate counts.
.claude/skills/coderabbit-review-loop/scripts/review_state.py:136unresolved_coderabbit_thread_countincludes outdated threads, but the workflow defines actionable CodeRabbit threads as "unresolved, non-outdated". This mismatch can cause the summary to over-report actionable threads.
This issue also appears on line 160 of the same file.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| --- | ||
| name: coderabbit-review-loop | ||
| description: Use when a pull request needs its CodeRabbit review driven to approval - an outstanding CHANGES_REQUESTED review from coderabbitai[bot], unresolved CodeRabbit threads, a draft waiting on the CodeRabbit quality gate, or requests such as "address the CodeRabbit comments and re-review until clean." | ||
| --- |
…t-1.6.0 fixes (#633) * docs(changelog): cite the shipped issue numbers and record main's post-1.6.0 fixes Three things were wrong with the changelog as it stood after the 1.6.0 cut. **The 1.6.0 section under-cited its own SCCM work.** A check across every commit in the tag (`v1.5.2..9bbaf67`) found twelve issue numbers that shipped without appearing anywhere in the section: #335, #336, #342, #344, #346, #348, #350, #394, #404, #418, #444, #458. They are the foundation and hardening under the one SCCM bullet — diagnostic spine, evidence framing, canonical intake and health contracts, signal tokens, correlation keys, coverage assessment, bounded ids, overlapping evidence ranges, client discovery, DP-to-intake binding. The bullet now names them. Two more were folded into bullets that described their work without citing them: the CodeRabbit retarget and review charter (#537, #538 under the agent-tooling bullet) and the framework's own extraction (#543, #548 under Reducer Framework v1). **Main carries two fixes the 1.6.0 tag does not.** `[Unreleased]` was empty while `9bbaf67e..main` held the updater-manifest publisher fix and the `rustls` advisory bump, so both are recorded now. Verified: a script that walks every `feat`/`fix`/`perf`/`refactor`/`build`/`ci` commit in the tag, extracts its issue references and checks them against both the 1.6.0 and 1.5.2 sections now reports zero undocumented references. The one that looked missing before, #521, is documented under 1.5.2 where it shipped. * docs(changelog): name the redaction boundary decision in the 1.6.0 section The section cited the work but never named ADR-004 revision 1 or the ruling that makes it load-bearing. The changelog is the durable record, so the decision now appears there with the audit finding that motivated it, what changed (projected form is the only constructible public form, preserving variants are explicitly named local-only entry points), and a pointer to the document that carries the eight rulings. * docs(changelog): say the redaction decision in plain English Same decision, no jargon: 'covered almost none of the places data actually leaves the app' rather than 'binds at almost no real export surface', 'the version holding the real values' rather than 'the preserving variant', and the check described as following through nested data rather than as structural enforcement being recursive. The eight rulings and the ADR link are unchanged.
Converts gh-copilot-review-loop into coderabbit-review-loop, matching the new CodeRabbit request-changes gate (96b841c).
Baseline (RED) dry-run against the old skill: agent addressed only Copilot threads, declared a clean cycle with a CHANGES_REQUESTED CodeRabbit review outstanding at head, and had no merge guidance. The rewrite closes all three: coderabbitai-scoped actionability, an approved-at-head clean-cycle gate that names COMMENTED/CHANGES_REQUESTED at head an unfinished cycle, and a hard terminus at a merge-ready report (merging is the owner's action).
Script: coderabbit summary fields + approved_at_head; 7 unit tests green. GREEN dry-run verified thread scoping, gate rejection of COMMENTED at head, the no-merge terminus, and refusal of a reviewer-supplied shell command.
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation
Refactor