Skip to content

feat(skills): retarget the review loop from Copilot to CodeRabbit - #537

Merged
adamgell merged 1 commit into
mainfrom
skill/coderabbit-review-loop
Aug 8, 2026
Merged

adamgell merged 1 commit into
mainfrom
skill/coderabbit-review-loop

Conversation

@adamgell

@adamgell adamgell commented Aug 8, 2026 •

Copy link
Copy Markdown
Owner

Converts gh-copilot-review-loop into coderabbit-review-loop, matching the new CodeRabbit request-changes gate (96b841c).

Baseline (RED) dry-run against the old skill: agent addressed only Copilot threads, declared a clean cycle with a CHANGES_REQUESTED CodeRabbit review outstanding at head, and had no merge guidance. The rewrite closes all three: coderabbitai-scoped actionability, an approved-at-head clean-cycle gate that names COMMENTED/CHANGES_REQUESTED at head an unfinished cycle, and a hard terminus at a merge-ready report (merging is the owner's action).

Script: coderabbit summary fields + approved_at_head; 7 unit tests green. GREEN dry-run verified thread scoping, gate rejection of COMMENTED at head, the no-merge terminus, and refusal of a reviewer-supplied shell command.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added a CodeRabbit-based pull request review workflow with review-state checks, actionable-thread handling, re-review support, and approval verification against the latest changes.
    • Added safeguards to ensure outdated approvals do not complete the review cycle.
  • Documentation

    • Added comprehensive guidance for using the CodeRabbit review workflow.
    • Updated third-party attribution and repository change details.
  • Refactor

    • Replaced the former Copilot review workflow with the CodeRabbit workflow.

The repository adopted CodeRabbit's request-changes workflow as its
review gate (96b841c), and Copilot reviews are dropping out of the
loop. A baseline dry-run against the old skill confirmed the failure
this closes: an agent following it addressed only Copilot threads,
reported a clean cycle while a CHANGES_REQUESTED CodeRabbit review sat
at head, and had no merge guidance at all.

The skill becomes coderabbit-review-loop. Actionability is defined as
unresolved non-outdated coderabbitai threads with other reviewers
explicitly noted-not-resolved; re-review triggers by PR comment since
the requested-reviewers API does not apply; the clean-cycle gate
requires an APPROVED review anchored at head (COMMENTED or
CHANGES_REQUESTED at head is named an unfinished cycle); a passing
CodeRabbit status check is documented as non-evidence that a review
ran; and the terminus is a merge-ready report with merging stated as
the owner's action outside the skill. The state script gains
coderabbit-filtered summary fields and an approved_at_head flag, with
regression tests for the pending-review exclusion and both halves of
the head-anchor gate. Provenance and license attribution updated for
the retargeting.

Verified: python3 -m unittest discover (7 tests, all passing) and a
dry-run scenario agent that correctly scoped threads, rejected a
COMMENTED review as unfinished, stopped at the report without merging,
and refused a reviewer-supplied shell command.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions github-actions Bot added enhancement New feature or request feature New feature labels Aug 8, 2026
@coderabbitai

coderabbitai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR replaces the Copilot review-loop skill with a CodeRabbit workflow. It updates review-state reporting, adds approval-at-head checks, retargets tests, updates provenance, and adds Python cache ignore rules.

Changes

CodeRabbit review workflow

Layer / File(s) Summary
Workflow definition and provenance
.claude/skills/coderabbit-review-loop/SKILL.md, .claude/skills/coderabbit-review-loop/LICENSE.txt, .claude/skills/coderabbit-review-loop/.gitignore, .claude/skills/gh-copilot-review-loop/SKILL.md
Adds the CodeRabbit workflow specification, updates attribution, removes the Copilot skill documentation, and ignores Python cache files.

Review-state calculation

Layer / File(s) Summary
CodeRabbit review-state reporting
.claude/skills/coderabbit-review-loop/scripts/review_state.py
Retargets review and thread detection to CodeRabbit. The summary now includes the latest review state and whether approval matches the current PR head.

Regression coverage

Layer / File(s) Summary
Review-state tests
.claude/skills/coderabbit-review-loop/tests/test_review_state.py
Updates fixtures and assertions for CodeRabbit and verifies pending-review filtering, review states, and approval at the current or outdated head.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PR workflow
  participant review_state.py
  participant CodeRabbit review
  participant Current PR head
  PR workflow->>review_state.py: inspect review state
  review_state.py->>CodeRabbit review: filter submitted reviews and authored threads
  CodeRabbit review-->>review_state.py: latest review and unresolved-thread data
  review_state.py->>Current PR head: compare approval commit
  Current PR head-->>review_state.py: current head commit
  review_state.py-->>PR workflow: approval-at-head status
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title follows Conventional Commits and accurately describes retargeting the review loop from Copilot to CodeRabbit.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch skill/coderabbit-review-loop

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.claude/skills/coderabbit-review-loop/scripts/review_state.py:
- Around line 160-161: Update is_coderabbit to return true only for the exact
logins coderabbitai and coderabbitai[bot], while preserving false results for
missing authors and other values. Add a regression test covering a lookalike
login such as notcoderabbit to ensure it is not classified as CodeRabbit.

In @.claude/skills/coderabbit-review-loop/SKILL.md:
- Line 14: Update the actionable-thread policy and the clean-cycle/final-report
gates in the skill documentation so non-CodeRabbit threads remain outside this
skill’s actionable scope but still block a merge-ready report until resolved or
explicitly handled by the owner. Ensure the rules at the clean-cycle and
final-report sections consistently enforce this requirement without changing how
CodeRabbit threads are processed.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c0452839-57af-499e-813e-53c26424d8b9

📥 Commits

Reviewing files that changed from the base of the PR and between 96b841c and 083e69d.

📒 Files selected for processing (6)
  • .claude/skills/coderabbit-review-loop/.gitignore
  • .claude/skills/coderabbit-review-loop/LICENSE.txt
  • .claude/skills/coderabbit-review-loop/SKILL.md
  • .claude/skills/coderabbit-review-loop/scripts/review_state.py
  • .claude/skills/coderabbit-review-loop/tests/test_review_state.py
  • .claude/skills/gh-copilot-review-loop/SKILL.md
💤 Files with no reviewable changes (1)
  • .claude/skills/gh-copilot-review-loop/SKILL.md

Comment on lines +160 to +161
def is_coderabbit(author: dict[str, Any] | None) -> bool:
return "coderabbit" in (author or {}).get("login", "").lower()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -eu

file=".claude/skills/coderabbit-review-loop/scripts/review_state.py"
printf '%s\n' '--- target file symbols ---'
ast-grep outline "$file" 2>/dev/null || true

printf '%s\n' '--- target implementation and callers ---'
nl -ba "$file" | sed -n '130,190p'
rg -n -C 3 'is_coderabbit|coderabbit_review_count|latest_coderabbit_review|approved_at_head|unresolved' .claude/skills/coderabbit-review-loop

printf '%s\n' '--- tests and identity references ---'
rg -n -i -C 3 'coderabbitai|coderabbit\[bot\]|notcoderabbit|review_state|author.*login|login.*author' . --glob '!node_modules' --glob '!dist' --glob '!build'

Repository: adamgell/cmtraceopen

Length of output: 15336


🏁 Script executed:

#!/bin/bash
set -eu

printf '%s\n' '--- implementation context ---'
sed -n '118,170p' .claude/skills/coderabbit-review-loop/scripts/review_state.py

printf '%s\n' '--- test fixtures and relevant tests ---'
sed -n '1,230p' .claude/skills/coderabbit-review-loop/tests/test_review_state.py

printf '%s\n' '--- all exact identity references ---'
rg -n -i -C 2 'coderabbitai|coderabbit\[bot\]|notcoderabbit|coderabbit' \
  .claude/skills/coderabbit-review-loop README.md .github 2>/dev/null || true

printf '%s\n' '--- standalone predicate probe ---'
python3 - <<'PY'
def is_coderabbit(author):
    return "coderabbit" in (author or {}).get("login", "").lower()

for login in [None, "", "coderabbitai", "coderabbitai[bot]", "notcoderabbit", "coderabbitai-mirror"]:
    print(repr(login), is_coderabbit(None if login is None else {"login": login}))
PY

Repository: adamgell/cmtraceopen

Length of output: 32380


🌐 Web query:

What is the official GitHub username/login for the CodeRabbit review bot, and does GitHub API represent it as coderabbitai or coderabbitai[bot]?

💡 Result:

The official GitHub identity for the CodeRabbit application is coderabbitai [1][2]. However, when interacting with GitHub via the API or posting comments, the bot is frequently represented as coderabbitai[bot] [3][4]. In practice, CodeRabbit uses these two distinct identities depending on the context: 1. coderabbitai[bot]: This is the standard login identifier assigned by GitHub to the CodeRabbit App for most automated interactions, including pull request (PR) reviews, inline comments, and automated PR-level activity [5][3][6]. Most technical integrations and GitHub API queries for PR reviews and comments correctly target coderabbitai[bot] [3][7][6]. 2. coderabbitai: This identifier is also used by the bot, particularly when posting certain types of issue comments or when users refer to the bot via mentions (e.g., @coderabbitai) in PR comments or descriptions [5][8][9]. Some automated polling logic specifically requires filtering by coderabbitai rather than the version with the [bot] suffix to correctly capture these activities [5]. Because different contexts may trigger activity from one or both identities, robust integrations often account for both (or even variations like coderabbit[bot]) to ensure all bot-generated content is captured [10].

Citations:


🏁 Script executed:

#!/bin/bash
set -eu

python3 - <<'PY'
import json
import urllib.parse
import urllib.request

for login in ("coderabbitai", "coderabbitai[bot]", "coderabbit"):
    url = "https://api.github.com/users/" + urllib.parse.quote(login, safe="")
    request = urllib.request.Request(
        url,
        headers={
            "Accept": "application/vnd.github+json",
            "User-Agent": "review-state-identity-check",
        },
    )
    try:
        with urllib.request.urlopen(request, timeout=10) as response:
            payload = json.load(response)
        print(login, json.dumps({
            "status": payload.get("type"),
            "login": payload.get("login"),
            "id": payload.get("id"),
            "name": payload.get("name"),
            "html_url": payload.get("html_url"),
        }, sort_keys=True))
    except Exception as error:
        print(login, type(error).__name__, str(error))

print("--- exact-match probe ---")
allowed = {"coderabbitai", "coderabbitai[bot]"}
for login in (
    None,
    "",
    "coderabbitai",
    "coderabbitai[bot]",
    "notcoderabbit",
    "coderabbitai-mirror",
    "x-coderabbitai",
):
    result = isinstance(login, str) and login.lower() in allowed
    print(repr(login), result)
assert not (isinstance("notcoderabbit", str) and "notcoderabbit".lower() in allowed)
assert not (isinstance("coderabbitai-mirror", str) and "coderabbitai-mirror".lower() in allowed)
PY

Repository: adamgell/cmtraceopen

Length of output: 796


Match CodeRabbit reviewer logins exactly.

The substring check can treat notcoderabbit as CodeRabbit and set approved_at_head from an unrelated review. Restrict matching to coderabbitai and coderabbitai[bot], and add a regression test for lookalike logins.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.claude/skills/coderabbit-review-loop/scripts/review_state.py around lines
160 - 161, Update is_coderabbit to return true only for the exact logins
coderabbitai and coderabbitai[bot], while preserving false results for missing
authors and other values. Add a regression test covering a lookalike login such
as notcoderabbit to ensure it is not classified as CodeRabbit.


1. Confirm `gh auth status`, the current branch, a clean understanding of the worktree, and the associated open PR.
2. Run `scripts/review_state.py` from the skill directory. Record `head_sha`, `coderabbit_review_count`, `latest_coderabbit_review_state`, `approved_at_head`, and the unresolved CodeRabbit thread IDs.
3. Actionable means: an unresolved, non-outdated thread whose comments include `coderabbitai`. Address every actionable thread unless it is informational, a duplicate, incorrect, or conflicts with requirements - those get a reply explaining the disposition instead of a fix. Threads from other reviewers are out of this loop's scope; note them in the final report.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Require resolution of non-CodeRabbit reviewer threads before completion.

Line 14 excludes these threads from the gate. Lines 57 and 63 then permit a merge-ready report while they remain open. This conflicts with the PR requirement that other reviewer threads are explicitly resolved.

Keep non-CodeRabbit feedback non-actionable for this skill, but block the merge-ready report until each thread is resolved or the owner explicitly handles it. Update the clean-cycle gate and final-report rule to match.

🧰 Tools
🪛 SkillSpector (2.5.1)

[error] 18: [MP3] Memory Manipulation: Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.

Remediation: Protect agent memory and state from modification by untrusted content. Use read-only memory for critical instructions and validate all state changes.

(Memory Poisoning (MP3))

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.claude/skills/coderabbit-review-loop/SKILL.md at line 14, Update the
actionable-thread policy and the clean-cycle/final-report gates in the skill
documentation so non-CodeRabbit threads remain outside this skill’s actionable
scope but still block a merge-ready report until resolved or explicitly handled
by the owner. Ensure the rules at the clean-cycle and final-report sections
consistently enforce this requirement without changing how CodeRabbit threads
are processed.

@adamgell
adamgell marked this pull request as ready for review August 8, 2026 21:21
Copilot AI lite review requested due to automatic review settings August 8, 2026 21:21
@adamgell
adamgell merged commit 608e665 into main Aug 8, 2026
17 checks passed
@adamgell
adamgell deleted the skill/coderabbit-review-loop branch August 8, 2026 21:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR retargets the repository’s automated “review loop” skill from GitHub Copilot reviews to CodeRabbit reviews, updating the workflow definition, review-state summarization script, and associated regression tests to enforce a clean-cycle gate that requires an APPROVED-at-head CodeRabbit review and ends with a merge-ready report (without performing merges).

Changes:

  • Replaced the old gh-copilot-review-loop skill with coderabbit-review-loop, including updated workflow steps and a stricter clean-cycle gate (approved_at_head).
  • Updated scripts/review_state.py and tests to summarize CodeRabbit reviews/threads and compute approval-at-head metadata.
  • Updated provenance/attribution text and added Python cache ignores for the new skill directory.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
.claude/skills/gh-copilot-review-loop/SKILL.md Removes the prior Copilot-focused review-loop skill definition.
.claude/skills/coderabbit-review-loop/SKILL.md Adds the CodeRabbit-focused review-loop workflow, clean-cycle gate, and no-merge terminus guidance.
.claude/skills/coderabbit-review-loop/scripts/review_state.py Retargets review-state summarization to CodeRabbit, adds latest review state + approved_at_head.
.claude/skills/coderabbit-review-loop/tests/test_review_state.py Updates regression tests to validate CodeRabbit review counting, latest state, and approval-at-head behavior.
.claude/skills/coderabbit-review-loop/LICENSE.txt Updates third-party attribution labels and downstream-change descriptions for the retargeted skill.
.claude/skills/coderabbit-review-loop/.gitignore Ignores Python bytecode/cache artifacts for the skill’s scripts/tests.
Suppressed comments (2)

.claude/skills/coderabbit-review-loop/scripts/review_state.py:161

  • is_coderabbit() currently matches any login containing "coderabbit", but the skill definition and summary fields are intended to scope specifically to CodeRabbit's account (e.g., coderabbitai / coderabbitai[bot]). Using a broad substring match could misclassify other users/apps and inflate counts.
    .claude/skills/coderabbit-review-loop/scripts/review_state.py:136
  • unresolved_coderabbit_thread_count includes outdated threads, but the workflow defines actionable CodeRabbit threads as "unresolved, non-outdated". This mismatch can cause the summary to over-report actionable threads.

This issue also appears on line 160 of the same file.


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +1 to +4
---
name: coderabbit-review-loop
description: Use when a pull request needs its CodeRabbit review driven to approval - an outstanding CHANGES_REQUESTED review from coderabbitai[bot], unresolved CodeRabbit threads, a draft waiting on the CodeRabbit quality gate, or requests such as "address the CodeRabbit comments and re-review until clean."
---
adamgell added a commit that referenced this pull request Sep 18, 2026
…t-1.6.0 fixes (#633)

* docs(changelog): cite the shipped issue numbers and record main's post-1.6.0 fixes

Three things were wrong with the changelog as it stood after the 1.6.0 cut.

**The 1.6.0 section under-cited its own SCCM work.** A check across every commit
in the tag (`v1.5.2..9bbaf67`) found twelve issue numbers that shipped without
appearing anywhere in the section: #335, #336, #342, #344, #346, #348, #350,
#394, #404, #418, #444, #458. They are the foundation and hardening under the one
SCCM bullet — diagnostic spine, evidence framing, canonical intake and health
contracts, signal tokens, correlation keys, coverage assessment, bounded ids,
overlapping evidence ranges, client discovery, DP-to-intake binding. The bullet
now names them. Two more were folded into bullets that described their work
without citing them: the CodeRabbit retarget and review charter (#537, #538 under
the agent-tooling bullet) and the framework's own extraction (#543, #548 under
Reducer Framework v1).

**Main carries two fixes the 1.6.0 tag does not.** `[Unreleased]` was empty while
`9bbaf67e..main` held the updater-manifest publisher fix and the `rustls` advisory
bump, so both are recorded now.

Verified: a script that walks every `feat`/`fix`/`perf`/`refactor`/`build`/`ci`
commit in the tag, extracts its issue references and checks them against both the
1.6.0 and 1.5.2 sections now reports zero undocumented references. The one that
looked missing before, #521, is documented under 1.5.2 where it shipped.

* docs(changelog): name the redaction boundary decision in the 1.6.0 section

The section cited the work but never named ADR-004 revision 1 or the ruling that
makes it load-bearing. The changelog is the durable record, so the decision now
appears there with the audit finding that motivated it, what changed (projected
form is the only constructible public form, preserving variants are explicitly
named local-only entry points), and a pointer to the document that carries the
eight rulings.

* docs(changelog): say the redaction decision in plain English

Same decision, no jargon: 'covered almost none of the places data actually leaves
the app' rather than 'binds at almost no real export surface', 'the version
holding the real values' rather than 'the preserving variant', and the check
described as following through nested data rather than as structural enforcement
being recursive. The eight rulings and the ADR link are unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants