chore(setup): drop the comdev-checkout check from setup-isolated-setup-verify - #1185
Merged
Merged
Conversation
…p-verify Check 9 asserted that the local `apache/comdev` checkout backing the ponymail and apache-projects MCP servers has an `apache/comdev` `origin` and sits on `main`. That checkout is a personal working copy; which branch it happens to be on, and whether `origin` names the canonical repo or a contributor's fork, are not properties of the secure agent setup this skill exists to certify. A contributor carrying a topic branch off their own fork was being reported as a broken install. The skill's own evals had already settled this: every case under `tools/skill-evals/evals/setup-isolated-setup-verify/` asserts exactly eight checks, numbered 1-8 and ending at project-root coverage, with no comdev case anywhere. SKILL.md and its fixtures have been out of step; they now agree, so no fixture changes were needed. Check 9 was last in the list, so checks 1-8 keep their numbers. The prompt-injection preamble was written specifically around check 9's parsing of untrusted git output (branch names, commit subjects, remote strings). Rather than lose the guardrail along with the check, it now points at the untrusted machine output the remaining checks do parse: the `git worktree list --porcelain` output read by check 8, settings-file contents, and command stderr. The canonical list in `docs/setup/secure-agent-setup.md`, which the skill defers to, drops the matching item. The ponymail and apache-projects tool docs no longer claim verify asserts the checkout is on `main` and not behind; `setup-isolated-setup-update` still surfaces the behind-`origin/main` warning and prints the pull command, which is where that check belongs. Generated-by: Claude Code (Opus 5)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
main,originisapache/comdev)from
setup-isolated-setup-verify. That checkout is a personal working copy —which branch it sits on, and whether
originnames the canonical repo or acontributor's fork, are not properties of the secure agent setup the skill
exists to certify. A contributor carrying a topic branch off their own fork
was being reported as a broken install.
tools/skill-evals/evals/setup-isolated-setup-verify/asserts exactly eightchecks, numbered 1–8 and ending at project-root coverage, with no comdev case
anywhere. SKILL.md and its fixtures were out of step; they now agree.
preamble was written around check 9's parsing of untrusted git output, so
rather than lose the guardrail with the check it now points at the untrusted
machine output the remaining checks do parse: the
git worktree list --porcelainoutput read by check 8, settings-file contents, command stderr.Also updated so nothing claims a check that no longer exists: the canonical list
in
docs/setup/secure-agent-setup.md(which the skill defers to) drops thematching item, and
tools/ponymail/tool.md/tools/apache-projects/tool.mdnolonger say verify asserts the checkout is on
mainand not behind.setup-isolated-setup-updatestill surfaces the behind-origin/mainwarningand prints the pull command — that check keeps living there.
Type of change
.claude/skills/<name>/) — eval fixture note belowtools/<system>/*.md)tools/*/withpyproject.toml)docs/,README.md,CONTRIBUTING.md)projects/_template/)prek, workflows, validators)Test plan
prekpasses on all four changed files — includingcheck-doc-sync(declared skill counts) and
skill-and-tool-validate, the two hooks thatwould catch a stale "nine checks" claim in the frontmatter.
--cli. The runner'sdefault mode is print-only; all 12 fixtures across the three steps load
and render cleanly, but this PR carries no live-CLI eval result. Happy to
run it if a reviewer wants it before merge.
fixtures already assert eight checks and never covered comdev, so this
change makes SKILL.md match its regression tests rather than requiring new
ones.
case-6-injection-attemptstill exercises the preamble's guardrail.RFC-AI-0004 compliance
that resolved a path outside the project from MCP config and shelled
gitagainst it.
check-placeholderspasses; no placeholder prose changedLinked issues
None.
Notes for reviewers
The judgement call worth a look is the prompt-injection preamble. It could have
been deleted along with check 9, since check 9's untrusted git output was its
stated subject. I retargeted it instead — check 8 parses
git worktree list --porcelain, which carries attacker-influenceable branch names and paths, sothe guardrail still has a real referent. Say the word if you'd rather see it
dropped.
🤖 Generated with Claude Code
https://claude.ai/code/session_01KP5vkDusxrhi77Nx9xp9C1