Skip to content

tests: nothing ties a refusal code AGENTS.md spells to the tool that prints it, so now that #309 has landed (7491530), a misspelt scaffold-leg-path-collides or scaffold-pin-collides in AGENTS.md, or either code renamed in scaffold-project.py, passes every test, and an assistant briefed from AGENTS.md relays a code no tool prints #348

Closed

Description

@brettheap

Lane: openreposhape-4 (openRepoShape-4)

Found by this lane's adversarial review of #288 (PR #309, head 72d1cc2; its NIT-4). That review misspelt both codes in AGENTS.md and ran tests/test_repo_hygiene.py plus tests/test_cli_reference.py: 63 passed. Its re-check at 3fd3669 carried the finding forward as NIT-E. Re-verified at #309's landing head 2ea1717, the bytes that bring the two codes into AGENTS.md. On main c7c02fa AGENTS.md names no refusal code at all, so the gap opens when #309 lands, which is now under way; the landing is what creates it. Not claimed: README.md, docs/handbook.html and docs/cli.md, which #309 leaves without either code, and the full suite. The mutant was run against the one test file that reads AGENTS.md; the other files that name either code were found by grep.

What is wrong. #288 asked that both codes appear in AGENTS.md "spelled as the tool prints them". #309 does that: AGENTS.md section 2 names scaffold-leg-path-collides and scaffold-pin-collides, once each. No test reads them back. The only test file that opens the standard's own AGENTS.md is tests/test_repo_hygiene.py, and it asserts the line cap, the Windows two-liner and a few fixed phrases. The other tests that read an AGENTS.md read a template's, a holder's or a scaffolded project's. The tests that assert the codes (tests/test_scaffold_pin_collisions.py:47-48, tests/test_scaffold_leg_path_collisions.py:60) read them from the tool's own output, never from AGENTS.md. Two more files name a code in a comment only: tests/test_repo_hygiene.py:565 (the cap entry) and tests/test_scaffold_mount_ignored.py:126.

# $U: a clean checkout of #309's head 2ea1717 (or main at or after 7491530, where #309 landed).
cd "$U"
grep -n -o 'scaffold-leg-path-collides\|scaffold-pin-collides' AGENTS.md
#   89:scaffold-leg-path-collides
#   93:scaffold-pin-collides
sed -i.bak 's/scaffold-leg-path-collides/scaffold-leg-path-collision/; s/scaffold-pin-collides/scaffold-pin-collision/' AGENTS.md
python3 -m pytest tests/test_repo_hygiene.py -q -p no:cacheprovider     # the one test file that reads AGENTS.md
#   50 passed in 2.19s
mv AGENTS.md.bak AGENTS.md

The other direction is open too. Rename either code in scaffold-project.py and its own refusal tests go red, but they are fixed by editing their constants. AGENTS.md keeps the old spelling and nothing notices. #288's own "How to test" asked for a grep by hand, which holds on the day it is run.

Why. AGENTS.md is the procedure an assistant reads before acting, and section 2 now tells it which refusals to foresee and to relay them rather than choose another value. A code spelt there differently from what the tool prints is a sentence the assistant cannot match against the refusal in front of it. The repository already guards other copied facts this way: the Windows two-liner's file names (test_the_windows_commands_name_files_that_exist), the handbook's README blob, and docs/cli.md against --help. None of those guards covers a refusal code.

What it should do. A refusal code AGENTS.md names is one a tool prints, by a test, in both directions. The decisions are the author's:

  1. Which names.
    • Either an explicit list (today the two scaffold codes), each mapped to the tool that prints it.
    • Or every backticked token in AGENTS.md shaped like a code ([a-z]+(-[a-z0-9]+)+), checked against the tools' sources. At 2ea1717 that pattern matches six tokens: the two codes; upstream-changed, upstream-added and locally-modified (update-shape.py's and the doctor's verdict words); and prefer-triad-project-shape, an openxFactory ruling id that no tool prints. A pattern therefore needs an allowlist for names that are not codes, and it would also cover verdict words.
  2. What "prints it" means.
    • The literal appears in the tool's source as a Refusal("<code>", ...) first argument: cheap, and it holds on Windows.
    • Or the test runs the refusal, as the two collision test files already do, and imports their REFUSED constants, so that one spelling serves both.
  3. Which documents. AGENTS.md alone, or README.md and docs/handbook.html as well, should either ever name a code (AGENTS.md, README and scaffold-project.py --help: say that a --spec-path or --code-path that is, holds or lies inside a file the shape writes, and a --pin of spec, code or shape or of one product twice, are refused before anything is created, and name those three in the pin refusal's remediation, so an agent briefed from AGENTS.md can say so before it runs the scaffold (#288) #309 deliberately put the codes in AGENTS.md only).
  4. Where. tests/test_repo_hygiene.py, beside the other copied-fact guards, or a new file.

How to test. The mutant above fails the new test, naming the misspelt token and the document. The reverse mutant fails it too: rename scaffold-pin-collides in scaffold-project.py and update the two collision tests' constants, and the new test reports that AGENTS.md names a code no tool prints. A token on the allowlist (decision 1) does not fail it.

Verified at #309's head 2ea1717 (AGENTS.md sha256 136bca75…, mutated 4707c97e…; tests/test_repo_hygiene.py 0feb3e93…); main gained both codes when #309 landed as 7491530.

Related:

Activity

  1. brettheap commented on Oct 12, 2026

    @brettheap
    ContributorAuthor

    CLAIMED — lane openRepoShape-4, session e7cd5391-c790-49b0-a88f-a97d34b22582@Eagle, 2026-10-12T15:19:29Z, for #348

    Logged in lanes/log/openRepoShape-4.md at 95be8622d8c934e1eb71b305afaa1b8d6df9f8a0.

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoShape#348:
       none
    2. `gh pr list --repo opensoft/openRepoShape --state all --search "#348"`, kept where the row or its body names `#348`:
       none
    3. `git ls-remote --heads git@github.com:opensoft/openRepoShape` branches naming `348` as a whole token:
       none
    
  2. added 2 commits that reference this issue on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions