Skip to content

shape-doctor.py: validator_row reads a validator's exit 1 as "reports a finding" whatever caused it, so a validator that CRASHED (a : in the standard's path splits the PYTHONPATH the template fallback runs under, and validate-manifest.py dies on import repo_shape) is a FINDING row quoting ModuleNotFoundError, and a root whose manifest is valid goes from COMPLIANT, SHAPE BEHIND to INVALID (manifest) #278

Closed

Description

@brettheap

Lane: openreposhape-4 (openRepoShape-4)

Found by this lane's adversarial review of the #238 fix branch (head 59724c9; its out-of-scope item O3: "the doctor reading a validator traceback as a finding is pre-existing"), and re-verified on main eb6df75 (d89fffa plus #220; every line cited was read on eb6df75) with a standard checkout copied under a directory whose name holds a :. The general half, a crash read as a finding, is shown with a second, synthetic crash. Pre-existing and not caused by any open branch. Not claimed.

What is wrong. The doctor judges a validator by its exit code alone. run_validator returns (exit code, one line worth quoting, whole output) (shape-doctor.py:805-847); validator_row turns 0 into ok ... passes and anything else into a FINDING, with verb = "refuses" if code >= 2 else "reports a finding" (:876-879). The validators' own protocol is 0 valid · 1 findings · 2 refusal (templates/assembly-root/scripts/validate-manifest.py:53), and every finding or refusal they print starts FINDING or REFUSED, which run_validator already looks for to pick its quoted line (:841). A Python traceback also exits 1, so a validator that cannot start is a "finding" the project is said to have, quoting the traceback's last line, and the verdict follows it (a manifest FINDING is INVALID (manifest)).

The trigger the review found: when a root has no scripts/validate-manifest.py (or validate-pins.py or validate-family.py) of its own, the doctor runs the standard's template copy, which imports repo_shape from beside itself (templates/assembly-root/scripts/validate-manifest.py:63-64) and so needs <standard>/scripts on PYTHONPATH; run_validator sets PYTHONPATH to <standard>/scripts joined with os.pathsep (:816-820). A standard whose path holds : (POSIX) or ; (Windows, where a path may hold one) is split by that very separator, and the child cannot import repo_shape:

# $U: a clean checkout at eb6df75. $STD: `git clone $U "$R/dir:with-colon/standard"` at the same commit. $C: a scaffolded root, then
# `git rm scripts/validate-manifest.py` and its row removed from contracts/shape-pin.yaml, so the template is the only manifest validator and the manifest is valid.
python3 $U/shape-doctor.py --root $C
#   ok  manifest   the standard's templates/assembly-root/scripts/validate-manifest.py passes
#   COMPLIANT, SHAPE BEHIND (0 upstream-changed, 1 upstream-added)   (exit 1)
python3 "$STD/shape-doctor.py" --root $C
#   FINDING  manifest  the standard's templates/assembly-root/scripts/validate-manifest.py reports a finding (exit 1): ModuleNotFoundError: No module named 'repo_shape'
#   INVALID (manifest)   (exit 1)
# the child's own run, as the doctor starts it:
cd $C && PYTHONPATH="$STD/scripts" python3 "$STD/templates/assembly-root/scripts/validate-manifest.py" --root $C
#   File "$STD/templates/assembly-root/scripts/validate-manifest.py", line 64, in <module>
#   ModuleNotFoundError: No module named 'repo_shape'          exit 1

--json carries it as status: FINDING, detail: {"own_copy": false, "exit": 1}. The same root and the same commit, only the directory the standard sits in differs, and a project with a valid manifest is told it has a finding.

The general half does not need a colon. With a scaffolded root whose own scripts/validate-pins.py is replaced by raise RuntimeError("boom") (a synthetic crash; the shape currency row is red as well, for the edited copy, and carries the verdict):

python3 $U/shape-doctor.py --root $C2 --json      # row pins: FINDING | the project's own scripts/validate-pins.py reports a finding (exit 1): RuntimeError: boom

How it reads once #238 lands. The fix round on origin/fix/doctor-template-validator-next (head 88c81e0, not on main, no PR yet; it waits for #232) gives PYTHONPATH to the template copy only (validator_scripts: None if own_copy else shape / "scripts", run_validator(..., pythonpath=...), 88c81e0:shape-doctor.py:884-941), and prints the same directory in the NEXT it hands over (validator_command: PYTHONPATH=<q> python3 <template> ..., :993-1064). The : trigger therefore keeps biting on the one copy that needs the variable: I put that head's shape-doctor.py over $STD and ran it on the same root, which gave the same FINDING quoting ModuleNotFoundError, with next = PYTHONPATH=$STD/scripts python3 $STD/templates/assembly-root/scripts/validate-manifest.py --root $C # read its output in full ..., a line that splits the same way when pasted (PYTHONPATH has no way to quote a separator). (That is the one file from the branch over main's tree, not a merge of the branch, which is stacked on #232.) A root's own copy stops getting PYTHONPATH at all (today it gets the split value and ignores it, because it imports repo_shape from beside itself), so it is unaffected either way. The general half, the exit code read as a finding, is not touched by #238.

Why. A crash and a finding are told apart by nothing in the doctor but the exit code, and the two share 1. The doctor already has the verdict for "this checkout could not ask": a row whose detail carries "environment": True makes the verdict CANNOT ANSWER, exit 3 (cannot_answer, :3641; verdict_for, :3789-3791; its one user, shape currency at :1251), which AGENTS.md defines as "THIS CHECKOUT failing to ask, never a statement about their repository" (AGENTS.md:198-200). No validator row uses it.

What it should do. A validator that did not run to a verdict must not be a statement about the project. The decisions are the author's:

  1. How a crash is told. Candidate test: a non-zero exit, no output line starting FINDING/REFUSED and a Traceback (most recent call last) in the output (the template validators and the naming validator print their findings as FINDING <id>: and their refusals as REFUSED <id>:, the two words run_validator already looks for at :841). What it becomes: CANNOT ANSWER for the template copy (the doctor's own environment failed), and for a root's OWN copy that crashes either the same or a FINDING with its own id that says "crashed", since a broken project gate is a fact about the project (shape-doctor.py check_shape_currency takes every refusal update-shape.py prepare raises over the ROOT's own contracts/shape-pin.yaml (a merge conflict, a commit: that is not 40 hex, a deleted file) for this checkout failing to ask, so the verdict is CANNOT ANSWER, exit 3, outranking INVALID, and its NEXT git -C <standard> fetch --all blames the standard for a fault in the repository #237 argues the verdict ladder should not call a repository's fault CANNOT ANSWER, so this is the same line to draw, in the other direction). The row's NEXT must still be a command that shows the traceback.
  2. The :/; trigger itself. The doctor can give the child <standard>/scripts without the environment (a -c wrapper that inserts the directory into sys.path and runs the template with runpy, which has no separator), refuse early with a sentence naming the path when it holds os.pathsep, or copy the template and repo_shape.py beside each other outside the tree. Whichever it is, shape-doctor.py validator_row falls back to the standard's TEMPLATE validator when a root carries no copy of its own, and prints that path as the NEXT command without the PYTHONPATH it runs it under, so pasting the manifest, pins or family row's NEXT dies with ModuleNotFoundError: No module named 'repo_shape' and exit 1, the code a validator uses for a finding #238's printed PYTHONPATH=<q> line has the same limit and should follow the same decision.
  3. Whether the Windows ; case is claimed: os.pathsep there is ; and a Windows path may hold one. It was not run (no Windows here); the colon is the POSIX half only.

How to test. In tests/test_shape_doctor.py: (a) a root with no copy of one validator and a valid manifest, the standard cloned to a directory named a:b (POSIX only, @WINDOWS_SKIP: a colon is not a legal file name there; the Windows spelling is a directory with a ; in it), run the doctor from it, and assert the manifest row is not a FINDING that quotes ModuleNotFoundError (it is ok, or CANNOT ANSWER if decision 2 refuses), and the verdict is the one the same root gets from a plain path; (b) the synthetic crash: a root's own validator replaced by raise RuntimeError("boom") makes the row say it crashed, not "reports a finding (exit 1)"; (c) tests/test_shape_doctor.py's existing finding rows still read as findings (exit 1 with a FINDING line).

Related: #238 (the PYTHONPATH for the template copy, its fix round and review), PR #232, #237 (the other direction: a repository fault read as CANNOT ANSWER), AGENTS.md "Checking a repository's compliance" 1 (the verdict ladder).

Activity

  1. brettheap commented on Oct 12, 2026

    @brettheap
    ContributorAuthor

    CLAIMED — lane openRepoShape-4, session 3431c370-df80-4c11-a041-41bce42d8b38@Eagle, 2026-10-12T11:07:12Z, for #278

    Logged in lanes/log/openRepoShape-4.md at dba638171b67487b4b50248b54dd2d3ae0a3efef.

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoShape#278:
       none
    2. `gh pr list --repo opensoft/openRepoShape --state all --search "#278"`, kept where the row or its body names `#278`:
       none
    3. `git ls-remote --heads git@github.com:opensoft/openRepoShape` branches naming `278` as a whole token:
       none
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions