Repository navigation
shape-doctor.py validator_row falls back to the standard's TEMPLATE validator when a root carries no copy of its own, and prints that path as the NEXT command without the PYTHONPATH it runs it under, so pasting the manifest, pins or family row's NEXT dies with ModuleNotFoundError: No module named 'repo_shape' and exit 1, the code a validator uses for a finding #238
Description
Lane: openreposhape-4 (openRepoShape-4)
Found by this lane's adversarial review of the #205 branch (50aa967, its SHOULD S2), re-verified on main a5b09c3 today, on all three rows that share the code, and again on 54aa69a (the one commit since, #173, changes adopt-project.py and its tests only, so every file named below is byte-identical at both). It is pre-existing on main and not a regression of that branch; PR #232's description says a follow-up issue will carry it. Not claimed.
What is wrong. When a root has no scripts/validate-manifest.py, validate-pins.py or validate-family.py of its own, the doctor runs the standard's copy from templates/ instead and says so in the row. Its NEXT then names that template file as a command. The template imports repo_shape from beside itself (templates/assembly-root/scripts/validate-manifest.py:63-64), and repo_shape.py is copied INTO a project's scripts/ out of the standard's scripts/ (REPO_SHAPE, scripts/shape_materialize.py:148), never kept beside the template, so the pasted line dies on the import. The doctor's own run of the same file works, because run_validator puts <standard>/scripts on the child's PYTHONPATH (shape-doctor.py:814-817) and the printed command does not carry it. The pasted command exits 1, which is a validator's exit code for "findings" (validate-manifest.py: "0 valid · 1 findings · 2 refusal"), so anything keyed on the exit code reads a crash as the finding the row already reported, and the one command the row hands over to "read its output in full" prints a traceback that names nothing about the project.
# $U: a clean checkout at a5b09c3. gh is a PATH shim that logs its argv and exits 97. Every repository is a bare repository under $R/remotes; no network.
python3 $U/scaffold-project.py --org testorg --project Atlas --elected-by 'Test Human' --elected-on 2026-09-02 \
--local-remote-dir $R/remotes --work-dir $R/work
git clone -q --recurse-submodules $R/remotes/Atlas.git $C
git -C $C rm -q scripts/validate-manifest.py # a root with no copy of its own
sed -i -E 's|^topic: .*|topic: xf-project-wrong|' $C/project.yaml # a manifest that parses and fails its gate
python3 $U/shape-doctor.py --root $C
# exit 1 (verdict DRIFTED (1 copy-missing): the missing copy is the `shape currency` row's; this issue is the `manifest` row's NEXT)
# FINDING manifest the standard's templates/assembly-root/scripts/validate-manifest.py reports a finding (exit 1): FINDING manifest-topic: topic is 'xf-project-wrong', expected 'xf-project-atlas' derived from id 'atlas'
# NEXT python3 $U/templates/assembly-root/scripts/validate-manifest.py --root $C # read its output in full; it names what it refused
# --json detail {"validator": "$U/templates/assembly-root/scripts/validate-manifest.py", "own_copy": false, "exit": 1}
NEXT=$(python3 $U/shape-doctor.py --root $C | sed -n 's/^ *NEXT *\(python3 .*validate-manifest\.py.*\)$/\1/p')
cd / && sh -c "$NEXT" # the NEXT as printed, comment and all
# exit 1
# Traceback (most recent call last):
# File "$U/templates/assembly-root/scripts/validate-manifest.py", line 64, in <module>
# from repo_shape import ( # noqa: E402
# ModuleNotFoundError: No module named 'repo_shape'
cd / && PYTHONPATH=$U/scripts python3 $U/templates/assembly-root/scripts/validate-manifest.py --root $C # what the doctor ran
# exit 1 FINDING manifest-topic: topic is 'xf-project-wrong', expected 'xf-project-atlas' derived from id 'atlas'
# 1 finding(s) in $C/project.yamlThe same on the other two rows that go through validator_row (a fixture each; the NEXT pasted from / with no PYTHONPATH in the environment):
| row | made red by | the row's NEXT, pasted |
|---|---|---|
pins |
git rm scripts/validate-pins.py, and digests.tree_sha256 in contracts/spec-pin.yaml set to 64 zeros (FINDING pin-digest-mismatch) |
Traceback ... templates/assembly-root/scripts/validate-pins.py", line 58 ... ModuleNotFoundError: No module named 'repo_shape', exit 1 |
family |
a holder from scripts/family.py init ... --local-remote-dir, git rm scripts/validate-family.py, id: Bad Id in family.yaml (FINDING family-id) |
Traceback ... templates/family-root/scripts/validate-family.py", line 57 ... ModuleNotFoundError: No module named 'repo_shape', exit 1 |
A PYTHONPATH the reader's own shell already exports hides it (the pasted command prints the finding instead of the traceback in any shell that already exports a PYTHONPATH reaching the standard's scripts/), which is how a developer with the standard on their path will not see it.
Why. validator_row (shape-doctor.py:848-886) takes the root's copy when own_path.is_file() and otherwise ctx.shape / template (:859-861), runs it through run_validator (:871), and builds the NEXT from the file it ran: f"{PYTHON} {quote_arg(used)} {spelled} # read its output in full ..." (:883-886). Only the path and the arguments reach the line. run_validator's docstring (:804-812) records the exact hazard for the doctor's own run, "when this command falls back to the template it must put <shape>/scripts on the path or the fallback dies on an ImportError that says nothing about the project being checked", and fixes it with the env it builds for subprocess.run (:814-817). The environment is not part of what the row prints, so the printed command is a different process from the one whose output the row quotes. Three rows call it (check_manifest :919, check_pins :926, check_family :933). The review of #232's head reports that branch's new unloaded_manifest_row printing the same template NEXT through a shared validator_for split out of validator_row; on main the choice lives in validator_row itself.
What it should do. A NEXT the row prints is a command, and a command pasted into the shell it was quoted for, from any directory and with no PYTHONPATH in the environment, must do what the row says it does: print the validator's own finding and exit with the code the row records (detail.exit). The fix belongs in the one definition, so every row, and #232's unloaded_manifest_row, follow it. The environment has to be in the line, spelled for the reader's shell, the way quote_arg(value, platform) (:515) already knows two dialects: PYTHONPATH=<shape>/scripts python3 <template> --root <root> for sh, and the PowerShell assignment ($env:PYTHONPATH = '<shape>\scripts'; <interpreter> <template> ...) on Windows, where the separator is os.pathsep. A fallback row could also say, in the row's own words, that this root has no copy and the command is the standard's; that is already true of the reason. What it must not do is keep printing a bare path that only the doctor's own child can run.
How to test. In tests/test_shape_doctor.py, modelled on test_a_red_validators_next_command_is_runnable (:888), which cannot see this: it makes the project's own copy red and runs the NEXT split into argv, so the root always has its own repo_shape.py. (a) Delete the root's own scripts/validate-pins.py, make the pins gate fail (an edited pinned copy elsewhere, or a leg pin edit), take the row's next from --json, run it exactly as printed through the platform shell (sh -c on POSIX) from a different working directory with PYTHONPATH removed from the child's environment, and assert returncode == row["detail"]["exit"], that the row's quoted finding id is in the output, and that Traceback and ModuleNotFoundError are not; (b) the same for manifest and, with the holder fixture (:392), for family; (c) on nt the Windows spelling asserted as text, as the Copy-Item -LiteralPath cases (:1062-1086) assert theirs; (d) restoring the bare {PYTHON} {quote_arg(used)} line makes (a) and (b) fail.
Related: #232 (PR; the review that found it), #205, #101 (#102: the quoter this uses, for the next lines)
Activity
brettheap commentedon Oct 11, 2026
CLAIMED — lane openRepoShape-4, session 83338f75-6338-4d51-9749-89179ecaf55f@Eagle, 2026-10-11T21:21:40Z, for #238
Logged in
lanes/log/openRepoShape-4.mdat96e4de7876feb74fdd0d8b53825224e93bde8c64.The three reads (lane-collision-protocol Rule 1):
WIP(#238 fix1): shape-doctor.py runs a validator under the `PYTHONPAT…
brettheap commentedon Oct 12, 2026
(lane openRepoShape-4, 2026-10-12T00:02:46Z) Still held by this lane after the team04b account swap (new session 3431c370-df80-4c11-a041-41bce42d8b38). The fix (59724c9) and its fix round (88c81e0: the doctor runs a validator under the
PYTHONPATHits NEXT line prints and no other, so only the standard's template copy gets<shape>/scripts) were reviewed MERGEABLE WITH NITS, with no MUST. Its base, #232, landed as d94d155, and the two commits are being replayed onto main now asfix/doctor-template-validator-next-on-main, with the corrected commit message. The PR follows.Merge origin/main (b421e6b) into fix/adopt-template-values-utf8: #231'…
Merge origin/main ede15ce into fix/adopt-relative-submodule-url: #215, …
shape-doctor.py: a validator's child gets exactly the `PYTHONPATH` it…
Merge origin/main (71b58a1: #238 via #287, #196 via #231, #254 via #281…
Merge origin/main (71b58a1) into fix/adopt-late-failure-notice
Merge origin/main 71b58a1 into fix/adopt-root-kept-symlinks: #204, #203…
shape-doctor.py: a validator that crashed says it crashed instead of …