[brettheap]

Lane: openreposhape-5 (openRepoShape-5)
Claim: #338 (comment)

What was wrong

Two faults, one in the parser and one in the lookup.

• The parser. _link_sources_from_args split every --link-source value at its first =. A link tree under a directory named a=b, passed bare, became a source keyed to a link a that no chain names. Nothing asked for it, so the link stayed declared-unverified, exit 0. The WARNING then offered --link-source openXdox=<path> to the person who had passed exactly that. A mistyped key failed the same silent way.
• The lookup. repo_shape.link_manifest_path asked Path.is_dir() and then is_file(). Before Python 3.14 those swallow only ENOENT, ENOTDIR, EBADF and ELOOP. So each of these ended the naming gate in a traceback, exit 1:
  • a keyed source under a parent this user cannot enter;
  • SHAPE_PIN_SOURCE_<LINK> pointing there;
  • a checkout ../openXdox at mode 000, which every --project run asks for;
  • a 300-character name.

What changed

How a value is read (LinkSources in scripts/validate-repository-naming.py). This mirrors #312's PinSources, with a chain link where that file has a declared pin:

• The text before the first = is a LINK only when it casefolds to a link of a chain this run reads. Those are the entries of --referent-chain, and of each leg's naming.referent_chain: under --project. Both are read before any leg is classified.
• The keyed reading wins even over a directory spelled like the whole value. ./ reads that directory.
• Any other value is a bare PATH, taken whole.

What a lookup says (scripts/repo_shape.py):

• link_manifest_lookup asks stat() inside a guard, as _digest_from_source does in validate-pins.py: a --pin-source value is PRODUCT=PATH only when the text before its first = names a declared neutral-product pin, so a bare stand-in under a=b/ recomputes the digest instead of falling to gh api and a skip, an explicit source that does not exist or is not a directory is the neutral-pin-source-unreadable finding instead of a FileNotFoundError traceback, and a value no pin used is noted (#295) #312. It returns the system's reason when a source cannot answer, or says which of four plain cases it is (no such file or directory, it holds no project.yaml, its project.yaml is not a file, it is neither a directory nor a file), and it never raises. It replaces link_manifest_path, which is gone: nothing calls it. link_declarations gives the reason a manifest could not be read, and declared_neutral_pins, which it left with no caller, is removed (fix round 1).
• resolve_link_source and link_pins_from_trees take a keyword-only tried. It records each source asked, in order, with its reason. Nothing in repo_shape prints.
• link_pins_from_trees also takes a keyword-only with_reasons (fix round 2). Asked, it returns a LinkPins, a dict equal to the plain mapping that also remembers why a tree it reached could not be read. Only the scaffold asks.

What the validator prints (stderr, after each name's WARNINGs):

• WARNING link-source-unreadable: <path> (<how>) could not answer for <link>: <reason>. This covers a --link-source asked for a link that holds a pin, a SHAPE_PIN_SOURCE_<LINK> likewise, and a checkout beside the project that exists and still cannot answer. A keyed value that also names a directory gets a second line saying how to ask for that directory with ./. A bare value holding = says it was read as a PATH.
• NOTE --link-source <typed>: not used — <why>. for a value no such link asked for. The why is one of five: a later value replaced it; no name reads that link's own declaration (a keyed value); no name records a chain at all; no name reads any chain link's own declaration (a bare value, where chains exist and none is read); or every link that is read had a more specific source.
• Three kinds of source are never named: a variable or a checkout asked only for a chain's LAST link (no one ever reads that link's own declaration); a variable or a checkout for a chain no classification reads (fix round 1: a spec leg's chain, a name with no referent, a chain whose shape fails; a --link-source value for such a chain is the NOTE); and a checkout that is absent or holds no manifest.
• --quiet drops both, with the WARNINGs. So make bootstrap and the doctor's naming row print nothing new.

--help, the module docstring and docs/cli.md (regenerated) say all of this.

Fix round 1 (lane 5's review of 40762ca: 0 blockers, 2 should, 4 nits)

Second commit, 4b21263. Each review finding, in order:

• A link manifest whose neutral_product_pins: is not a list of names is a tree that cannot answer (SHOULD-1). link_declarations iterated data.get(key) or [] as it stood, against the link_pins_from_trees docstring this change wrote ("nothing a source does raises out of here"). 5 ended the naming gate, the scaffold and the project's validate-manifest.py in TypeError: 'int' object is not iterable. A scalar openDox was read as the characters D, e, n, o, p, x, so the chain broke, and under --project that is a FINDING. A mapping was read as its keys, so openDox: yes DECLARED openDox. Each is now (None, why), as not a mapping is: declared-unverified, with the reason named ("neutral_product_pins is not a list", "holds an entry that is not text"). An absent or null value, an empty list, and a blank entry (null, '') in a list, declare nothing, as ever. The fix is in the shared function, so the scaffold and validate-manifest.py are reached too.
  Three behaviours change on purpose here, and the tests of fix round 2 pin each:
  • A falsy value is no longer an empty list. neutral_product_pins: 0, false or '' was read as no pin at all (or []), so a chain through the tree was BROKEN, and under --project that is a FINDING. It is now a tree that cannot answer: the link is declared-unverified, a WARNING, exit 0.
  • An entry that is not text is no longer read as text. [openInk, 0] and [openInk, false] dropped the entry as a blank, and [openInk, true] read it as the text True, so a chain through the list VERIFIED. Each is now declared-unverified, with a WARNING saying the list "holds an entry that is not text". That is stricter than the link tree's own validate-manifest.py, which takes str(pin) of an entry (_declared_pins, templates/assembly-root/scripts/validate-manifest.py:85): a chain is better left unverified than verified through a name nobody wrote.
  • The scaffold no longer refuses chain-not-declared over a pins value that is a scalar. With --pin-source a tree whose neutral_product_pins: is the scalar openInk, the scaffold used to refuse (REFUSED chain-not-declared: ... openXdox declares I, e, k, n, o, p, not openInk, exit 2, from the characters). It is now the declared-unverified WARNING and the scaffold goes on, exit 0, which is its documented rule for a tree that cannot answer: "a warning, never a refusal, because the check is offline". The warning says the tree was reached and could not be read (fix round 2, below).
• The reach wording is corrected (SHOULD-2): "Who else this reaches" below, and the docstring of the test that made the claim. What stops crashing is the naming and manifest steps. make validate, validate.yml and make bootstrap still end in a traceback on the sibling row before Python 3.14, at a file this change does not touch (below). On 3.14 the sibling is passed over without a word there instead, which is Every Path.is_file() / is_dir() / exists() guard in the shipped tools answers False to any OSError on Python 3.14 (3.12 and 3.13 raise PermissionError), so a directory this user cannot search reads as ABSENT: validate-pins.py skips its workflow @<sha> check, update-shape.py says copy-missing and update-pin-missing ("adopt the project") for files that are there, the doctor says DRIFTED (5 copy-missing), and bump-leg.py's remaining guards, and every other tool's, skip or misreport instead of refusing by name #430.
• The mutation gaps (nit). Six of the review's ten mutations survived. Five now die, each by a new test: a project.yaml that is a directory; a device or a FIFO as the source; the order A, B, A; a value holding = with no chain at all; and every failure of SHAPE_PIN_SOURCE_<LINK> for a link that holds a pin (a path that is not there, a directory with no manifest, a file that is no manifest), not only a permission. The sixth, the typed in self.used test in unread_lines, could not be killed because it was dead: a value is read as one link's keyed source or is the one bare PATH, so every lookup asks it of the same path and it is used or unread, never both. That was asserted over 293 tests and the test is removed.
• A chain no classification reads (nit). classify consults a link's own declaration only for a name that claims a referent, with a recorded chain whose shape holds. For any other chain (one recorded on a spec leg; one on a name with no open<Product> to reach; one that does not begin at a held pin or does not end at a referent of the name) a source was still asked, so a path that was not there was reported as link-source-unreadable and a good tree was counted used, each for a question nobody put. NamingPolicy.reads_link_declarations now says when resolve_referent reads a chain, and LinkSources.link_pins asks it first: such a chain asks nothing, and a value for it is the NOTE that no name reads that link's declaration. A test runs resolve_referent with a mapping that records whether it was read, over every way a chain is and is not, and holds the new method to it.
• Dead code and stale sentences (nit). declared_neutral_pins is removed: git grep over the tree, every pushed branch and every sibling worktree finds no caller. link_manifest_lookup's docstring no longer says "replaces" over a name that is gone. This body no longer says tests/test_naming_policy.py takes SEARCH_DENIED from validate-pins.py: a --pin-source value is PRODUCT=PATH only when the text before its first = names a declared neutral-product pin, so a bare stand-in under a=b/ recomputes the digest instead of falling to gh api and a skip, an explicit source that does not exist or is not a directory is the neutral-pin-source-unreadable finding instead of a FileNotFoundError traceback, and a value no pin used is noted (#295) #312: it defines a mark of its own, with the same condition and its own reason text.

Fix round 2 (lane 5's re-review of 4b21263: 0 blockers, 0 should, 3 nits)

Third commit, 2ac911c. Each finding, in order:

• The three behaviours the body did not state are stated and tested (nit 1). Round 1's not-a-list fix changed three behaviours on purpose, each listed under SHOULD-1 above, and the re-review found two of them had no test. R1 (if declared is None read as if not declared, which makes 0, false and '' declare nothing again) and R2 (if pin is None or pin == "" read as if not pin, which drops a 0 or false entry as a blank) both passed all 210 cases of tests/test_naming_policy.py at 4b21263. Six rows join NOT_A_LIST_OF_NAMES, which runs at the CLI and in process: 0, false and '' as the value, and 0, false and true among the names. R1 now fails 9 cases and R2 fails 6.
• The scaffold's warning is true for a tree it reached (nit 1, the third behaviour). Over a --pin-source whose neutral_product_pins: is not a list, round 1 turned the chain-not-declared refusal into a declared-unverified WARNING that says the tree "is not reachable here (a sibling checkout, SHAPE_PIN_SOURCE_, or --link-source = would let it be read)". --pin-source did reach the tree, and only its declaration could not be read, so that blamed a path that was right. Now link_pins_from_trees takes a keyword-only with_reasons; asked, it returns a LinkPins, a dict equal to the plain mapping that also keeps unreadable, {link: why} for each link whose source answered and whose manifest could not be read. _chain_link_findings words such a link "which was reached but could not be read as a declaration ()", in a branch of its own, and only the scaffold asks. The new text is asserted for each of the eleven not-a-list rows, with scaffold-project.py --dry-run run as a subprocess.
  • Every other path's text is the bytes it was. The sentence for a link no source reached is the same code in the same lines, and four tests hold it: the generic warning for a plain mapping, for an empty LinkPins and for an absent tree; the scaffold's own old warning for a clone that holds no project.yaml; the naming validator's two lines (its warning, and the link-source-unreadable line that names the reason); and validate-manifest.py's. The naming validator and validate-manifest.py do not ask for reasons, so they print what they printed.
  • One more scaffold path changes with it, on purpose. A tree the scaffold reached and could not read as a declaration for any other reason (a manifest that does not parse, or is not a mapping) used to say "not reachable here" too, and was no truer. It now says what it is, with the system's reason.
  • _declaration still returns its four values, so nothing else in scaffold-project.py moved.
• A qualified chain link in the spy test (nit 2). _chain_read_cases had no case whose chain link is qualified, so reads_link_declarations could take the chain as typed and not through _chain_links (R3) and pass every test. With --referent-chain testorg/openXdox,openDox that turns a verified chain into a declared-unverified one. ("codexDox", {"openXdox"}, ("testorg/openXdox", "openDox"), True) is a case now, and R3 fails it.
• The body (nit 3): the ten corrections of the re-review are applied (the reason a lookup returns, the fifth NOTE reason, the third kind of silent source, the SHOULD-1 bullet, "before Python 3.14", the validate-pins.py bullet, the 3.14.7 rows, the merge-tree section, the filed numbers, and the project's own validate-manifest.py).

The mutation proofs: all three survivors were reproduced first at 4b21263 (210 passed each) and each dies at 2ac911c. Six mutations of the new code each die too: the warning ignoring the reasons (12 cases fail), the lookup recording none (12), the scaffold not asking (11), with_reasons on by default (3), an absent tree called reached (3), and the reached sentence used for a tree nothing reached (17).

Where it departs from #312, and why

• An explicit source that cannot answer stays a WARNING, exit 0. validate-pins.py: a --pin-source value is PRODUCT=PATH only when the text before its first = names a declared neutral-product pin, so a bare stand-in under a=b/ recomputes the digest instead of falling to gh api and a skip, an explicit source that does not exist or is not a directory is the neutral-pin-source-unreadable finding instead of a FileNotFoundError traceback, and a value no pin used is noted (#295) #312 made an unreadable --pin-source a finding. Three reasons for the difference:
  • Exit 1 here means "a name is wrong", and this module has always said an unread link tree is a warning.
  • SHAPE_PIN_SOURCE_<PRODUCT> also answers validate-pins.py's digest question, from a checkout that need not hold a manifest.
  • The lookup falls through to the next source, which may still verify the link.
• The fall-through is kept, as it always was. With --pin-source, a keyed value is final.
• The lines go to stderr, spelled WARNING/NOTE as this file spells them. validate-pins.py: a --pin-source value is PRODUCT=PATH only when the text before its first = names a declared neutral-product pin, so a bare stand-in under a=b/ recomputes the digest instead of falling to gh api and a skip, an explicit source that does not exist or is not a directory is the neutral-pin-source-unreadable finding instead of a FileNotFoundError traceback, and a value no pin used is noted (#295) #312's notes are note lines on stdout, where that file's report lives.

Who else this reaches

The guard is in repo_shape.py, so the same link_pins_from_trees lookup stops crashing for these callers too:

• The project's own validate-manifest.py. It reaches the lookup through SHAPE_PIN_SOURCE_<LINK> or a sibling checkout, and now gives its usual declared-unverified WARNING instead of the traceback.
• scaffold-project.py --referent-chain. A SHAPE_PIN_SOURCE_<LINK> under a locked parent no longer kills the plan, and --pin-source verifies the link.

Neither of these prints the reason a source could not be entered; the scaffold says it only for a tree it reached and could not read (fix round 2, above). A --pin-source the scaffold cannot enter is still a traceback: that is #339 (pin_digest_from_source), and this PR does not touch it.

What this does not reach, and what the doctor says.

• The project's gates still end in a traceback on the sibling row. The NAMING and MANIFEST steps of make validate, validate.yml and make bootstrap stop crashing for a checkout ../openXdox at mode 000. Those targets also run validate-pins.py, whose resolve_neutral_pin_source asks (sibling / ".git").exists() with no guard (templates/assembly-root/scripts/validate-pins.py:463 on main), so before 3.14 a sibling at mode 000 still ends them in a PermissionError. That file is validate-pins.py: a --pin-source value is PRODUCT=PATH only when the text before its first = names a declared neutral-product pin, so a bare stand-in under a=b/ recomputes the digest instead of falling to gh api and a skip, an explicit source that does not exist or is not a directory is the neutral-pin-source-unreadable finding instead of a FileNotFoundError traceback, and a value no pin used is noted (#295) #312's, and fix/pin-source-env-collisions (PR scaffold-project.py, validate-pins.py, repo_shape.py: open-glass and open_glass share one SHAPE_PIN_SOURCE_<PRODUCT>, so --pin refuses the pair and validate-pins.py notes a declared one and reads the variable for neither (#275) #409), fix/adopt-pin-writes-pin-file and open PRs update-shape.py read_pin refuses update-pin-row, and the pinned validate-pins.py and validate-family.py report shape-pin-row, when a shape pin's files: is not a list, so files: 5, true or 1.5 no longer ends check, apply, shape-doctor.py and the project's own validator in TypeError: ... is not iterable (#290) #343 and validate-pins.py (assembly-root template): a workflow file under .github/workflows/ that is not UTF-8, or that this user cannot open, is ONE workflow-unreadable refusal naming every such file, exit 2, so the project's pins gate no longer ends in a UnicodeDecodeError traceback, exit 1, with no line naming the file (#359) #419 touch it, so this PR does not edit it. The same line is listed for Python 3.14 by Every Path.is_file() / is_dir() / exists() guard in the shipped tools answers False to any OSError on Python 3.14 (3.12 and 3.13 raise PermissionError), so a directory this user cannot search reads as ABSENT: validate-pins.py skips its workflow @<sha> check, update-shape.py says copy-missing and update-pin-missing ("adopt the project") for files that are there, the doctor says DRIFTED (5 copy-missing), and bump-leg.py's remaining guards, and every other tool's, skip or misreport instead of refusing by name #430 (lane 2), where the helper that would cure both halves is asked for; the pre-3.14 traceback reproduction is added there as a comment, not filed apart. A SHAPE_PIN_SOURCE_<LINK> under a locked parent is different: validate-pins.py makes that a neutral-pin-source-unreadable FINDING (validate-pins.py: a --pin-source value is PRODUCT=PATH only when the text before its first = names a declared neutral-product pin, so a bare stand-in under a=b/ recomputes the digest instead of falling to gh api and a skip, an explicit source that does not exist or is not a directory is the neutral-pin-source-unreadable finding instead of a FileNotFoundError traceback, and a value no pin used is noted (#295) #312), and that stays.
• The doctor's naming row runs the STANDARD's copy of the validator, so it stops crashing at once. Its manifest row runs the project's OWN validate-manifest.py on the project's own repo_shape.py, so it changes only once the project has applied this change. Over an existing project the doctor says INVALID (manifest, pins) until update-shape.py apply lands the two upstream-changed rows below.
• The verdict. For SHAPE_PIN_SOURCE_<LINK> under a locked parent, on a project that carries these copies, INVALID (naming, manifest, pins) becomes INVALID (pins), and pins is validate-pins.py: a --pin-source value is PRODUCT=PATH only when the text before its first = names a declared neutral-product pin, so a bare stand-in under a=b/ recomputes the digest instead of falling to gh api and a skip, an explicit source that does not exist or is not a directory is the neutral-pin-source-unreadable finding instead of a FileNotFoundError traceback, and a value no pin used is noted (#295) #312's finding for that locked variable, which is right. For a sibling at mode 000 the pins that remains is the traceback above, not a finding.

The two copied scripts this change edits (scripts/repo_shape.py and scripts/validate-repository-naming.py) are copied into every project and pinned there. A project takes the change through update-shape.py (two upstream-changed rows), and a family holder takes the repo_shape.py row. Neither file has a second copy in this repository.

Evidence

• Issue rows, main ab623dd (repro338.sh, a gh stub first on PATH). Under 3.10.21 and 3.12.3, rows 4-7 (keyed under a mode-000 parent, SHAPE_PIN_SOURCE_OPENXDOX there, a sibling at mode 000, a 300-character name) are a traceback, exit 1. Rows 2-3 (a=b, the typo) are a silent declared-unverified, exit 0. Under 3.14.7 row 1 verifies, and rows 2-7 are the silent declared-unverified WARNING, exit 0.
• Issue rows, this PR. Under 3.10.21, 3.11.16, 3.12.3, 3.12.14, 3.13.15, 3.14.7 and 3.15.0rc2 (first commit), all 7 rows exit 0 with no traceback, and the output is byte-identical across versions. a=b verifies. The other five rows each name their source with its reason. Re-run on the fix-round head, 4b21263, under 3.10.21, 3.12.3 and 3.14.7: byte-identical to the first commit's run. Re-run on the fix-round-2 head, 2ac911c, under the same three: all 7 rows exit 0 with no traceback, the output is byte-identical across the three versions, and it is identical to the re-review's runs of the previous head over main.
• Reach on main. Main gives a PermissionError traceback, exit 1, for scaffold --dry-run, the project's validate-manifest.py, validate-repository-naming.py --project and its --quiet form, each with SHAPE_PIN_SOURCE_OPENXDOX under a locked parent. This PR gives exit 0 for all four. validate-pins.py on the sibling row is the exception said above.
• New tests. 17 cases (11 functions) in the first commit; 16 fail on ab623dd. The fix round adds 31 cases in tests/test_naming_policy.py, and 17 of them fail on 40762ca: the five shapes of pins that are not a list (each at the CLI, in-process and, for the scalar, under --project), and the chains nobody reads. The other 14 guard what a mutation could break (below) or the side the fix must not move (an absent, null or blank pin list still answers).
• Targeted run on 4b21263, files test_naming_policy, test_manifest_validator, test_scaffold_referent_chain_links, test_cli_reference, test_repo_hygiene, test_neutral_pin_validator, test_scaffold_e2e, test_shape_doctor, test_git_output_decoding and test_pin_validator: 765 passed, 3 skipped. The 3 skips are the PowerShell-only rows. (40762ca: 734 passed, 3 skipped.) render-cli-reference.py --check is ok.
• Mutation probes, first commit. Reverting the stat guard, the parse, keyed-over-directory, the NOTE lines, the chain's-end restriction, the sibling-absence filter, the manifest reason, the --quiet drop, the "read as a PATH" sentence, the ./ line, the project-chain read, the WARNING lines or the env reporting each fails at least one new test.
• Targeted runs on 2ac911c (fix round 2). Under 3.12.3, test_naming_policy (238 cases: 210 and 28 new), test_cli_reference, test_manifest_validator, test_scaffold_referent_chain_links, test_repo_hygiene and test_setup_project_py: 468 passed; and test_scaffold_e2e, test_scaffold_pin_and_reuse, test_scaffold_leg_path_collisions, test_scaffold_pin_collisions, test_neutral_pin_validator, test_git_output_decoding and test_pin_validator: 312 passed. Under 3.10.21 and under 3.14.7, test_naming_policy, test_manifest_validator and test_scaffold_referent_chain_links: 321 passed each. render-cli-reference.py --check is ok, and ast.parse(feature_version=(3, 9)) passes on the three changed files.
• Mutation probes, fix round. The review's ten, re-run on 4b21263: MW1, MW2, MW3, MW5 and MW11 now die (they survived), and MW4, MW7, MW12 and MW13 still die; MW6 is the dead test, removed. (Fix round 2 kills the three the re-review found, R1 to R3, and six more: see above.) Nine more for the new code (the list check, the entry check, the null and blank tolerances, reads_link_declarations always true, without the shape check, without the held pins, and the validator ignoring it): eight die. The ninth, reads_link_declarations ignoring whether the name has a referent, survives as an equivalent mutant: with no referent _chain_shape_problem's last question always reports a problem, so the explicit early return is redundant, and is kept because resolve_referent has the same first line. The spy test holds the two together.
• Windows. The mode-000 rows skip on Windows and as root, using a SEARCH_DENIED mark this file defines, with the same condition as validate-pins.py: a --pin-source value is PRODUCT=PATH only when the text before its first = names a declared neutral-product pin, so a bare stand-in under a=b/ recomputes the digest instead of falling to gh api and a skip, an explicit source that does not exist or is not a directory is the neutral-pin-source-unreadable finding instead of a FileNotFoundError traceback, and a value no pin used is noted (#295) #312's. The new FIFO and /dev/null rows skip where those do not exist. a=b and the 300-character name run everywhere. On Windows the long-name row asserts the line, not POSIX's reason. None of it was run on Windows or macOS.

Merge-tree

Recomputed at 19:04Z with git merge-tree --write-tree of 2ac911c against current origin/main 53bee5b and against every one of the other 37 pushed branches that touches a file this PR changes (a scan of each head against its merge-base with main):

• Clean: origin/main 53bee5b (it carries scaffold-project.py and adopt-project.py ask the project's YAML reader and validate-manifest.py about the manifest's values before anything exists, so --id null, --tracking-branch true and a blank --elected-by are refused (#258, #261) #314 and repo_shape.py, validate-manifest.py: a bracket in a plain flow scalar is refused and a name holding a lone surrogate is a finding, so pins: [a, b]] is not the pin b] and name: "a\ud83db" no longer kills make validate (#301, #280) #390); adopt-project.py: execute records in project.yaml which leg the plan gave each submodule a leg keeps, and shape-doctor.py holds the submodule to that row, so one in the other leg is MISPLACED and a real adoption's placement reads ok (#189) #394 fix/adopt-record-submodule-placement 563ea16 (scripts/repo_shape.py and validate-manifest.py) and validate-family.py: a gitlink in members/ or a .gitmodules section whose path no row names is a finding in every mode, a hand-edited path: is member-path with nothing read through it and recorded_gitlink reads its path literally, and no git to run refuses git-not-runnable instead of a traceback that exits 1 (#361, #367, #368) #406 fix/validate-family-unnamed-mounts 369af2e (scripts/repo_shape.py); family.py: init asks the holder's reader and validate-family.py about family.yaml before creating anything, --dry-run too, so --id null, --tracking-branch true, a blank --created-by and --created-on 20261011 are refused (#307) #397 abb27e6, family.py: add --dry-run makes every refusal add makes before it mounts, reads the member from throwaway clones outside the holder for the row's id and digest, and prints the row, gitlink, .gitmodules section and trailers without mounting, writing or committing anything, and add takes --trailer as bump does, so every subcommand has the dry run the module's EXIT CODES line promises (#326) #400 7062a47, adopt-project.py: plan --help says what --spec-path and --code-path are, the default and key each records, and the two refusals plan makes on them, so docs/cli.md no longer lists the two flags that choose where a leg is mounted as bare lines (#340) #401 409f10b, family.py: bump and remove put the holder back when a hook, a lock or git refuses them after they write, and say what they put back or what is left with the commands that put it back; they ask first what their dry runs ask first; and remove refuses a held index.lock before it unmounts and a .gitmodules that gives the member's path to two submodules, of which git rm took out one (#365, #360) #416 7cca975, setup.sh: --help from anywhere but a checkout is the shim's own few lines and clones nothing, and render-cli-reference.py refuses a tree that is not a checkout, so asking a download for help is no longer a clone of GitHub (#341) #452 e2df9a8 and templates: project.yaml, the leg pins and family.yaml quote their text placeholders, so --id 007, --tracking-branch 2026, --spec-path true or --org null read back as typed, and family.py add quotes a member id alike (#259) #395 1ef3310 (docs/cli.md); scaffold-project.py and family.py: --tracking-branch is asked of git check-ref-format --branch before anything exists, so ~, a..b, x.lock and a/ are refused under --dry-run and no longer die at the first git init -b after gh repo create has made the repositories (#283) #355 6fdbe3d, shape_materialize.py: RULESET_HINT names the run's tracking branch where it named main, as the pull request's base and the ref the scaffold and the family holder pushed (#324) #373 741cbf2 and templates: project.yaml, the leg pins and family.yaml quote their text placeholders, so --id 007, --tracking-branch 2026, --spec-path true or --org null read back as typed, and family.py add quotes a member id alike (#259) #395 (scaffold-project.py).

• Conflict, four hunks each, all in scripts/repo_shape.py: Q, fix/pin-source-env-collisions (PR scaffold-project.py, validate-pins.py, repo_shape.py: open-glass and open_glass share one SHAPE_PIN_SOURCE_<PRODUCT>, so --pin refuses the pair and validate-pins.py notes a declared one and reads the variable for neither (#275) #409) 6494714, and fix/adopt-pin-writes-pin-file 421e4e5 (no PR). Both also touch scaffold-project.py, and that merges clean with this PR's one edit there. Fix round 2's new hunks merge with neither: Q's change to the "not reachable here" sentence (the offered text, when a variable is shared) and the reached branch sit in different lines of _chain_link_findings. The four hunks, each taking both sides:

  1. the resolve_link_source signature: default_source=None, env_shared: bool = False, *, tried: list | None = None;
  2. its docstring: both paragraphs;
  3. the link_pins_from_trees docstring and the lines after it: both paragraphs, then found, this PR's unreadable and Q's links and shared;
  4. the call: resolve_link_source(name, root, keyed_sources, default_source, pin_product_key(name) in shared, tried=asked) (the same spelling on both branches), followed by if tried is not None: tried[name] = asked.

  Resolved that way in a throwaway and committed there, the merge with 6494714 passed 496 tests (test_naming_policy, test_pin_source_env_collisions, test_neutral_pin_validator, test_scaffold_referent_chain_links, test_manifest_validator, test_cli_reference), and the merge with 421e4e5 passed 559 tests (the same files and test_pin_source_unreadable). render-cli-reference.py --check is ok on both, pyflakes is clean, and the grammar parses as 3.9's. Neither branch carries the resolution. The earlier counts (468 and 434 from the first fix round, 430 from the re-review) are those runs' own, at their heads.

Left out

Each of these was found by lane 5's reviews of this change, and is filed or folded:

• A bare --link-source answers for every link with no more specific source (scripts/repo_shape.py resolve_link_source (:2401-2402) and scripts/validate-repository-naming.py _link_sources_from_args (:408): a bare --link-source <path> answers for EVERY chain link that has no more specific source, so in a chain of three or more links one tree is read as two links' declarations and a correct chain is reported broken, a FINDING (exit 1) for a name that holds no direct pin on its referent #441). In a chain of three or more links, one bare tree is read as two links' declarations, and a chain that holds is reported broken, a FINDING for a name with no direct pin on its referent. Pre-existing, the same on main; validate-pins.py's --pin-source has the same documented rule and was not examined.
• An empty path after a key (--link-source openXdox=) is Path(""), the current directory (scripts/validate-repository-naming.py _link_sources_from_args (:406) and templates/assembly-root/scripts/validate-pins.py PinSources (:409): a key with an empty path (--link-source openXdox=, --pin-source openXdox=) is Path(""), which is the current directory, so run from a project root the naming gate reads the project's OWN project.yaml as the link's declaration and reports a chain that holds as broken, a FINDING (exit 1) for a name with no direct pin #442). Run from a project root, the project's own project.yaml is read as the link's declaration. Pre-existing; validate-pins.py's --pin-source openXdox= has the same shape (there a finding naming .).
• validate-pins.py's (sibling / ".git").exists() is unguarded (the sibling row above), folded into lane 2's Every Path.is_file() / is_dir() / exists() guard in the shipped tools answers False to any OSError on Python 3.14 (3.12 and 3.13 raise PermissionError), so a directory this user cannot search reads as ABSENT: validate-pins.py skips its workflow @<sha> check, update-shape.py says copy-missing and update-pin-missing ("adopt the project") for files that are there, the doctor says DRIFTED (5 copy-missing), and bump-leg.py's remaining guards, and every other tool's, skip or misreport instead of refusing by name #430, which already covers the Python 3.14 half: the pre-3.14 traceback and its reproduction are added there as a comment. validate-pins.py: a --pin-source value is PRODUCT=PATH only when the text before its first = names a declared neutral-product pin, so a bare stand-in under a=b/ recomputes the digest instead of falling to gh api and a skip, an explicit source that does not exist or is not a directory is the neutral-pin-source-unreadable finding instead of a FileNotFoundError traceback, and a value no pin used is noted (#295) #312's file.
• The project's OWN manifest with neutral_product_pins: 5 ends in a TypeError in two gates before either can say anything about it (validate-repository-naming.py (_pins_from_project :302, _chain_from_leg :318), validate-manifest.py (_declared_pins :85) and validate-pins.py (PinSources :390, called at :698) iterate the project's own neutral_product_pins: and a leg's naming.referent_chain: without asking they are lists, so 5 (or true, 1.5) in either field ends the naming gate in TypeError: 'int' object is not iterable, exit 1, neutral_product_pins: 5 ends the other two validators the same way before validate-manifest.py reaches the manifest-neutral-product-pins finding it has for it, and make validate stops in a traceback at its first command; a scalar openXdox in neutral_product_pins: is read as the characters X, d, e, n, o, p, x by the naming gate, which classifies and exits 0 #443): _pins_from_project in the naming gate (scripts/validate-repository-naming.py:302 on main), and _declared_pins in the project's own validate-manifest.py (templates/assembly-root/scripts/validate-manifest.py:85), which ends it before its own manifest-neutral-product-pins finding (_neutral_product_pins_type_finding, :524 on main) can print. Reproduced on 3ef059c. That is the project's own declaration, read by the project's own gates, and not a link source, so it is outside this change; "read by the project's own gates" covers two crashing gates.

Also not done here:

• Python 3.9 (the floor) was not run because it is not installed here. The grammar of the three changed files was checked with ast.parse(feature_version=(3, 9)). Windows and macOS were not run.

Closes #338

🤖 Generated with Claude Code

Summary by Sourcery

Make link-source handling unambiguous and ensure inaccessible or invalid link trees produce actionable warnings instead of tracebacks.

New Features:

• Interpret --link-source values as keyed sources only when their key names a chain link, while preserving paths containing = and reporting unused values.
• Report unreadable link sources and their system-provided reasons as warnings or notes without changing successful validation into findings.

Bug Fixes:

• Prevent link-source lookups from raising tracebacks for inaccessible, invalid, overlong, non-file, or malformed manifest paths.
• Treat malformed neutral_product_pins declarations as unverifiable rather than crashing or inferring pins from invalid data.
• Avoid querying or reporting sources for chains that no classification actually reads.

Enhancements:

• Replace link manifest path probing with guarded, reason-returning lookups and optional source-trial diagnostics.
• Improve scaffold warnings to distinguish trees that were reached but whose declarations could not be read from trees that were unreachable.

Documentation:

• Update CLI documentation and help text with source parsing, diagnostics, and warning behavior.

Tests:

• Add comprehensive coverage for source parsing, unreadable paths, malformed manifests, unused sources, unconsulted chains, and scaffold diagnostics across supported environments.

Chores:

• Remove obsolete manifest lookup helpers and align shared source handling across naming validation, manifest validation, and scaffolding.

[brettheap]

(lane openRepoShape-5, 2026-10-12T19:11Z) @codex review

[sourcery-ai]

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 4 days and 2 hours by commenting @sourcery-ai review. Upgrade to get a review now.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[sourcery-ai]

Reviewer's Guide

The PR makes link-source interpretation chain-aware, hardens repository and manifest lookup against filesystem and data-shape errors, and reports unread or unused sources as non-failing diagnostics while preserving verification fall-through; it also updates scaffold behavior, documentation, and extensive regression tests.

Sequence diagram for chain-aware link-source resolution

sequenceDiagram
    participant CLI as Naming validator
    participant Sources as LinkSources
    participant Policy as NamingPolicy
    participant Shape as repo_shape
    participant FS as Filesystem

    CLI->>Sources: LinkSources(values, chain_links)
    Sources->>Sources: classify each value as keyed LINK=PATH or bare PATH
    CLI->>Sources: link_pins(policy, target)
    Sources->>Policy: reads_link_declarations(name, pins, chain)
    Policy-->>Sources: whether classification reads link declarations
    Sources->>Shape: link_pins_from_trees(chain, sources, tried)
    loop each chain link
        Shape->>Shape: resolve_link_source(link, tried)
        Shape->>FS: link_manifest_lookup(source)
        FS-->>Shape: manifest or system reason
        alt manifest available
            Shape->>Shape: link_declarations(manifest)
            Shape-->>Sources: declared pins
        else source or declaration unreadable
            Shape-->>Sources: fall through or record reason
        end
    end
    Sources-->>CLI: link pins plus source diagnostics

Loading

Sequence diagram for source diagnostics and scaffold warnings

sequenceDiagram
    participant Validator as Naming validator
    participant Sources as LinkSources
    participant Shape as repo_shape
    participant Scaffold as scaffold-project
    participant Report as stderr report

    Validator->>Sources: link_pins(..., tried=tried)
    Sources->>Shape: link_pins_from_trees(..., tried=tried)
    Shape-->>Sources: pins and attempted source reasons
    Sources->>Sources: record used, unread, and unanswered values
    Validator->>Report: unread_lines()
    Report-->>Validator: WARNING link-source-unreadable
    Validator->>Report: unused_lines()
    Report-->>Validator: NOTE source not used
    Scaffold->>Shape: link_pins_from_trees(..., with_reasons=True)
    Shape-->>Scaffold: LinkPins with unreadable reasons
    Scaffold->>Report: report reached-but-unreadable declaration
    Report-->>Scaffold: declared-unverified WARNING

Loading

Flow diagram for non-raising link manifest lookup

flowchart TD
    A[Source path] --> B[stat inside guard]
    B -->|missing or invalid path| C[Return system reason]
    B -->|directory| D[stat project.yaml]
    B -->|regular file| E[Use source as manifest]
    B -->|other file type| C
    D -->|missing| F[Return no project.yaml]
    D -->|not regular file| G[Return project.yaml is not a file]
    D -->|stat or filesystem error| C
    D -->|regular file| E
    E --> H[load_yaml and validate mapping]
    H -->|neutral_product_pins absent or empty| I[Return empty declarations]
    H -->|not a list or non-text entry| J[Return declaration reason]
    H -->|valid list of names| K[Return declared pins]

Loading

File-Level Changes

Change	Details	Files
Make --link-source parsing depend on known chain links rather than blindly splitting on '='.	Added LinkSources to resolve CLI and project referent-chain links before classifying targets. Treat only matching chain-link prefixes as PRODUCT=PATH; preserve other values, including paths containing '=', as bare paths. Track replacement, usage, unreadable, and unused source values for diagnostics, including ./ disambiguation and --quiet behavior.	scripts/validate-repository-naming.py docs/cli.md
Make link-source and manifest lookup failure-safe while preserving fall-through semantics and exposing reasons.	Replaced is_dir/is_file probing with guarded stat-based lookup covering permission, path-length, device/FIFO, and malformed manifest cases. Return explicit reasons for inaccessible sources, missing manifests, invalid manifest types, and invalid neutral_product_pins values without raising. Add ordered tried-source tracking and optional LinkPins unreadable metadata for callers such as the scaffold.	scripts/repo_shape.py
Prevent unrelated or unreadable chains from producing incorrect classification failures and improve scaffold reporting.	Added reads_link_declarations to avoid querying chains that no classification uses. Treat non-list or non-text neutral_product_pins declarations as declared-unverified warnings rather than iterating invalid data or creating findings. Update scaffold warnings to distinguish a reached tree whose declaration could not be read from a tree that was never reachable.	scripts/repo_shape.py scaffold-project.py
Add broad regression coverage for parsing, lookup safety, diagnostics, malformed declarations, and downstream callers.	Cover equals-containing paths, keyed-versus-directory precedence, mistyped and replaced values, source usage notes, quiet mode, environment/sibling sources, and inaccessible or invalid paths. Cover malformed neutral_product_pins values, qualified chains, chains not read by classification, and reached-versus-unreachable warning text. Exercise naming validation, manifest validation, scaffold dry runs, direct lookup APIs, and CLI documentation generation.	tests/test_naming_policy.py docs/cli.md

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#338	Interpret each --link-source value as PRODUCT=PATH only when PRODUCT names a chain link; otherwise treat the whole value as a bare path, with ./ available to force the path reading.	✅
#338	Report every --link-source value that no link used, including why it was unused, without turning it into a finding.	✅
#338	Handle unreadable or invalid keyed, bare, environment, and sibling sources without tracebacks, reporting the system’s reason as a warning; document the behavior in help and the CLI reference.	✅

Possibly linked issues

• scripts/validate-repository-naming.py: _link_sources_from_args splits every --link-source value at its first = and nothing reports a value no chain link used, so a link tree under a directory named a=b, or behind a mistyped key, is silently skipped (declared-unverified, exit 0), and repo_shape.link_manifest_path lets a source this user cannot enter (keyed, SHAPE_PIN_SOURCE_<LINK> or a sibling checkout) end the naming gate in a PermissionError traceback before Python 3.14 #338: The PR fixes both reported faults: recognizes keyed sources only for chain links and guards lookups, reporting failures as warnings with reasons.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sonarqubecloud]

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud