Repository navigation
Both roots' scripts/bootstrap.py: main escapes what stdout cannot encode and spells each value it prints, so name: "a\ud83db" or 日本 under PYTHONIOENCODING=ascii no longer ends make bootstrap in a traceback (#391) - #456
Conversation
…t/scripts/bootstrap.py: `main` escapes what stdout cannot encode and spells each manifest, register and member value it prints, so `name: "a\ud83db"`, or `日本` under `PYTHONIOENCODING=ascii`, is no longer a traceback before the validators run, and an ESC or CR in a name no longer reaches the terminal (#391) `main` opened with `print(f"bootstrap: {name} ({root})")`, before any step, and the family root's with the same line. `name: "a\ud83db"` reads as a string holding a lone surrogate, which no UTF-8 stdout can write: `scripts/bootstrap.py` exited 1 with a `UnicodeEncodeError`, `make bootstrap` exited 2, and `validate-manifest.py`, the validator that names the problem, was never reached. `日本` under `PYTHONIOENCODING=ascii` ended the same way. The family root's line also holds a literal dash, so under that stdout it raised for every name, `InkRouter` included. A raw ESC or CR in a name is encodable on every stdout and reached the terminal as the control it is. Both files are shape copies, digest-pinned in the project that holds them, and a project carries `scripts/repo_shape.py` and no `shape_materialize.py`, so neither can call #214's helper. Each file now carries the same two functions: - `escape_unencodable_stdout`, the first statement of `main`. A `strict` or `surrogateescape` stdout is reconfigured to `backslashreplace`, the handler stderr already has, so what it holds is written as it was and what it cannot hold as `日`. Any other handler is left alone and the encoding is never changed. It is `validate-pins.py`'s guard from #295, which is inline there and cannot be imported (a hyphenated name and a validator). - `spelled`, which writes every character `str.isprintable` refuses as the escape Python spells it (`\x1b`, `\r`, ` `, `\ud83d`) and keeps every printable one, a backslash included, so a name a UTF-8 stdout can hold prints byte for byte. `repo_shape.double_quoted` was read first and cannot serve: it quotes, and escapes `"` and `\`. Every print of a value the script read from the manifest, a register row or the disk is spelled: the name and the root on the first line; a leg's role, path and tracking branch in step (a); a register row's holder, act, objects, state and expiry, its path, and the reader's reason when it cannot be read; in the family root, a member's project and directory name and `--members-make`. The member's own command is still handed the raw target. Output for a name every stdout holds is unchanged: the new tests spell out the whole of a plain project's and a plain holder's output. What the run does next is not this change's. `validate-manifest.py` still prints its own `manifest ok: <name>` through a stdout that has no handler, so on a tree without #280's fix the lone-surrogate row now ends in the validator's traceback and `bootstrap FAILED: manifest (exit 1)`, and with it at `FINDING manifest-name`. Tests: tests/test_bootstrap_spelling.py (new). Each file is run as a real project and a real holder under six stdouts; every other print above has a row; the two functions and the identity of their two copies are tested directly. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…escaped Main has moved past this branch's base (cd1fc79), and #390 (cb7063f, the #280 validator half) has landed, so a lone-surrogate name now ends the run at `FINDING manifest-name` instead of validate-manifest.py's own traceback. The merge is clean and touches neither bootstrap.py template. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t/scripts/bootstrap.py: `spelled` escapes the controls, the surrogates, the line and paragraph separators and the bidirectional controls and nothing else, so a Persian name with its `\u200c`, an emoji sequence with its `\u200d`, and a name with `\u3000`, `\xa0` or `\xad` print byte for byte as they did before #391 (#391) Review of the first head found that `spelled` wrote as an escape every character `str.isprintable` refuses. That is wider than a control, a surrogate or a line separator: it also refuses every space but ASCII space, every format character and every unassigned code point. So a valid name that `validate-manifest.py` accepts, and prints raw a few lines below, was escaped by bootstrap's own first line: `\u200c` (the zero-width non-joiner a Persian name needs), `\u200d` (the joiner of an emoji sequence), `\u3000` (the ideographic space between a Japanese family name and a given name), `\xa0`, `\xad` and `\u200f`. The `git submodule update --init <path>` command printed for a leg path holding one of them was no longer right to paste, and what was escaped moved with the interpreter's Unicode database. The test is now the category (the coordinator's decision; Brett may reverse it). `spelled` writes the escape Python spells for EXACTLY these characters: - the controls (Cc); - the surrogates (Cs); - the line and paragraph separators (Zl, Zp); - the bidirectional controls `\u061c`, `\u200e`, `\u200f`, `\u202a` to `\u202e` and `\u2066` to `\u2069`, so a name cannot reorder the text around it on a terminal. Every other character prints as it did before #391. The four categories are fixed ranges, so the answer no longer depends on the Unicode database. The two sets are module-level constants in both copies, and the identity test compares them with the two functions. `unicodedata` is the standard library's. Tests: `tests/test_bootstrap_spelling.py` pins each newly raw character (`\u200c` in a Persian name, `\u200d` in an emoji sequence, `\u3000` in a Japanese full name, `\xa0`, `\xad`, and a zero-width space with a byte-order mark) as the bytes a plain `print` wrote on the base, in a scaffolded project and in a family holder, and with the validators' own `manifest ok:` line beside it. `\u202e`, `\u200f` and `\u2028` are pinned as escaped. Every code point is swept once against a set written from the specification, which is itself checked against `unicodedata.category`. The lone-surrogate rows now assert `FINDING manifest-name` unconditionally, since #280 landed in #390 (cb7063f), and the `make bootstrap` row asserts there is no `Traceback`. Reverting the set to `isprintable` fails 59 of the new rows. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
(lane openRepoShape-5, 2026-10-12T19:28Z) @codex review |
|
Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 1 day and 23 hours by commenting |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Reviewer's GuideBoth bootstrap templates now avoid stdout encoding tracebacks and terminal control injection by safely configuring stdout and spelling only hazardous Unicode in values they print, while preserving raw values for operations and adding extensive cross-encoding regression tests. Sequence diagram for encoding-safe bootstrap outputsequenceDiagram
participant User
participant Bootstrap
participant Stdout
participant Validator
User->>Bootstrap: main(argv)
Bootstrap->>Stdout: escape_unencodable_stdout()
alt strict or surrogateescape handler
Bootstrap->>Stdout: reconfigure(errors=backslashreplace)
else other handler
Bootstrap->>Stdout: keep encoding and handler
end
Bootstrap->>Bootstrap: spelled(manifest values)
Bootstrap->>Stdout: print escaped hazardous values
Bootstrap->>Validator: run validation steps
Validator-->>Bootstrap: report finding without bootstrap traceback
Flow diagram for spelling bootstrap output valuesflowchart TD
A[Value read from manifest register or disk] --> B{Hazardous character?}
B -->|Cc Cs Zl Zp or bidi control| C[Convert with unicode_escape]
B -->|Any other character| D[Preserve raw character]
C --> E[Print value]
D --> E
E --> F[Continue bootstrap and reach validators]
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
…ut writing bytecode into templates/, so the template tree a later test reads holds no .pyc CI run 38297896001 (Linux and macOS) failed three cases of `tests/test_scaffold_leg_path_collisions.py::test_the_paths_refused_are_the_paths_the_materializer_writes` ([no-pin], [one-pin], [two-pins]) with a `UnicodeDecodeError` on a CPython `.pyc` header: the test reads every file under `templates/assembly-root` as UTF-8 text, and there was a `.pyc` in it. This file wrote it. Its `load()` runs each root's `scripts/bootstrap.py` with `SourceFileLoader`, which caches the bytecode it compiles in a `__pycache__` beside the source, so the module-scoped `template` fixture left `templates/<role>-root/scripts/__pycache__/bootstrap.cpython-3XX.pyc` in the tree. `.gitignore` ignores `__pycache__/`, so `git status` never showed it, and `test_scaffold_leg_path_collisions.py` sorts after this file, so it read it. Reproduced locally on CPython 3.12 with the two files in one process, in that order: 3 failed, 361 passed. A tree a materializer walks is open to the same stray. `load()` now sets `sys.dont_write_bytecode` around `exec_module` and puts it back in a `finally`. It keeps importlib and the module's `__spec__` as they were, and the flag also covers the `repo_shape` a root imports as it runs, which compiling the source into a bare `types.ModuleType` would have left to the normal import machinery. One regression test, `test_loading_a_root_leaves_no_file_where_it_was_loaded_from`, snapshots every path under `templates/`, loads both roots, and asserts the snapshot is unchanged. It does not ask that no `__pycache__` exists, because a checkout may hold a stale one from an older run. A tree that already holds the cache would pass for any `load`, so each root is also loaded from a copy in a directory that holds nothing. With the flag removed, the test fails on a fresh tree (the real-tree assertion) and on a stale one (the copy assertion). Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(), which both roots' bootstrap.py now import instead of defining twice (#391) Why. SonarCloud failed PR #456 on duplication (14.7% on new code against a 3% gate): spelled() and escape_unencodable_stdout(), with the two sets spelled() reads, were written out once in each root's scripts/bootstrap.py, and a test held the two copies identical. The premise behind the copies, that a root cannot import anything of this repository's, was false: every project and every family holder carries scripts/repo_shape.py, the pinned shared library, and both bootstraps already import Refusal, find_repo_root and load_yaml from it. Brett Heap's decision (2026-10-12): move the two helpers into repo_shape.py and have both bootstraps import them; no Sonar exclusion. Where. A new section at the END of scripts/repo_shape.py, after die(): ESCAPED_CATEGORIES, ESCAPED_BIDI_CONTROLS, escape_unencodable_stdout() and spelled(), copied from the bootstrap source byte for byte. Their code is unchanged (the AST of each, docstrings aside, equals both old copies); only the docstring sentences the move made untrue are reworded ("this file", "this script", "the same in the assembly root's bootstrap.py and the family root's"). `import unicodedata` joins the imports after `import sys`, and the module docstring's "WHAT THIS MODULE OWNS" list gains one bullet after `Refusal`. Those three places are clear of the open PRs that also edit this file (#394, #409, #416, #453): each of them merges with this commit without a conflict in repo_shape.py. Both templates/*/scripts/bootstrap.py lose the two definitions, the two sets and `import unicodedata`, and add the two names to their existing `from repo_shape import` line (the family root's becomes the parenthesised form). escape_unencodable_stdout() is still main's first statement, and the module level is still only `sys.path.insert` and the import. Against origin/main, each bootstrap now differs only in that import line and the spelled() call sites. Tests (tests/test_bootstrap_spelling.py). The test test_the_two_roots_carry_the_same_function is replaced by an ast check that each root's `from repo_shape import` names both and that nothing in the file defines or assigns either function or either set; one more test per root asserts that the name its module exposes IS repo_shape's object. The unit rows of spelled() (keep-raw, escape, the full code-point sweep, the non-string values, the one-line check) and of escape_unencodable_stdout() now run once against repo_shape itself instead of once per root. The end-to-end rows run a real scaffolded project and a real family holder, which carry scripts/repo_shape.py, so they exercise the moved code where it ships; they and the bytecode regression test are unchanged. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
36538fd..437a360 is #355, #401, #456 and #343; #401 adds `_leg_path_help` and the `--spec-path`/`--code-path` help to adopt-project.py, away from every line this branch changes. The merge has no conflict. Lane: openreposhape-3 (openRepoShape-3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s cite #343, #401 and #456, drop the known limits main no longer has (#290, #391, #225, #294), list all 30 open milestone issues, state the Windows park-and-resume exception beside the 1.0 promise, and give a pin procedure that pastes into macOS's zsh (#472) A fact-check of cbe44fe found nothing wrong at the drafting sha, but main had moved. Three PRs landed and fourteen milestone issues were filed in the forty-five minutes after the draft, and several sentences were true only with a qualifier the notes had dropped. This commit refreshes the notes against 437a360 and takes the checker's corrections. Refresh: - Section 5 adds #343 (`update-pin-row` and `shape-pin-row` for a shape pin whose `files:` is not a list), #401 (`plan --help` for `--spec-path` and `--code-path`) and #456 (`make bootstrap` escapes what the console cannot print). - Section 6 drops #290 and #391, which those PRs closed. - Section 6 also drops #225 and #294. Both are still open, but offline probes at 437a360 show #314 fixed them: - `scaffold-project.py --spec-path null --dry-run` refuses `scaffold-manifest-invalid` before anything exists; - `adopt-project.py check` on a plan with `elected_on: someday` and no `reference:` refuses `election-date-malformed`. - #181 now names the input that still reaches it, a source file named `null`. The probe showed `plan` writing `path: null` and `check` then finding the path uncovered. - New unfiled checklist items join section 6: - #458's `ImportError` item, now that #456 is on main; - #457's L2-6; - the `CONTRACTS_DIR` item; - four adoption inputs that cannot be tested on Windows. - Section 7 is rebuilt from the milestone, which now holds 30 open issues. #340 is closed, so its line is gone. Corrections: - **Section 1** states the promise as ruled, then says that on Windows park and resume run under WSL2, because native Windows parks nothing (README). - **The Windows CI sentence** says the adoption suites skip there until #471 lands, with a comment to drop the qualifier then. - **The pin procedure** ends its push line without a `# comment`. An interactive zsh without `interactivecomments` would hand the comment's words to `git push` as refspecs, the class #277 reports. The pull request step is in prose after the block. The section also says that the shape pin always records a commit, and that `apply` counts the files it re-copies before it asks. - **The YAML reader** sentence names `yaml-unparsable` beside `yaml-unreadable`, and claims no more than the reader does. - **The doctor** sentence keeps AGENTS.md's qualifier, "into the repository", and says that outside a checkout it clones the standard first. - **#303** gives both outcomes. **#389** is described as the gap that #305's refusal covers. - **The hermetic-tests** sentence claims only what its PRs and AGENTS.md establish: no test reaches the real `gh`, and the suite scaffolds into local bare repositories. Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_spelled` now spells every symlink this branch names, and #283's tracking-branch question is not asked by adopt-project.py, so it meets none of this branch's refusals Brings in, by sha, main's ten commits since 75052cb: ec8e09a (#406), 580d7df (#416), 1184f67 (#404), d6df36b (#346), 37665ff (#452), 36538fd (#378, #329), 6c2a371 (#355, #283), 8782333 (#401, #340), ec4a7a0 (#456, #391) and 437a360 (#343, #290). No conflict. The tree is `git merge-tree --write-tree 437a360 95f6fc7`'s, ae7be0a. Three files changed on both sides merged by git alone: - adopt-project.py: 36538fd rewrites `_spelled`'s docstring and `_spelled_character` (a byte that is not UTF-8 is `\udcNN`, a character `\xNN`, by `repo_shape.double_quoted`), and two docstrings (`_refused_by_policy`, `_index_entries`); 8782333 adds `_leg_path_help` and the `help=` of `plan`'s `--spec-path` and `--code-path`. This branch's hunks are elsewhere. - scripts/shape_materialize.py: 6c2a371 adds `import tempfile` and the branch-name question (`_ask_git_branch`, `_branch_verdict`, `branch_problem`, `refuse_bad_branch`) after `run`. This branch's hunks are elsewhere, and it adds no import of `tempfile` to either file (adopt-project.py has had one since before 75052cb), so no name is bound twice. - tests/test_adopt_finding_spelling.py: 36538fd's `\udcfe` lines and this branch's `follows too many symlinks` line are more than 100 lines apart. How main's changes meet this branch's code: - 36538fd: every source path and symlink target this branch's findings and the materializer's refusals name is spelled by `_spelled` (`execute` hands it to the materializer as `spell`), so they now spell a byte that is not UTF-8 as `\udcNN`, as the plan's own `path:` does. One test of this branch pinned the old spelling, `nowhere\xff`; the next commit pins `nowhere\udcff`. Nothing in this branch has an escape table of its own. - 6c2a371: `refuse_bad_branch` is called by scaffold-project.py and scripts/family.py only. adopt-project.py's `--tracking-branch` is not asked of git (6c2a371 leaves it as its own change), so the question falls nowhere among `check`'s findings or `execute`'s refusals. - 8782333: `plan --help` text only; `plan` refuses nothing new, and the help names `shape/` as where `execute` writes its copy, which this branch does not move. Checked on the merged tree: no top-level name in adopt-project.py, scaffold-project.py, scripts/repo_shape.py, scripts/shape_materialize.py or scripts/family.py is bound twice that was not already bound twice on main (shape_materialize.py's `YamlError` and `parse_yaml`, as on main), no name is loaded and bound nowhere, and each side's own change reappears whole in the merge (no line of either side lost). Lane: openreposhape-1 (openRepoShape-1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… conflict with #394 (#225) 7a8b981 (#394, issue #189) moved a leg path's segment rule into `repo_shape.path_spelling_fault` and added it, with `PATH_CLIMBS`, `PATH_ITSELF` and `PATH_NOT_CANONICAL`, to the `from repo_shape import (...)` list in scripts/shape_materialize.py, the list this branch had added `parse_yaml` to. GitHub could no longer update the branch. The one conflicting hunk is resolved as main's list. The branch's `parse_yaml` is left out because main has imported it at the top of the module since #380 (`from repo_shape import YamlError, _strip_comment, parse_yaml`), and `_read_back` uses that binding. Naming it in both imports was a redefinition that pyflakes reported. With this merge the pull request changes no import line. Everything else merged without a conflict and keeps both sides. From main: the `EXCEPT THE SEGMENTS` comment, `LEG_SEGMENT_REASONS`, and `_spelling_reason` asking `path_spelling_fault`. From this branch: the #225 paragraph, `_leg_path_lines`, `_misread_reason`, and the read-back step in `leg_path_problem` and `_canonical_spelling`. The read-back is still asked last, after the spelling, Git's and Windows' names and the alphabet. 7a8b981's `submodules:` record adds no line that holds a leg path. Each row's `path:` is a submodule's path inside its leg, and `leg:` is `spec` or `code`. `execute` appends the block in code instead of rendering it from a template, and it double-quotes every path. The rule still finds the four real template lines. An adoption at `--spec-path legs/v2 --code-path c0de` with a recorded submodule writes a leg path on those four lines only, and its bootstrap passes 7a8b981's validator. The merge also brings in ec4a7a0 (#456), 437a360 (#343) and d222a12 (#397). None of them touches a file of this pull request. Lane: openreposhape-2 (openRepoShape-2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… writes its pin file as the scaffold does, so an adopted root passes `validate-pins.py` (#273); a `--pin-source` that cannot answer is `pin-source-unreadable` (#339) (#508) **Brought up to date after #409 squash-landed** by merging origin/main (9736424, #409's squash) as f9ed46f. Its `scaffold-project.py` conflict was resolved to this branch's side once `git diff b70a07d 9736424 -- scaffold-project.py` (#409's final head against its squash) came out empty. So the file is the one tested on this branch plus main's own later changes to it, and holds nothing of #409's; details below. This branch carries #275's adopt half. It was built on `fix/pin-source-env-collisions`, PR #409 (#275's scaffold half and its fix round), merged at 6494714 as 583f647, over the earlier 5690235 (34dbd86). After 6494714, #409 only merged main, five times, up to its final head b70a07d, and none of those merges has a hand edit (`git show --remerge-diff` is empty for each). Adopt's refusal of two products that share a source variable goes through #409's function, so this branch lands after #409. It was never rebased. `git diff b70a07d 9736424` is empty for each file #409 touched: `scaffold-project.py`, `scripts/repo_shape.py`, `templates/assembly-root/scripts/validate-pins.py` and `tests/test_pin_source_env_collisions.py`. What f9ed46f resolves, four files with one hunk each: - **`scaffold-project.py`.** - The conflict hunk is main's block from `PIN_COLLISION_REMEDIATION` through `_pin_collision_remediation`, which #409's squash edits: it adds `PIN_VARIABLE_REMEDIATION`, `_colliding_pin_variables` and `_pin_collision_remediation` and changes `_refuse_colliding_pins`. This branch moved all of that to `scripts/shape_materialize.py`. The hunk takes this branch's side. (The merge commit's message says "re-adds"; this is the exact account.) - #409's auto-merged import of `PIN_SOURCE_ENV_PREFIX, pin_source_env_collisions` is dropped, because nothing in the file uses either name any more. - It is not `git checkout --ours` of the whole file. Main changed the file beyond #409 since this branch last merged it: #355's `--tracking-branch` check (6c2a371) and a73b4e9's `RULESET_HINT`. Those lines stay. - The result is the file `git merge-tree` gives for this branch with main before the squash (518010b). Its diff from the file tested here has the same patch-id as main's own 3ef059c..518010b change to it. - **`templates/assembly-root/scripts/validate-pins.py`.** Main's side, which is #409's final blob, with main's #359 workflow reader in it. This branch had no change of its own to the file beyond #409. - **`adopt-project.py`, `_commit_the_split`.** - Main: #445 wraps the function in its late-failure `try`, and #394 builds an `append` with the plan's submodule record. - This branch: its only edits there were a comment and `neutral_pins=_recorded_pins(plan)` on the materializer call. - Both now sit on main's call inside the `try`: `append=append, neutral_pins=_recorded_pins(plan)`. - **`docs/cli.md`.** It is regenerated with `scripts/render-cli-reference.py`, and `--check` is ok. It differs from main's only in the `--pin`, `--pin-owner` and `--pin-source` help of `adopt-project.py plan`. `scripts/repo_shape.py`, `templates/assembly-root/scripts/validate-pins.py` and `tests/test_pin_source_env_collisions.py` are main's blobs, which are #409's: this branch adds nothing to them. Nothing of #409 is lost. Its scaffold behaviour is `shape_materialize.refuse_colliding_pins` and `pin_variable_collisions`, keyed by #409's `pin_product_key`. **73d65c6, after the merge.** - **The change.** `_read_pins` reads `plan.get("pins")` where it read `plan.data.get("pins")`. - **Why.** #445's `tests/test_adopt_late_failure.py` hands `_commit_the_split` a `SplitPlan` as its plan, a `dict` that adds `entries`. On f9ed46f, three of its tests stopped at "committing the split" with `AttributeError: 'SplitPlan' object has no attribute 'data'` (3 failed, 83 passed in that file). - **Nothing else changes.** `Plan.get` with no default returns `plan.data.get(key)` exactly, so a loaded plan reads as before. `adopt-project.py plan --pin` took a NAME only (`--pin openGlass`) and refused a trailing `@<commit>`. `plan`, `check` and `execute` each exited 0, and `execute` printed `adoption verified`. The manifest then listed the name in `neutral_product_pins:` with no `contracts/<product>-pin.yaml` beside it, and on its first run the adopted root's own `validate-pins.py` refused it as `neutral-pin-missing`. #273's table, reproduced on c7c02fa with the issue's own commands (local bare remotes, and a stub `gh` that exits 97): | `--pin` | c7c02fa | this branch | |---|---|---| | `openGlass` | `execute` exit 0, `- openGlass`; clone's `validate-pins.py` exit 2 `neutral-pin-missing` | `plan` exit 2 `pin-malformed`, with the scaffold's remediation (`--pin openGlass@<40 hex>`); no plan is written | | `opensoft/openGlass` | exit 0, `- opensoft/openGlass`; exit 2 `neutral-pin-missing` for `contracts/opensoft/openglass-pin.yaml` | `plan` exit 2 `pin-malformed` | | `spec` | exit 0; `validate-pins.py` exit 0, reading the leg's `contracts/spec-pin.yaml` as the neutral pin | `spec@<sha>` is `adopt-pin-collides` at `plan` | | `shape` | exit 0; `contracts/shape-pin.yaml` read as the neutral pin | `shape@<sha>` is `adopt-pin-collides` at `plan` | | `openGlass@<sha>`, `opensoft/openGlass@<sha>` | `plan` exit 2 `adopt-pin-malformed` | `plan`, `check` and `execute` exit 0, and `gh` is called 0 times. `contracts/openglass-pin.yaml` is written and the manifest lists `- openGlass`. In the clone, `validate-manifest.py` exits 0, and `validate-pins.py` exits 0 with `openGlass: pin commit … digest recomputes` | Brett ruled for decision 1's first option ("go with your recommendation on #273"): adopt takes the scaffold's pin and writes the scaffold's pin file. That reverses the NAME-only `--pin` from #6, and the comment that described it is replaced with the new contract. - **One implementation.** The scaffold's pin code moved into `scripts/shape_materialize.py`, next to `neutral_pin_path` and the materializer that writes the file. What moved: - `PIN_ARG_RE`, `parse_pin`, `pin_digest_from_source` and `pin_digest_from_gh`; - `declared_pin_values`, `declared_pin_products` and `PIN_COLLISION_REMEDIATION` (in #309's wording); - from #275's branch, `PIN_VARIABLE_REMEDIATION` and the variable check; - `_refuse_colliding_pins`, split into the pure `pin_collisions` and `pin_variable_collisions`, and `refuse_colliding_pins(refusal=...)`, which raises one refusal for both. The code did not move verbatim. `_refuse_colliding_pins` became the three functions above; `pin_digest_from_source` was rewritten for #339 (below); and fix round 1 changed three of them (below): `pin_variable_collisions` keys each pin by #275's `pin_product_key`, `pin_digest_from_source` spells a path that is not printable, and `pin_digest_from_gh` refuses a missing `gh`. The scaffold's behaviour is otherwise unchanged. `scaffold-project.py` imports what it calls under its old names. Lane 5's review ran 34 scaffold cases (every pin refusal, the `gh` stub's four modes, dry and real successes) on 34dbd86 and on this branch at 7a94b6b. rc, stdout, stderr, the `gh` log and the files made were identical, apart from the shape's own commit and digest and #339's row. Its pin tests and #275's pass as they are. `tests/test_git_output_decoding.py`'s exemption for `pin_digest_from_gh` follows the function to its new file. - **`plan`.** `--pin`, `--pin-owner` and `--pin-source` mean what they mean for the scaffold. `plan` reads the digest from `--pin-source` or `gh api`, and records product, repository, commit, digest and `digest_source` in the plan, so `check` and `execute` read no network for a pin. `plan` refuses: - a name with no `@<commit>`, as `pin-malformed`. That includes an empty `--pin ''`, which adopt used to drop and now refuses, as the scaffold always has; - `spec`, `code` and `shape`, in any case, as `adopt-pin-collides` (#273 decision 4), in the scaffold's sentence and remediation (#241, #309); - one product given twice, the same way, whether it is a repeat or two names that share one file (decision 5); - two products that one `SHAPE_PIN_SOURCE_<PRODUCT>` names, such as `open-glass` and `open_glass`, the same way, through #275's check; - a leg path over a pin's file, as `adopt-leg-path-collides`; - a `--pin-source` path that `check` would refuse in `digest_source:` (fix round 1), as `adopt-unwritable-value`. It is asked only when a `--pin` records the path; - with no `gh` on PATH and no `--pin-source`, `gh-not-found` (fix round 1). - **`check` and `execute`.** These are new `check` findings, and `execute` refuses each one before any leg exists. They come from `_leg_findings`, the list `check` and `_refuse_what_check_finds` share: - `plan-pin-unrecorded`: a name alone (an old plan's `pins: [openGlass]`, or a hand edit), or an entry missing its commit or digest; - `plan-pin-malformed`; - `plan-pin-collides`, which covers two pins at one file, a pin at a file the shape writes, and two products that share one source variable; - `plan-root-holds-pin`: a source path kept in the root at the pin's file. The materializer would write the shape's pin beside it under `shape/`, and `validate-pins.py` would read the source's file instead. How each finding ends: - `plan-pin-unrecorded` and `plan-pin-malformed` end with the `plan --pin <product>@<40 hex commit>` that records the pin whole. - `plan-pin-collides` ends with re-running `plan` with each product once, or, for two products that share a source variable, with one product of each such group. - `plan-root-holds-pin` ends with the kept path's own remedy: send that entry to a leg, `drop` it, or rename it in the source and re-run `plan`. A leg moved over a pin's file in the plan is `plan-leg-path-collides`. - **Also part of #386: its `pins:` row.** A hand-edited `pins:` that is not a list (a name such as `pins: openGlass`, a number, a mapping, a quoted string) is now `plan-pin-malformed` in `check` (`` `pins:` is 'openGlass', not a list of pins ``), and `execute` refuses it before any leg exists. - Where: `_read_pins` in `adopt-project.py`; the test row is `plan-pin-malformed-not-a-list` in `tests/test_adopt_pins.py`. - Before: on main, `check` said `plan ok` for the name, the mapping and the string, and gave a traceback for the number (reproduced by lane 5's review on cb7063f). #386 records that `execute` then wrote the name as eight one-letter pins. - Not fixed here: #386's other keys, `allow_empty_legs:`, `paths:`, `legs:` and `source:`, so #386 stays open. Lane 2's #435 takes all of #386, `pins:` included, and its `pins:` rule keeps only text items where this branch writes mappings; whichever lands second reconciles them (#435 (comment), and "Where the meaning meets" below). - **`execute`.** It writes each pin through `materialize_assembly_root(neutral_pins=...)`. The manifest lists the product with the owner stripped (decision 3), so the entry and the file agree. - **Docs.** `--help` is updated and `docs/cli.md` regenerated (`render-cli-reference.py --check` ok). `pin_digest_from_source` caught only `Refusal`. A directory that is no clone is git's own failure, so it was always `pin-source-unreadable`, exit 2. A path `subprocess.run` cannot enter never reaches git: it is a `FileNotFoundError`, `NotADirectoryError` or `PermissionError` out of `cwd=`, and nothing caught it. `scaffold-project.py --pin ... --pin-source <that>` ended `--dry-run` and the real run alike in a traceback, exit 1. Since the move above, `adopt-project.py plan --pin` reads its digest through the same function and ended the same way. Both reach it through `declared_pin_values`, and `check` and `execute` read no source, so that call is the only reader. The issue's table, run on ab623dd for the scaffold and on d806e8f for `adopt-project.py plan` (adopt has no `--pin-source` on ab623dd), then on this head. `gh` is a stub that logs and exits 97, with `GH_TOKEN` bogus and an empty `GH_CONFIG_DIR`; it was called 0 times in every run: | `--pin-source` | before | this branch | |---|---|---| | a directory that is not a clone | `pin-source-unreadable`, exit 2 | unchanged | | a path that does not exist | `FileNotFoundError` traceback, exit 1 | `pin-source-unreadable`, exit 2: `<path> could not answer for opensoft/openGlass @ <40 hex>: no such directory` | | a file | `NotADirectoryError`, exit 1 | exit 2, `... it is a file, not a directory` | | under a parent this user cannot enter | `PermissionError`, exit 1 | exit 2, `... Permission denied` | | a symlink loop (not in the issue) | `RuntimeError` out of `Path.resolve()` on 3.12, exit 1 | exit 2, `... Too many levels of symbolic links` | The real run and `--dry-run` give the same stdout and stderr, and nothing exists afterwards (no remote directory, no work directory; for adopt, no plan file). What changed, in `scripts/shape_materialize.py` only: - `pin_digest_from_source` catches `OSError` as well as `Refusal` and raises the existing `pin-source-unreadable` with its remediation unchanged. The message names the path and the reason. - `stat()` judges the source inside the guard, as #312 does in `validate-pins.py`, and the sentences are that file's: a missing path, or one beneath a file, is "no such directory"; a file is "it is a file, not a directory"; any other `OSError`, from `stat()` or from git's `cwd=`, is the system's own reason (`strerror`). It is not `exists()`/`is_dir()`, which before 3.14 swallow only a missing path. A source `stat()` can read but `cwd=` cannot enter, a directory with no search bit, fails in `tree_digest` and the same guard answers it. - `declared_pin_values` resolves the path through a guarded `_resolved_pin_source`. `Path.resolve()` raises a `RuntimeError` on a symlink loop before 3.13 (an `OSError` since), before the digest was asked for. As typed, `stat()` meets the loop and says so. - A source that answers is read exactly as before: a relative path resolves against the working directory, a symlink to a clone is followed, `digest_source` reads `local-clone (<path as typed>)`, and the digest is `tree_digest`'s. `tests/test_pin_source_unreadable.py` (46 tests in the #339 commit; fix round 1 adds 17 for S1 and moves one, below) runs each case through `scaffold-project.py --dry-run`, its real run and `adopt-project.py plan`: missing, a file, beneath a file, relative and missing, a parent and a source with no search bit (skipped on Windows and as root, by a mark, so before any fixture builds a directory), a 300-character name, a symlink loop (skipped on Windows) and a directory that is no clone. Each asserts exit 2, the refusal naming the path and the reason, its remediation, no traceback and nothing created. It also holds `--dry-run` and the real run to the same stdout and stderr, checks the path is printed escaped under an ASCII stderr and as a surrogate when it is not UTF-8, and checks that a relative source and a symlink to a clone read as before. The session `gh` guard fails any test whose run called `gh`. origin/main ab623dd reached this branch through implQ's branch: 34dbd86 is a30091f merged with ab623dd. One file conflicted there, `templates/assembly-root/scripts/validate-pins.py`, in seven hunks, because #312 (8cdca5e) replaced `keyed_sources` and `default_source` with `PinSources` and #275 added `env_shared` and `shared`. #312's blob is the same at its PR head, at the squash and at ab623dd, so it landed the file implQ resolved against. The resolution keeps #312's side in all seven and adds #275's on top: | # | hunk | resolution | |---|---|---| | 1 | `resolve_neutral_pin_source` signature | #312's `sources: PinSources` and 3-tuple return, plus #275's `env_shared: bool = False` | | 2 | its docstring tail | #312's `Returns (path, how, typed)` text, plus #275's "Rule 2 is skipped when `env_shared`" paragraph | | 3 | `_recompute_neutral_pin_digest` signature | #312's `sources`, plus #275's `shared: frozenset[str]` after `report` | | 4 | the `resolve_neutral_pin_source(...)` call inside it | #312's 3-tuple unpack, plus `env_shared = product.casefold() in shared` and the extra argument | | 5 | `_check_one_neutral_product_pin` signature | #312's `sources`, plus #275's `shared` | | 6 | its call of `_recompute_neutral_pin_digest` | `sources, report, shared` | | 7 | `_check_neutral_product_pins` docstring and body | #312's `PinSources(...)` and `_note_unused_pin_sources`, plus #275's paragraph, `_report_pin_source_collisions(products, report)` and `shared = frozenset(pin_source_env_shared(products))` | This branch's own merge of 34dbd86 (5690235) had no conflict, and neither had its merge of #409's fix round, 6494714 (583f647). That fix round made `neutral-pin-source-collides` a note and keyed "one product" by `repo_shape.pin_product_key`. At 421e4e5, `templates/assembly-root/scripts/validate-pins.py`, `scripts/repo_shape.py` and `tests/test_pin_source_env_collisions.py` were byte-identical to the tree `git merge-tree` gives for 6494714 with origin/main 3ef059c (blobs 5604007, 091966b, 5220787), so this branch added nothing to those three files beyond #409. The 34dbd86 merge brought in #335, #312, #317, #270 (f795cf7), #344 and #349 with no conflict. Earlier merges of origin/main (87373a6, for #309's reworded constant, and e43fa95) are unchanged. Fix round 1 merged origin/main three more times: - **cd1fc79 (2bcfa69).** It brought #245, #351, #350, #345, #372, #371, #354, #380, #377 and the rest since ab623dd, with no conflict. - **45e01f4 (4098b61).** It brought #396's `tests/test_agents_md_codes.py`. That test finds `scaffold-pin-collides` as a string constant of `scripts/shape_materialize.py`, which `SHIPPED` lists. - **3ef059c (421e4e5), #314.** One conflict, one hunk in `adopt-project.py`. This branch's `_refuse_an_unrecordable_pin_source` and #314's section "The project must be able to hold what `execute` writes into it (#261)" were added at the same place; both are kept. Where the two meet, lane 5's earlier recipe applies: - `_flag_plan` builds each pin as the `PIN_FIELDS` mapping `plan` writes, not `list(pins)` of products. So the plan `plan` asks the validator about is the plan it writes. - #314's `test_plan_asks_what_check_asks_of_the_same_keys` plans `--pin openGlass@<commit> --pin-source <stand-in>` where it planned the name alone, which `plan` now refuses. Then f9ed46f, after #409 squash-landed, and 73d65c6 after it (top of this description). On the merged tree exactly one test failed: `tests/test_adopt_template_values.py::test_check_finds_and_execute_refuses_a_hand_edited_value[pins-surrogate]`. It plans, then hand-edits the plan to `pins: ["openX\ud800"]`, a name-only pin with a lone surrogate. Before this branch the refusal's headline was `REFUSED plan-unwritable-value`. A name-only pin in a plan is now `plan-pin-unrecorded` (#273), so `check` has a second, independent finding for that very plan, and the headline is the first finding's: ``` REFUSED plan-pin-unrecorded: 2 finding(s) ... FINDING plan-pin-unrecorded: `pins:` entry 1, 'openX\ud800', is a name with no commit and no digest ... FINDING plan-unwritable-value: a `pins:` item is 'openX\ud800', which `execute` writes into the assembly root's `project.yaml`, and it holds a lone surrogate ... ``` Nothing about #270's contract changed. The lone surrogate is still found as `plan-unwritable-value`, listed under the refusal, and `execute` still refuses before any leg exists. `check` builds `_leg_findings`, where `_pin_findings` sits, before it calls `_value_findings`, so the pin finding comes first. d806e8f changes one assertion for that row, `pins` only: the headline is `REFUSED plan-pin-unrecorded: ` where it was `PLAN_REFUSED`. The other 15 rows and the finding assertions, including the `plan-unwritable-value` line, are untouched. Lane 5's review of 7a94b6b found 3 should-fix and 5 nits. Each is answered here: - **#409's `pin_product_key` (review N1).** The merge of #409's fix round could not carry it, because this branch had moved the function that used it out of `scaffold-project.py`. `pin_variable_collisions` now keys each pin by that function, where it casefolded. `openß` and `openss` are two pin files and one `SHAPE_PIN_SOURCE_OPENSS`, and a casefold named each of the two pins twice. No `--pin` or recorded `pins:` entry `PIN_ARG_RE` admits can spell either, so the test asks the function directly. `refuse_colliding_pins`' docstring now says, in #409's words, that the root's `validate-pins.py` only notes such a pair. - **S1, a `--pin-source` path that is not printable.** `pin_digest_from_source` names the path as typed unless it is not printable (`str.isprintable`), and then as Python's `repr` spells it (`shown_source`). That covers the refusal's head and git's own reason, which names the directory it ran in. Printed raw, an ESC reached the terminal, and a line break or a carriage return began a line of the path's choosing, a forged `REFUSED` line among them, from the scaffold and from adopt. The traceback #339 replaced had shown a `repr`. A printable path is named byte for byte as before. - **S2, no `gh` on PATH.** `pin_digest_from_gh` refuses as `gh-not-found`, exit 2, as `repo_shape.tree_digest_from_gh` names it, and offers `--pin-source`. It was a `FileNotFoundError` traceback, exit 1, from the scaffold and, new with this branch, from `adopt-project.py plan --pin`. - **S3, a hand-edited `commit:` or `tree_sha256:`.** The lane's decision is to accept it and document it (option (c)); no behaviour changed. A pair that disagrees passes `check` and `execute` (`adoption verified`), because neither reads a network for a pin, by design. The adopted root's own `validate-pins.py` then exits 1 with `neutral-pin-digest-mismatch` wherever it can read the product. `check` now says so once, after the pins it will write: "each pin as `plan` read it, not read again here: the root's own `validate-pins.py` recomputes its digest wherever it can read the product". Options (a), `execute` recomputes before it creates the legs, and (b), `check` recomputes from a clone still on the machine, are filed as #449. - **N2, a `--pin-source` that `plan` cannot record.** `plan` asks `check`'s question (`one_printable_line`) of the path it records in each pin's `digest_source:`, before any digest is read. It refuses one that fails as `adopt-unwritable-value`, in #270's words (`unwritable_reason`), or as "an invisible formatting character" where those name nothing. Before, it was planned, exit 0, and printed raw in the report. `check` then refused it as `plan-pin-malformed` and said to re-run the same `plan`. The report's `pin` line and `check`'s `write` line now spell the digest's source with `_shown`. The scaffold half is pre-existing (a line break in the path is written into the pin file; a byte that is not UTF-8 dies after all three repositories exist) and is filed as #450. - **N3, six hunks no test guarded.** Each now has a test that fails when it alone is reverted: - the two name guards (a `pins:` name holding an ESC and a forged line); - the lower-casing of the commit and the digest (an upper-cased hand edit); - the `digest_source` printability check; - the not-text check (`commit: 1234567` unquoted); - the gitlink filter (a submodule kept where the pin is written is `plan-submodule-root` alone). - **N4.** This body. - **N5, #339's third decision.** A `--pin-source` that nothing reads is still silent, now in adopt too. This PR does not decide it. #451, filed, lays out refuse, note and leave, so closing #339 loses nothing. Lane 5's second review, of 421e4e5, found no blocking item, 2 should-fix items and 3 nits. The two should-fix items are this body: the landing after #409 (above) and #386 (above). The code needed no change for them. f63d1ed adds two tests and no code: - **n1. `plan` asks nothing of a `--pin-source` no `--pin` records.** With the `args.pin and` guard dropped, such a path holding an ESC or a zero-width joiner was refused in a sentence naming no file ("`execute` writes into , and ..."), and every test still passed. `test_a_pin_source_no_pin_records_is_not_asked_about` holds the plan to exit 0 and `pins: []`, with the path named nowhere in the plan or the report. The ESC row is POSIX-only by the file's `POSIX_NAMES` mark; the zero-width joiner row runs everywhere. It holds #451's answer today, silence, so a decision there changes this test on purpose. - **n2. A printable `--pin-source` is printed as it is.** With `_spelled` in place of `_shown` in `plan`'s `pin` line, every test still passed. `test_a_printable_pin_source_is_printed_as_it_is` holds `plan`'s `pin` line, and `check`'s `write` line for the plan `plan` wrote, to their exact text, ending `(local-clone (<path>))` with no backtick. - **n3** needs a code change and is left out (below). **Fix round 2, on f63d1ed.** Each run had a `gh` stub first on PATH (it logs and exits 97), `GH_TOKEN` bogus, an empty `GH_CONFIG_DIR`, `git-filter-repo` 2.47.0 next on PATH, and the identity blanked with `GIT_CONFIG_COUNT`; the session `gh` guard came first inside each run. - **The pin set: 812 passed, 2 skipped.** It covers adopt pins, adopt plan, #339's file, #275's tests, the pin and neutral-pin validators, git output decoding, adopt template values, CLI reference, scaffold pin and reuse, scaffold pin collisions, #314's two files and #396's file. The skips are `test_adopt_plan.py:403` (no MedxEHR clone) and `test_adopt_template_values.py:223` (this host reads `git config user.name` from the repository, not from `GIT_CONFIG_GLOBAL`). That is fix round 1's 809 passed, plus the three new rows. - **Shape doctor and scaffold leg-path collisions: 369 passed, 3 skipped** (the PowerShell line exists only on a Windows host). - **Mutation probes**, each one line changed in a throwaway worktree, over `tests/test_adopt_pins.py` and `tests/test_pin_source_unreadable.py` less the five end-to-end adoptions: - the `args.pin and` guard dropped: 2 failed, both n1 rows; - `plan`'s `pin` line `_spelled`: 1 failed, the n2 test; - `check`'s `write` line `_spelled`: 2 failed, the n2 test and `test_an_upper_cased_hand_edit_is_written_lower_cased`; - `plan`'s `pin` line raw: 1 failed, the no-break-space test that already guarded that hunk. **On the merged tree, 73d65c6** (the merge f9ed46f and the `_read_pins` commit), with the same harness: - **The pin set** above, with #445's `test_adopt_late_failure.py`, `test_adopt_git_reads.py` and `test_adopt_filter_repo_failure.py` and #387's `test_adopt_empty_symlinks.py` and `test_adopt_root_manifest.py`: **988 passed, 2 skipped**. The skips are the two above. - **Shape doctor, scaffold leg-path collisions and the adopt suites:** **967 passed, 3 skipped** (the PowerShell line). The adopt suites are e2e, execute refusals, verify, root shape paths (#387's too), leg-path collisions, clone template, ignored shape files and submodules. - **On the merge alone, f9ed46f** (the same tree as before the `_read_pins` commit): `tests/test_adopt_late_failure.py` gave 3 failed, 83 passed, the three `AttributeError` rows above. - `render-cli-reference.py --check` is ok on the merged tree. pyflakes reports the same four warnings on `adopt-project.py`, `scaffold-project.py` and `scripts/shape_materialize.py` as before the merge. **Fix round 1, on 421e4e5.** Each run had `git-filter-repo` 2.47.0 on PATH and the identity blanked with `GIT_CONFIG_COUNT`. A `gh` stub that logs and exits 97 was first on PATH, with `GH_TOKEN` bogus and an empty `GH_CONFIG_DIR`, and the session `gh` guard came first inside each run. - **The pin set:** 809 passed, 2 skipped (the same files and skips as above). - **Doctor and scaffold:** 782 passed, 3 skipped (the PowerShell line exists only on a Windows host). It covers shape doctor, scaffold leg-path collisions, scaffold e2e, referent chain links, naming policy, repo hygiene and #351's free-text values. - **The adopt suites:** 882 passed. It covers e2e, execute refusals, verify, spec-only, leg paths, root shape paths, plan escapes, leg-path collisions, clone template and submodules. - **Before the #314 merge,** on 74ce4d9's tree: the first two sets gave 577 passed, 2 skipped and 782 passed, 3 skipped. On 2bcfa69's tree (the merges alone, before the fix) they gave 542 passed, 2 skipped and 782 passed, 3 skipped. After the 45e01f4 merge, `tests/test_agents_md_codes.py`, repo hygiene and CLI reference: 110 passed. - **Lane 5's second review** also ran 421e4e5 merged with main cb7063f and with 53bee5b: the pin set, the manifest-value set, and the doctor set with #357's `tests/test_doctor_validator_rows.py`. All passed, with only the skips above. Proof that fix round 1's tests test something: - **On 2bcfa69's tree,** with this head's two test files laid over it, 31 of the new rows fail: - S1: 17; - N2 and its report spelling: 7; - S2: 3; - the `pin_product_key` port: 2; - `check`'s S3 line: 1; - the not-UTF-8 `adopt plan` row: 1. The N3 rows pass there, as they must: they guard hunks this branch already had. - **Mutation probes.** Each hunk was reverted alone, in a throwaway worktree, and both test files were run: 18 of 18 were caught. - **The port:** the casefold keying (2 fail); the casefold lookup alone (2). - **S1:** the path named raw (13); git's reason named raw (2); every path `repr`'d, printable or not (27). - **S2:** no `except` (3). - **N2:** no `plan` check (11); only #270's question asked (1); the formatting-character reason said otherwise (1); the report's `pin` line raw (1); `check`'s `write` line raw (1). - **S3:** the line dropped (1). - **N3:** the `_pin_fix` name guard (1); the `_read_pins` path guard (1); the lower-casing (1); the `digest_source` check (1); the not-text check (1); the gitlink filter (1). - **The issue rows reproduced.** S1, S2 and N2 were reproduced on 2bcfa69's tree and shown gone on the fix (a probe driving both tools; `gh` was called 0 times). **Before fix round 1.** Runs were on 7a94b6b, with `git-filter-repo` 2.47.0 first on PATH and the repository-local `user.name` blanked with `GIT_CONFIG_COUNT` (a workstation whose standard checkout sets one fails `test_the_scaffold_refuses_with_no_elector` on main itself; CI sets none). 7a94b6b differs from ffd1297 in `tests/test_pin_source_unreadable.py` only, so the three suites that do not read that file ran on ffd1297: - the new file: 46 passed; - adopt pins, adopt plan, git output decoding: 172 passed, 1 skipped (the MedxEHR clone is absent); - scaffold pin and reuse, pin collisions, e2e, referent chain links, neutral pin and pin validators, #275's tests, naming policy, repo hygiene, CLI reference, scaffold leg-path collisions: 571 passed; - shape doctor: 254 passed, 3 skipped (the PowerShell line exists only on a Windows host); - the new file with repo hygiene, git output decoding and CLI reference, once more after the last edit: 176 passed. Earlier on 5690235 and d806e8f, before the #339 commit: the adopt suites (e2e, execute refusals, verify, spec-only, leg paths, root shape paths, plan escapes, leg-path collisions, clone template) 740 passed, and `tests/test_adopt_template_values.py` with #273's other adopt tests 489 passed, 2 skipped. They were not re-run on 7a94b6b: the #339 diff touches `pin_digest_from_source` and the call in `declared_pin_values`, which those suites reach through `test_adopt_pins.py` only. Nothing was run on macOS or Windows. Proof the new tests test something: - **On d806e8f.** 38 of the 46 fail, each on the traceback and its exit 1. The 8 that pass are the unchanged behaviour: the not-a-clone refusal (4 rows) and the valid-source rows (4). - **Mutation probes.** Ten hunks reverted one at a time, each failing at least one new test: the `OSError` catch (16 fail), the catch narrowed to `OSError` only (21), the missing-path clause (9), the clause narrowed to `FileNotFoundError` (3), the `S_ISDIR` judgement (3), the loop guard (4), no resolve at all (3), the old `.resolve()` call (4), the reason as `str(exc)` (12), and the file sentence as the OS's (3). - #273's own 52 tests still fail on c7c02fa (51 fail or error; the one that passes checks the test's own table), and its 18 mutation probes were each caught. **Re-verified after the merge.** An independent re-verification of f9ed46f and 73d65c6 found the code correct: - #409 is carried whole. A 30,000-case fuzz of `--pin` lists through #409's final refusal and this branch's port differed in 0 cases. - `scaffold-project.py` equals the tested file plus main's own changes that are not #409's (same patch-id). - `_commit_the_split` keeps #445's `try` and #394's `append`. - `_read_pins` reads a loaded plan as before. Its findings were all in this description (the #506 bullet and three stale facts), now fixed. Runs on the head and on its merge with main 48585e6: 988 passed, 2 skipped; and 967 passed, 3 skipped. `git merge-tree --write-tree` was run at 01:48Z on 2026-10-13, comparing hunks per file. Each PR head, fetched from `refs/pull/<n>/head` with no `gh`, was merged with 73d65c6 and with origin/main 9736424 alone. Two sets were asked about: - every PR this description listed before that main does not now contain; - every PR numbered above #468 whose head main does not contain: #473, #474, #490, #491, #493, #494, #495, #496, #497, #503, #504 and #505. **Open state, read with `gh` by the coordinator at 02:00Z and refreshed at 02:25Z after a re-verification on main 48585e6.** Every pull request named below is open. #184, #94, #96, #98, #100 and #161 are closed, and #465 landed as 48585e6. `fix/adopt-plan-quote-yaml-keywords` has no pull request. #506 (`fix/scaffold-reuse-seeded-legs`) opened after this list was made, and the re-verification reads it. - **Landed since the last draft of this list:** #343, #346, #355, #358, #373, #378, #382, #387, #394, #397, #401, #404, #409, #419, #452, #456, #465, #466 and #484. #394's `_commit_the_split` conflict and #401's `docs/cli.md` conflict are both resolved in f9ed46f, above. - **Clean, 20:** #266, #347, #356, #374, #376, #384, #395, #453, #467, #468, #473, #474, #490, #491, #493, #494, #495, #497, #503 and #505. #453, #468 and #491 were updated onto main since the first read and now merge clean. #507 (lane 5's #486-#488) shares only the generated `docs/cli.md` with this branch and merges clean with it. - **Conflicting with main alone, and this branch adds no hunk:** - #402: `shape-doctor.py` 5; - #496: `shape-doctor.py` 10 and `tests/test_shape_doctor.py` 1. **Conflicts this branch adds:** - **#435** (3efcc7a, lane 2's `fix/adopt-plan-field-types`, for #386). - It adds 2 hunks in `adopt-project.py`: 5 here, against 3 with main alone. They are `render_plan`'s `pins:` line and `Plan.pins`. - Keep this branch's `_pin_lines` and `_pin_products`. The rest is under "Where the meaning meets". - **#504** (37ef29a, `fix/adopt-root-kept-symlinks`). - It adds 1 hunk in `adopt-project.py`: 2 here, against 1 with main alone. It is the materializer call in `_commit_the_split`. - Keep both arguments and both comments: `append=append, neutral_pins=_recorded_pins(plan), spell=_spelled`. - **#506** (119d8c4, `fix/scaffold-reuse-seeded-legs`). - It adds 1 hunk in `scaffold-project.py`. With main alone, #506 conflicts only in `docs/handbook.html`. - #506 inserts `LEG_TEMPLATES` right after `declared_pin_products`, which this branch moved out, and rewrites `_reusable_remotes` further down. Keeping both sides breaks the file with an unterminated string literal. - Keep only #506's `LEG_TEMPLATES` block. Drop #506's `declared_pin_values`/`declared_pin_products`, which moved here, and this branch's three-line `_reusable_remotes` opener, which #506 rewrites below. Resolved that way, pyflakes is clean and #506's scaffold tests pass (88). - **`fix/adopt-plan-quote-yaml-keywords`** (ed9968f; a branch with no pull request). - It adds 2 hunks in `adopt-project.py`: 3 here, against 1 with main alone. - One is `render_plan`'s `pins:` line. Keep `_pin_lines`, which already sends product, repository and `digest_source` through `y`. - The other is `_refuse_unholdable_values`' signature. Keep `pins: dict`, with that branch's `_validator_judged` and `_named_as_read` above it. - `fix/adopt-late-failure-notice` is no longer on origin. #445, the late-failure notice, is on main and merged here. **Where the meaning meets:** - **#435 and #386.** The coordinator has said this on #435: #435 (comment) - #435's `_plan_key_findings` (`plan-bad-pins`) keeps only text items in `pins:`, and its `PINS_REMEDIATION` tells a person to write `pins: [openGlass]`. - This branch writes each pin as a mapping, and it makes a name alone `plan-pin-unrecorded`. - Read from #435's code, not run: on a tree with both, the plan `plan --pin` writes is `plan-bad-pins` in `check`. - Whichever lands second reconciles them, with three changes: - `_plan_key_findings` takes this branch's mappings, or leaves `pins:` items to `_read_pins`; - a `pins:` that is not a list gives one finding, not both `plan-bad-pins` and `plan-pin-malformed`; - `PINS_REMEDIATION` says the mapping `plan` writes. - Until that is done, `tests/test_adopt_pins.py::test_a_plan_with_a_whole_pin_checks_ok` fails. - **#314.** It landed, and it is merged here (above). - **`fix/adopt-plan-quote-yaml-keywords`.** Its pin tests are written against the name-only `--pin`. They need the `name@commit` form, and their expected `pins: [...]` line becomes `_pin_lines`'s mapping. - **Two unwritable values are refused in two runs (lane 5's second review, n3).** `plan --elected-by 'A<LF>B' --pin openGlass@<sha> --pin-source <a path holding an ESC>` refuses the elector alone, exit 2. Re-run with the elector retyped, it refuses the source; #270's refusal names every unwritable value at once. Reproduced on 421e4e5, with `gh` called 0 times. One refusal would need a code change, not made here: - `_refuse_an_unrecordable_pin_source` would split into a function that returns the sentence; - `_checked_plan_inputs` would ask it before #270's `unwritable_values` raise, and raise one `adopt-unwritable-value` that names both, with both remediations; - a test would add the two-value row. N2's question is `one_printable_line`, not `unwritable_reason`, so it cannot simply join #270's list. - **Docs.** The README, AGENTS.md and `docs/handbook.html` are unchanged, because none of them calls adopt's `--pin` a name. A sentence saying adopt now takes the scaffold's `--pin` is a follow-up. It costs an AGENTS.md line-cap raise, and a README edit forces the handbook's README blob to change. - **A product spelled `null`, `true` or `false`.** The scaffold already accepts `--pin null@<sha>`: it exits 0 and builds a root whose `validate-pins.py` exits 1 `neutral-pin-entry-malformed`. Adopt's `plan` writes `product: null` through `y`, and `check` then finds `plan-pin-malformed` and refuses it before any leg exists. The one fix for both tools is in `parse_pin`, and it would change the scaffold's behaviour. - **Wording shared with the scaffold.** `digest_source` is written bare into the pin file, so a `--pin-source` path holding ` #` is cut short there. `templates/assembly-root/contracts/neutral-product-pin.yaml` and `validate-pins.py`'s `neutral-pin-missing` remediation name only the scaffold as the writer. Two comments in `scripts/repo_shape.py` still name the scaffold's `pin_digest_from_gh`, in a digest-pinned shipped copy. - **Filed as follow-ups:** - #449: S3, a hand-edited `commit:` or `tree_sha256:` that disagrees, with options (a) and (b); - #450: the scaffold half of N2, a `--pin-source` path no file can hold; - #451: #339's third decision (N5), a `--pin-source` that nothing reads. - **`validate-pins.py` prints a `--pin-source` path raw** in `neutral-pin-source-unreadable`, the twin of S1. It is a shipped template that #409 and other open PRs edit, so it is left for its own change. No issue is filed for it yet. - **Windows and macOS** were not run here; CI ran both (below). Two assertions that depend on what Windows answers for a path (a name past the filesystem's limit, a path beneath a file) check only the refusal and its remediation there. Fix round 1's rows that need a control character or a byte that is not UTF-8 in a path, or a symlink to `git`, are skipped on Windows by a mark, and so is fix round 2's ESC row. The rows for a no-break space and a zero-width joiner run everywhere, and so does the in-process `gh-not-found` test. Reviewed independently by lane 5 at 7a94b6b (3 should-fix, 5 nits: fix round 1) and at 421e4e5 (0 blockers, 2 should-fix, 3 nits: fix round 2). After #409's squash, the re-verification of f9ed46f and 73d65c6 found the code correct; its 1 should-fix and 3 nits were in this description and are fixed. GitHub's update-branch then merged main twice (33bae3f, 2170526), with no hand edit. CI's tests, tests-macos and tests-windows and SonarCloud all passed at 2170526 (main cac6bb2 merged in). On that head merged with main 57c85e9 (#490, #474 and #468, none of which touches this branch's files), the pin and adopt set with `tests/test_update_shape_landing.py` gave 1112 passed, 2 skipped, and the doctor, adopt end-to-end, unfinished-merge and bootstrap set gave 1056 passed, 3 skipped (the doctor's PowerShell line, which exists only on a Windows host). Closes #273 Closes #339 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s a runnable `scripts/family.py`, printed commands quote paths, and `add`, `bump` and `remove` refuse the tracking branch and print their branch's push (#488, #486, #487) (#507) Three defects from lane 3's v1.0.0 acceptance pass of AGENTS.md's "Creating a family, and adding a member", one commit each, in this order, then one review commit for #486 and one for #487. All three change `scripts/family.py` and `add`'s NEXT line, so they are one pull request. **What was wrong.** - AGENTS.md spelled the command `python3 scripts/family.py init --org <org> --family <Name>`, a path that runs from the standard's root. `--into` defaults to where the person stands. - So the command, followed exactly, printed `landing <standard>/Orbit/Orbit` with no word, and `make siblings` would then clone every member into the standard's checkout too. - `init`'s NEXT STEPS printed `python3 family.py add …`, built from `Path(__file__).name`. From the root that fails `can't open file`, exit 2. **What changed** (commit 1). - With neither `--into` nor `--work-dir`, `init` refuses a landing inside a checkout of openRepoShape: `family-landing-in-standard`. A checkout is a directory holding `scripts/family.py` beside `templates/family-root/`, found in the landing's ancestry, so another clone or an install counts too. - The refusal comes before anything is created, under `--dry-run` alike. - The remedy names `--into ..` from the root. From deeper inside the checkout it names the relative path that reaches the checkout's parent. - An explicit `--into` or `--work-dir` is not second-guessed. `family-holder-already-here` is still asked first. - NEXT STEPS, and the `family-holder-already-here` remedy, name this file relative to the directory `init` ran in. From the root that is `scripts/family.py`, as setup-project.py, validate-family.py, AGENTS.md and README spell it. It is absolute where no relative path exists. - AGENTS.md's init command passes `--into ..`, as README's already does, and step 1 says why. The edit is line-neutral: step 1 is one line shorter. - `init --into --help` says it too, and docs/cli.md is regenerated. **How tested.** `tests/test_family_init_landing.py`, 9 tests: - the old command from the root is refused, as a dry run, so nothing can land in the checkout; - a real run inside a fake checkout creates nothing; - the remedy from a subdirectory names the right parent; - an explicit `--into .` or `--work-dir .` passes; - outside a checkout the default is unchanged; - AGENTS.md's line, as written, lands beside the checkout; - the printed NEXT runs, both from the root and from elsewhere. On origin/main code 6 of them fail. Removing the refusal turns 3 red, and putting `Path(__file__).name` back turns 2 red. **What was wrong.** `family.py` and the holder's `siblings.py` printed every path in their pasteable commands bare. Under a holder at `…/my work/Sonata/Sonata`: - the `member-git-dir-cached` remedy `rm -rf <store>` split into `rm -rf …/my`, an absolute directory that is NOT the store, and a relative `work/…`; - `init`'s NEXT STEPS and `add`'s NEXT split `--family-root` and `git -C` (`fatal: cannot change to '…/my'`, exit 128); - the `mv` lines for a misnamed family folder pasted as four arguments. **What changed** (commit 2, and the review commit after #487's). - `scripts/repo_shape.py` gains `shell_quoted(value, platform=None)` and `shell_line(words, platform=None)`, in a new section at its end, the library `family.py` already imports. - It is the doctor's `quote_arg` rule (#101/#102): the same two alphabets, and the same two escapes, `sh`'s `'\''` and PowerShell's doubled apostrophe. The dialect comes from `os.name` or a `platform` argument. An ordinary word is left bare. - It does not flatten to ASCII, as the doctor's report does by contract. A command that deletes or moves a directory has to name that directory. So PowerShell's typographic single quotes are doubled too. - A word that begins with `=` is quoted, in both dialects. zsh, macOS's login shell, expands a bare `=word` to a command's path (`=ls` reaches the command as `/bin/ls`). This is the one place it differs from the doctor's rule on ASCII. - `family.py`: - `init`'s NEXT STEPS print the `add` on ONE line. It used to paste only into a POSIX shell, because of its `\` continuation. `cd <holder>` and `make siblings` are two lines, with no `&&`, which Windows PowerShell 5.1 rejects between two statements. - Those lines and the `family-holder-already-here` remedy quote every path. - `add`'s NEXT quotes `git -C <holder>`, and `member-path-taken` quotes each spelling it names. - `member-git-dir-cached`, and `remove`'s note about the same store, print the directory alone on its own line. Then comes `rm -rf` with its one argument quoted for the POSIX shell `rm -rf` belongs to, and the sentence names that shell: Git Bash on Windows. PowerShell refuses `-rf` and deletes nothing. - `siblings.py`: the parent-folder warning's `mv` and `mkdir -p` lines are quoted for a POSIX shell with the standard library's `shlex.join`, and the warning says which shell. It does NOT import the new quoter: a holder that kept its own `repo_shape.py` through `update-shape.py apply --accept-local` takes the new `siblings.py` without it, and an import of a name its copy lacks stopped `make siblings`, `park` and `resume` with an ImportError that `check` and the validator never reported. - `setup-project.py`'s family hand-over quotes the holder in its `add` line (and names the branch first, see #487). It runs before any checkout of the standard exists, so it carries its own copy of the rule, and a test holds that copy to `shell_quoted`. **How tested.** `tests/test_family_printed_quoting.py`, 69 tests: - The quoter round-trips through `shlex.split`, and through a PowerShell reader, for a space and an apostrophe, `é` and a typographic apostrophe, shell syntax, the empty string, and every printable ASCII character alone and before, inside and after a word. - The host's own `sh`, `bash` and `zsh` hand each quoted word back byte for byte through `printf`, a leading `=` among them (skipped where a shell is absent, and on Windows). - It agrees with the doctor's `quote_arg` on every one of those ASCII values but a leading `=`, and its two alphabets are the doctor's two patterns. - Under a holder in `my work` and in `Dana's work`, each of these splits into the argv it means: `init`'s NEXT (with `cd` and `make siblings` on their own lines), `add`'s NEXT, the cached-store remedy and note, the `mv` lines, and setup-project's `switch` and `add`. - The `rm -rf`, run as split, deletes the store, and the decoy `<tmp>/my` survives. - The `mv` lines, run as split, move the holder. - The holder's own `siblings.py` runs `--help`, the misnamed-folder warning and `--make park` beside a `repo_shape.py` that lacks every name this change added. - Three existing tests' expected lines follow the new wording. Mutations, each turning tests red: - a bare quoter: 35 red; - bare `mv` lines in `siblings.py`: 3 red; - a bare `rm -rf`, a bare `add` NEXT, a bare `init` `add` (round 1, on the 85-test file): 2 red each; - `siblings.py` importing `shell_line` again: 1 red (the older-`repo_shape.py` test); - `%` dropped from the POSIX alphabet: 2 red (the every-ASCII agreement and the alphabet test). The round-1 sample test passed it, 36 of 36; - a leading `=` left bare: 10 red, among them the real-`zsh` test; - `cd … && make siblings` back on one line: 2 red; - setup-project's holder unquoted: 2 red. **What was wrong.** - AGENTS.md says each act's ONE commit lands as a pull request, and no step made the branch that pull request needs. - After `init` the holder stands on `main`, and all three acts committed straight onto it. - `add` printed `NEXT git -C <holder> push -u origin <branch> && open a pull request`. No branch is called `<branch>`, so the push failed: `src refspec … does not match any`. - `bump` and `remove` printed no NEXT at all. - `bump-leg.py` refuses exactly this for a leg (`bump-leg-on-tracking-branch`). **What changed** (commit 3, and the review commit after it). - The real run of each act refuses the holder's tracking branch, the one `family.yaml` declares (`family-on-tracking-branch`), and a HEAD on no branch (`family-holder-detached`, which is what `bump-leg.py`'s `bump-leg-root-detached` refuses too). - The check comes after the refusals about the arguments and the holder, and before the member is read, fetched or cloned, and before anything is written. - The remedy is a quoted `git -C <holder> switch -c <branch>`. The branch is `family/add-<project>`, `family/bump-<id>-<sha12>` or `family/remove-<id>`, and a name that is not a plain branch word becomes `member`. - `--dry-run` runs on any branch. On a work branch it prints where the commit would land. On the tracking branch it prints that the real run refuses there, with the refusal's code, and the `switch` command on a line of its own. The module's EXIT CODES paragraph names this as the one refusal a dry run does not make. - Each act that commits ends with ONE pasteable line: `NEXT git -C <holder> push -u origin <the branch it committed on>`, quoted, then `then open a pull request against <tracking>; never push to it directly`. `bump` and `remove` gain that NEXT. - `init`'s NEXT STEPS open with the `switch` before the `add`. - `add`, `bump` and `remove` `--help` say the branch rule, and that each ends by PRINTING the push; docs/cli.md is regenerated. - AGENTS.md's family block gains the `switch` step before the `add`, and step 3 says the tracking branch is refused, as `family-on-tracking-branch`, and that the branch is pushed as its NEXT line says. README's InkRouter block gains the same step, and its paragraph says it. Both edits are line-neutral: AGENTS.md tightens steps 2 and 5, and README tightens the `make bootstrap` paragraph. `docs/handbook.html` mirrors the block and the sentence, and its footer names #487 and the new README blob. - The holder's own docs, `templates/family-root/AGENTS-shape.md` and `templates/family-root/README.md`, open their add/bump/remove block with the `switch` and say the act refuses the tracking branch. `setup-project.py`'s family hand-over prints the `switch` before its `add`. - Test fixtures that committed on `main` now commit on a branch: - `tests/test_family.py`'s module fixture grows the family on `family/members` and pushes that branch to the remote's `main`, as the merged pull request would leave it; - `test_shape_doctor.py`'s holder fixture and `test_family_add_excludes.py`'s `empty_holder` switch to a branch; - `test_family_add_refusals.py` changes in four places: the `NEXT` expectation, `unborn_holder`, the empty-family test, and the corrupt-HEAD test. In that last test, the tag state's branch check is answered in-process, because the test is about the rollback's reading of HEAD. **How tested.** `tests/test_family_tracking_branch.py`, 17 tests: - For each act, on `main`: refused, with the holder byte for byte as it was, and for `add` nothing cloned. Then the printed `switch`, run as split, makes the branch; the act goes through on it; and its printed NEXT, run as split, pushes the branch, which the remote then holds at the act's commit. - A detached HEAD is refused. - Dry runs on `main` and on a branch say what they should. - A family made with `--tracking-branch trunk` refuses `add` on `trunk`; on a branch named `main` the dry run says the commit lands there for a pull request against `trunk`, the real run commits, and its NEXT pushes `main`. - The suggested names pass `git check-ref-format --branch`. - `init`'s NEXT STEPS open with the `switch`. - AGENTS.md, README.md and the holder's two docs show the `switch` in the same block as, and before, the first `add`. Mutations, each turning tests red: the branch check letting `main` through, 4 red; the NEXT line naming `<branch>` again, 3 red; `init`'s NEXT STEPS without the `switch`, 1 red; the tracking branch hard-coded to `"main"`, 1 red (the `trunk` test; every family test passed it before); the holder docs without the `switch`, 2 red; setup-project without the `switch`, 2 red. An independent review at 10c93de gave CHANGES NEEDED (0 blockers, 1 should-fix, 6 nits). The should-fix was a holder that kept an older `scripts/repo_shape.py` through `apply --accept-local`: its new `siblings.py` could not import the shared quoter, so `make siblings`, `park` and `resume` died. The round at bb11b63 fixes it and all six nits, as described above. Re-verified at bb11b63: APPROVE WITH NITS (0 blockers, 0 should-fixes, 3 nits), with SonarCloud's quality gate OK and 0.0% new duplication. The nits were the holder README's pointer to the push (now "as its `NEXT` line says"), two Sonar smells (S1192: `refs/heads/` is now `HEADS_PREFIX`; S3415: the two expected-first assertions now put the actual value first, as the file's other 32 do), and this description's list of other pull requests. All three are fixed at 3451a56: commit 11ae0a8, then a clean merge of main 994be39. The family set gave 566 passed, 8 skipped there. GitHub's update-branch then merged main cac6bb2 as 8e1471c, and main aa4e1f5 (#508, after #490, #474 and #468) was merged as 708f3bb. There only the `docs/handbook.html` footer conflicted: it takes main's clauses, #468's (#424) included, with the README blob recomputed, and `docs/cli.md` auto-merged and passes `--check`. Main 47b1e06 (#505 and #395, after #376, #374, #384, #495, #493 and #491) was then merged as fcbc7a2, again with only the footer in conflict: main's clauses stay in main's order, this branch's (#487) moves to the end, after (#483), and the README blob is recomputed. Every run used `/usr/bin/python3 -m pytest -q -p no:cacheprovider`, with a stub `gh` first on PATH, a bogus `GH_TOKEN`, an empty `GH_CONFIG_DIR` and git's identity blanked. Run on Linux. At e42ef74 (main d41cc29, #402, merged in by GitHub's update-branch), on its tree 655c927: the first set listed below for fcbc7a2, plus every `tests/test_update_shape*.py`, gave **1242 passed, 22 skipped**; the second set, plus `tests/test_leg_path_yaml_scalars.py`, `tests/test_ruleset_hint_tracking_branch.py`, `tests/test_scaffold_reuse_seeded_legs.py`, `tests/test_scaffold_pin_and_reuse.py` and `tests/test_doctor_validator_rows.py`, gave **1260 passed, 12 skipped**. The skips are the same as at 5a2776d, below. At 654df49 (main 8cd7dbe merged in, with #506 and #347): the first set listed below for fcbc7a2, plus every `tests/test_update_shape*.py`, gave **1242 passed, 22 skipped**; the second set, plus `tests/test_leg_path_yaml_scalars.py`, `tests/test_ruleset_hint_tracking_branch.py`, `tests/test_scaffold_reuse_seeded_legs.py` and `tests/test_scaffold_pin_and_reuse.py`, gave **1231 passed, 12 skipped**. The skips are the same as at 5a2776d, below. At 5a2776d (main f391e58 merged in, with #266, #509, #496 and #435): the first set listed below for fcbc7a2, plus every `tests/test_update_shape*.py`, gave **1213 passed, 22 skipped** (the template-quoting skips below, and three NTFS-junction cases, Windows only); the second set, plus `tests/test_leg_path_yaml_scalars.py`, gave **1171 passed, 12 skipped** (the four Windows-only skips below, and eight in that file, main's new one, whose cases the quoted templates (#259) leave with nothing to refuse). At 22733ce (main 631f087, #503, merged in by GitHub's update-branch): the first set listed below for fcbc7a2, plus every `tests/test_update_shape*.py`, gave **1213 passed, 22 skipped** (the template-quoting skips below, and three NTFS-junction cases, Windows only); the second set gave **944 passed, 4 skipped**. At fcbc7a2 (origin/main 47b1e06 merged in, with #505 and #395): - every `tests/test_family*.py`, `tests/test_validate_family*.py`, `tests/test_park_resume_targets.py`, `tests/test_free_plan_hint.py` and `tests/test_template_text_placeholders.py`: **844 passed, 19 skipped** (`tests/test_family_init_values.py:219` and `:291`, which skip because the template now quotes the placeholder and `id:`). - `tests/test_repo_hygiene.py tests/test_agents_md_codes.py tests/test_cli_reference.py tests/test_shape_doctor.py tests/test_setup_project_py.py tests/test_setup_sh.py tests/test_template_quoted_values.py tests/test_stdout_encoding.py tests/test_windows_paths.py tests/test_commit_trailers.py`: **944 passed, 4 skipped** (the doctor's PowerShell line and `tests/test_stdout_encoding.py:286`, Windows only). At 708f3bb (origin/main aa4e1f5 merged in): - every `tests/test_family*.py`, `tests/test_validate_family*.py`, `tests/test_park_resume_targets.py` and `tests/test_free_plan_hint.py`: **718 passed, 8 skipped** (`tests/test_family_init_values.py:242`). - `tests/test_repo_hygiene.py tests/test_agents_md_codes.py tests/test_cli_reference.py tests/test_shape_doctor.py tests/test_update_shape_landing.py tests/test_doctor_unfinished_merge.py tests/test_template_values_single_pass.py tests/test_stdout_encoding.py tests/test_windows_paths.py`: **813 passed, 4 skipped** (the doctor's PowerShell line and `tests/test_stdout_encoding.py:286`, Windows only). Earlier, with origin/main 48585e6 merged in: - `tests/test_family*.py tests/test_validate_family*.py tests/test_stdout_encoding.py tests/test_windows_paths.py tests/test_repo_hygiene.py tests/test_agents_md_codes.py tests/test_cli_reference.py tests/test_park_resume_targets.py tests/test_bootstrap_spelling.py tests/test_free_plan_hint.py tests/test_template_values_single_pass.py`: **1303 passed, 9 skipped**. - `tests/test_shape_doctor.py tests/test_update_shape*.py tests/test_commit_trailers.py tests/test_tracking_branch_ref.py tests/test_scaffold_ignored_shape_files.py tests/test_ruleset_hint_tracking_branch.py tests/test_bump_leg_dry_run_tracking.py tests/test_setup_project_py.py`: **999 passed, 6 skipped**. - The new files alone: `test_family_init_landing.py` 9, `test_family_printed_quoting.py` 69, `test_family_tracking_branch.py` 17, all passed. The full suite was not run locally. CI's three jobs are the check. #505 and #395 have since landed, and both are merged here at fcbc7a2. Each of #505's family sentences was read against this branch's code, and none is false: AGENTS.md 3 and 233; README 569, 822, 1152, 1269 and 1348; the handbook 1126 and 1506. #395 landed with its test switching to `family/members` before its real `add`, as the last bullet below says, and the first run above includes it. #503 then landed as 631f087, and GitHub's update-branch merged it here as 22733ce with no conflict. Main f391e58 (#266, #509, #496 and #435) was then merged by hand as 5a2776d, again with only the footer in conflict: main's clauses stay in main's order, this branch's (#487) moves after #509's (#498), and the README blob is recomputed; `git show --remerge-diff` shows nothing else. #509's reworded sentences in AGENTS.md, README.md and the handbook were read beside this branch's, and none contradicts or repeats another. Its `setup-project.py` changes (the clone-path check, the plan line, the step-(7) backstop and `_main`) and this branch's `shell_word` lines in the hand-over do not meet, and `docs/cli.md` passes `--check`. Main 8cd7dbe (#506 and #347) was then merged by hand as 654df49, again with only the footer in conflict: this branch's (#487) moves after #506's (#462), the README blob is recomputed, and `docs/cli.md`, which both sides changed, still passes `--check`. #506's two `scripts/family.py` hunks (the `FAMILY_RULESET_HINT` import, and its print when the holder's push is refused) sit unchanged inside this branch's code, and that hint prints no path. Under `--reuse-empty-repo`, `init` still refuses its default `--into` inside a checkout of this standard (`family-landing-in-standard`) before it reads the remote, as a dry run with a stub `gh` showed. #506's sentences in AGENTS.md, README.md and the handbook are still true beside this branch's. Main d41cc29 (#402, for #277), which rewrote the doctor's NEXT lines and `tests/test_shape_doctor.py`, was then merged by GitHub's update-branch as e42ef74, with no conflict; the `holder` fixture still switches to `family/members` before its `add`. AGENTS.md stays at 490 lines and README.md at 1422. `git merge-tree --write-tree` of 708f3bb, at 04:47Z on 2026-10-13: - **Clean:** #503 (`fix/family-makefile-without-siblings`) and #395. Each merged tree keeps AGENTS.md at 490 lines and README.md at 1422. - **`docs/handbook.html` footer only:** #505 (`docs/v1-accuracy`) and #491. #468 has landed and is merged here. #506 conflicts there too, and in `scaffold-project.py`, as it does with main. README.md and AGENTS.md merge clean with #505, and `setup-project.py` merges clean with #491. Whichever lands second keeps every footer clause and recomputes the README blob. - **#395 merges clean but one of its tests fails with this branch.** `tests/test_template_text_placeholders.py::test_a_member_whose_id_yaml_would_misread_is_pinned_under_that_id` inits its own holder and runs a real `add` on `main`, which #487 refuses with `family-on-tracking-branch`. Whichever of the two lands second adds `git("switch", "-q", "-c", "family/members", cwd=root)` after that test's `init`; lane 5 is adding it on #395's own branch now, so this branch needs no change for it. (#395's head d526f76 also conflicts with this head in `tests/test_adopt_manifest_values.py`, a file this branch does not touch: that is #395 against main, which the same round resolves.) - On Windows `shell_quoted` speaks PowerShell, the native shell this standard documents (`setup-project.py`), as the doctor's NEXT lines do. `git -C …`, `python scripts\family.py …` and `cd …` are shell-agnostic, so they get the platform's quoting, and no printed line joins two commands with `&&`, which Windows PowerShell 5.1 rejects. - `rm -rf`, `mv` and `mkdir -p` are POSIX commands. They are quoted for a POSIX shell on every host, and the text says "in a POSIX shell (Git Bash on Windows)". PowerShell refuses `rm -rf`'s `-rf` and deletes nothing. - The tests split a host-quoted line with a PowerShell reader on Windows and with `shlex` elsewhere, and split a POSIX-only line with `shlex` everywhere. The real-shell test is skipped on Windows. - On Windows the tests do not execute `rm` or the `mv` lines. They delete the split path with `shutil.rmtree`, and stop after checking the `mv` argv. - Paths: the cached store keeps git's POSIX spelling (`as_posix`), as before. Test paths are `resolve()`d, which expands an 8.3 short name such as `RUNNER~1`. - Nothing here was run on Windows or macOS locally. The `tests-windows` and `tests-macos` CI jobs are the check; on macOS the real-shell test runs the shipped `zsh`. - **A shipped template that imports a newer shared helper can break a holder that kept an older copy.** This change no longer does (`siblings.py` uses `shlex.join`), but nothing stops the next one: `update-shape.py apply --accept-local scripts/repo_shape.py` keeps a `repo_shape.py` that lacks a name a newly copied file imports, and `check` and the validator then read the holder as fine. The general guard is for `apply` to refuse that, as rv456 found for `bootstrap.py` and `spelled` (#456). - **The doctor's `quote_arg` leaves a word that begins with `=` bare**, which zsh expands. `shell_quoted` quotes it; the doctor is left as it is here, and the agreement test excludes that one case. - **The doctor and `update-shape.py` keep their own quoters.** The doctor's `quote_arg` flattens to ASCII by its report's contract, so moving it onto `repo_shape.shell_quoted` is more than an import change. `update-shape.py`'s POSIX `shlex` lines are lane 3's open #374. A test holds `shell_quoted` and `quote_arg` to one answer on every printable ASCII value but a leading `=`. - **`family.py`'s by-hand rollback commands (#365) and its `git config -f .gitmodules` removals (#317, #360) keep `shlex.quote`.** They were already quoted, they are POSIX-only (`printf … | git update-index`), and they are worded for a POSIX shell. - **The holder's `validate-family.py` remedies that name `family.py add`** (`member-gitlink-absent`, `member-gitlink-unnamed`) print no branch step: they are a holder copy, and the `add` they name refuses the tracking branch with its own `switch` remedy. - **`remove`'s store note still computes `.git/modules/<path>`** rather than asking `git rev-parse --git-path`, as `add` does. In a linked worktree it therefore prints no note. That predates this change. - **PowerShell verbs** (`Remove-Item`, `Move-Item`) are not printed for the POSIX-only commands. - **README's `--into` sentence gives the default but not the refusal.** README.md (861) and the handbook (1246) say `--into <dir>` is the PARENT directory, "the directory you are standing in, by default". They do not say that `init` refuses that default inside a checkout of this standard (`family-landing-in-standard`). AGENTS.md item 1 says so, and the README's own example passes `--into ..`. README.md is at its 1422-line cap, so the sentence is on #459's checklist. - **`setup-project.py`'s own `cd <clone>` line and its step-(7) backstop lines print their paths unquoted.** This change quotes only the family lines it prints. Those two predate it and #509, and they are on #459's checklist. Reviewed independently at 10c93de: CHANGES NEEDED (0 blockers, 1 should-fix, 6 nits). The round at bb11b63 fixed the should-fix and all six nits, and its re-verification gave APPROVE WITH NITS (0/0/3), with SonarCloud's quality gate OK; those three are fixed at 3451a56. Main was then merged by GitHub's update-branch (8e1471c), by hand at 708f3bb and at fcbc7a2, where each time only the handbook footer conflicted, and by update-branch again at 22733ce (main 631f087, #503), with no conflict. CI was green there on Linux, macOS and Windows, with SonarCloud passing, but main had moved on: main f391e58 (#266, #509, #496 and #435) was merged by hand at 5a2776d, where again only the footer conflicted. CI was green there too, but main moved on again: main 8cd7dbe (#506 and #347) was merged by hand at 654df49, where once more only the footer conflicted. CI was green there as well, and main d41cc29 (#402) then came in by update-branch at e42ef74, with no conflict. CI is green on Linux and macOS, with SonarCloud passing, at e42ef74, a clean merge of main. All three OSes and SonarCloud passed at 654df49, and under Brett Heap's landing rule of 22:08:06Z (ruling 27) macOS and Windows carry over from that head. Main e6fbd74 (#515, for #514), which landed after it, touches none of its files, and the combined tree passed `tests/test_adopt_pins.py`, `tests/test_shape_doctor.py`, `tests/test_scaffold_leg_path_collisions.py`, the hygiene, AGENTS-code and CLI-reference tests, `tests/test_family_printed_quoting.py` and `tests/test_family_tracking_branch.py` (777 passed, 3 skipped). Closes #488 Closes #486 Closes #487 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a helper that lacks a name a copy it writes imports from it, before anything is written, so a kept `scripts/repo_shape.py` no longer leaves `make bootstrap` dying on an ImportError after the validators passed (#536) (#562) Since #456 both roots' `bootstrap.py` import `spelled` and `escape_unencodable_stdout` from `scripts/repo_shape.py`. A project or holder whose `repo_shape.py` was `both` could keep its own copy with `apply --accept-local scripts/repo_shape.py` while the same run copied a `bootstrap.py` importing a name only the upstream's `repo_shape.py` defines: the validators passed, `check` then read every row `unchanged`, and the first `make bootstrap`, `make siblings` or `bootstrap.py --help` died on an ImportError. #507 took the one other live case out of `siblings.py`; nothing guarded the class. `cmd_apply` now calls `_refuse_kept_copies_missing_names` after `update-conflict`, `update-local-drift` and `update-rewrite-symlink` and immediately before `confirm`, so nothing has been written and nobody has been asked for a yes. It refuses as `update-accept-local-missing-name`, exit 2, when a `.py` copy the run writes (an `upstream-changed` row's target bytes or an `--add` file's) imports, with `from M import`, a name the kept copy `<dir>/M.py` (a `both` or `locally-modified` row `--accept-local` named) does not bind at module scope. Both sides are read with `ast`, from their bytes as Python's own import reads them (a BOM or a coding cookie is honoured); the detail names each kept copy, each importer and each missing name, and its clause that the validators pass is printed only when no importer is a validator; the remediation gives the human's two exits in prose (merge the upstream's version in by hand and re-run with the same `--accept-local`, or drop the edit and take the upstream's bytes, which for a `both` copy is `update-conflict`'s merge again) and prints no NEXT line. What it cannot tell, it does not refuse: a star import, `import M` and an attribute, bytes Python could not parse either, a kept copy with a star import or a module `__getattr__`. An import under a `try` counts unless the first handler that catches ImportError binds that name itself as a fallback; one under `if TYPE_CHECKING:` does not count. The kept copy's bindings include `:=` targets, `match` captures, `type X =` aliases and a `global` declared in a function or class. `check` is unchanged. Tests in a new tests/test_update_shape_kept_helper.py: the issue's case end to end with and without `--branch` (nothing written, no branch, and the premise measured: the target's bootstrap.py beside the kept repo_shape.py dies at `--help`), the two controls, an `--add` file, a family holder's siblings.py, the readers held to every shipped importer on this checkout, a validator as the importer (refused without the validators clause), BOM and coding-cookie bytes, the fallback rule for guarded imports, the newer binding forms, and what the check cannot tell. The reverse direction, a kept importer of a helper this run rewrites without a name it uses, is a known limit on #458's checklist. Closes #536 Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…resume` print the family's name and every member row spelled and never end in a traceback on `name: "a\ud83db"` or a dash under `PYTHONIOENCODING=ascii`, and the holder's README names its folder after the repository, so `--family Quill --name QuillWorks` lays out as its README says (#454, #629) (#642) #454. `templates/family-root/scripts/siblings.py` opened each of its two reports with a plain `print` of `family.yaml`'s `name:` (`_run_placement` for `make siblings`, `dispatch_all` for `--make <target>`, which `make park` and `make resume` call) and printed every member row raw. On origin/main 8f453d0: - `name: "a\ud83db"` is a string holding a lone surrogate, which no UTF-8 stdout can write: `siblings.py --dry-run` ended in `UnicodeEncodeError` at the first line (exit 1), `make siblings`, `make park` and `make resume` exited 2, and `make park`, which exists to save work, parked nothing. - The line holds a literal dash, so under `PYTHONIOENCODING=ascii` it raised for EVERY name, `InkRouter` included. - A raw ESC or CR in a value reached the terminal as the control it is. #391 gave both roots' `bootstrap.py` `escape_unencodable_stdout()` and `spelled()`, written once in `scripts/repo_shape.py`. `siblings.py` now does the same in the same way: - both names join the `from repo_shape import` line it already has, and it defines neither; - `escape_unencodable_stdout()` is the first statement of `main`; - `spelled` is applied to every value a line prints that a manifest, a member row or a disk gave: the name and the root (both headers), the folder and the link, a member's project, path, url and the reason its tracking branch is not known, the `Sibling.row` project and branch, the `FINDING` lines' project, directory, `origin` and `--make-arg`s, and the member's own `[project]` and `---` header lines. The member's own `make` is still handed the RAW target, the `mv` lines a person pastes stay quoted and unspelled (an escape in a command would not run as typed), and git's own words are left as git wrote them. A plain name prints exactly what it printed before: the tests spell out the whole output of both runs for a plain holder, and that golden passes on the base too. #629. `family.py init --family Quill --name QuillWorks` wrote a holder whose README said `cd QuillWorks` and laid the folders out as `QuillWorks/QuillWorks`, with "the doubled `QuillWorks/QuillWorks` is deliberate". `siblings.py`'s `family_folder_name` names the folder after the REPOSITORY (the basename of `repository:`, `Quill`), so a holder laid out exactly as its README said got `WARNING the holder's parent folder is 'QuillWorks', not 'Quill'`, cloned nothing, and `make` exited 2. No test built a holder whose `name:` differed from its repository and ran `make siblings`. - `templates/family-root/README.md` names the folder with `{{FAMILY}}`, the placeholder `family.py init` already fills with `--family` (the same string `family_folder_name` reads out of `repository:`): the `cd`, the two layout lines and the sentence about the doubled name. `{{FAMILY_NAME}}` is left only as the README's title. A sentence says both are named after the REPOSITORY and that the display name names no folder. No new placeholder, and `scripts/family.py` is untouched. - `templates/family-root/AGENTS.md` is unchanged: its two uses of `{{FAMILY_NAME}}` (`the family holder **X**`, `X's own`) are titles and name no folder, which the issue's list had as lines 3 and 7. - `siblings.py` says which name it means. Where `name:` and the repository's name differ, the placement header gets one line under the folder's own, on a stopped run and a passing one alike: `the folder is named after the holder's REPOSITORY, Quill, and never after name: QuillWorks, which is only a display name`. Where they are one string nothing is added. The module docstring says which name `<Family>` is. `--make` asks nothing of the folder and keeps its header. - `family_folder_name` and the README now agree, and a test holds them to it. **Pinned shape copies.** `templates/family-root/scripts/siblings.py` is in `FAMILY_COPIED_VERBATIM` and digest-pinned in every holder: a holder that re-syncs reads `upstream-changed` for it, and a holder pinned since #456 gets that one row alone, since its `scripts/repo_shape.py` already defines both names (no digest of either is recorded in this repository: the files: rows are computed from the bytes each tool writes, as for #391 and #578). `templates/family-root/README.md` is TEMPLATED, not pinned: an existing holder keeps its own README, and only a holder made after this has the corrected one. `scripts/repo_shape.py` is not changed. A holder that keeps its own edit of `repo_shape.py` without the two names `siblings.py` now imports is refused by `apply` as `update-accept-local-missing-name` (#536), which already lists `siblings.py` as an importer. **Files.** - `templates/family-root/scripts/siblings.py` - `templates/family-root/README.md` - `tests/test_siblings_family_name.py` (new) `docs/cli.md` is unchanged: no `--help` text moved, and `AGENTS.md`, `README.md` and `docs/handbook.html` say nothing this makes false ("named after the family" is `--family`'s name, as before). Closes #454 Closes #629 Lane: openreposhape-4 (openRepoShape-4) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>



Lane: openreposhape-5 (openRepoShape-5)
Claim: #391 (comment)
What was wrong
templates/assembly-root/scripts/bootstrap.pymainopened withprint(f"bootstrap: {name} ({root})"), before any step ran, andtemplates/family-root/scripts/bootstrap.pywith the same line (:180, with a literal dash in it).name: "a\ud83db"reads as a string holding a lone surrogate, which no UTF-8 stdout can write.scripts/bootstrap.pyexited 1 withUnicodeEncodeError,make bootstrapexited 2, andvalidate-manifest.py, the validator that names the problem, was never reached.日本underPYTHONIOENCODING=asciiended the same way.—, so under that stdout it raised for EVERY name (InkRouterincluded). The assembly root's own wording has the same dash in two lines and a middle dot in the register readout.bootstrap: a^[b^Mc (...)).Both files are shape copies, digest-pinned in the project that holds them, and a project carries
scripts/repo_shape.pyand noshape_materialize.py, so neither can call #214's helper. Both already importRefusal,find_repo_rootandload_yamlfrom thatrepo_shape.py.What the fix does
scripts/repo_shape.py, the pinned shared library every project and every family holder carries beside itsbootstrap.py, gains two functions and the two sets one of them reads (a new section at its end,scripts/repo_shape.py:2524). Both roots'bootstrap.pyimport the two functions on thefrom repo_shape importline they already had, and define neither:escape_unencodable_stdout()is the first statement of each root'smain. Astrictorsurrogateescapestdout becomesbackslashreplace, the handler stderr already has: what the stdout holds is written as it was, what it cannot hold as\u65e5. Any other handler is left alone and the encoding is never changed. It isvalidate-pins.py's guard from templates/assembly-root/scripts/validate-pins.py: a bare--pin-source <path>whose path holds an=is read as<product>=<path>(raw.partition("=")), so a stand-in clone under a directory nameda=bis never consulted and the pin's digest silently goes togh apiand askip(pins ok, exit 0), or, when the text before the=is a declared pin's name, to aFileNotFoundErrortraceback #295 (validate-pins.py:851), which is inline there and cannot be imported.spelled(value)writes the escape Python spells (\x1b,\r,\n,\u2028,\ud83d,\u202e) for EXACTLY four kinds of character, chosen by Unicode category and not bystr.isprintable:Every other character prints as it did before this change: the zero-width non-joiner (U+200C) a Persian name needs, the zero-width joiner (U+200D) of an emoji sequence, the ideographic space (U+3000) of
山田U+3000太郎, the no-break space (U+00A0), the soft hyphen (U+00AD), any other format, space or unassigned character, and a backslash. The four categories are fixed ranges, so the answer does not move with the interpreter's Unicode database;unicodedatais the standard library's.Helper decision:
repo_shape.double_quoted/_escaped(scripts/repo_shape.py:347,:371) was read first and still cannot serve: it quotes and escapes"and\, so a plain printable name would not stay byte-identical. The two new helpers now live beside it, in the same file, the one library both roots import from.shape_materialize.escape_what_stdout_cannot_encodeis in no project;shape_advisory.printableflattens to?and is in no copy list.Every print of a value the script read from the manifest, a register row or the disk is spelled:
--members-make(heading and failure line). The member's own command is still handed the RAW target.A plain name prints exactly what it printed before: the new tests spell out the whole output of a plain project and a plain holder, and that golden also passes on the base. So does every valid name that holds one of the characters kept raw above (see the tests).
Files
scripts/repo_shape.py:escape_unencodable_stdout(),spelled(),ESCAPED_CATEGORIESandESCAPED_BIDI_CONTROLS, in a new section afterdie();import unicodedata; one bullet in the module docstring's "WHAT THIS MODULE OWNS".templates/assembly-root/scripts/bootstrap.pyandtemplates/family-root/scripts/bootstrap.py: both names on the existingfrom repo_shape importline (the family root's becomes the parenthesised form);escape_unencodable_stdout()first inmain; every print of a value read from the manifest, a register row or the disk goes throughspelled. Against origin/main each differs in nothing else.tests/test_bootstrap_spelling.py(new).docs/cli.mdis unchanged: no--helptext moved.Fix round 1 (review of 118c85a)
The first head's
spelledwrote as an escape every characterstr.isprintablerefuses. Independent review (its S1) showed that is wider than controls, surrogates and line separators: it also refuses every space but ASCII space, every format character and every unassigned code point, so a valid name thatvalidate-manifest.pyaccepts and prints raw on the very next lines was escaped by this script's first line (a Persian name with its ZWNJ, an emoji sequence with its ZWJ,山田U+3000太郎, a no-break space, a soft hyphen). The printedgit submodule update --init <path>command was then wrong to paste for such a leg path, and what was escaped moved with the interpreter's Unicode database (Python 3.9 ships Unicode 13, 3.12 ships 15). The claim "prints byte for byte" was false for those names.The coordinator decided, and Brett may reverse it: escape by category, not by
str.isprintable, the four kinds listed above. Everything else prints raw, as on the base. The bidirectional controls stay escaped on purpose, so a name cannot reorder the text around it on a terminal. The alternative the review offered, and this does not take, is to keepisprintable, correct the claims, and pin ZWNJ and U+3000 as escaped by a test row.The other round-1 changes: the lone-surrogate rows now assert
FINDING manifest-nameand noTracebackunconditionally, since #280 landed in #390 (cb7063f), and themake bootstraprow asserts there is noTracebackeither; the docstrings, the test module's docstring and the unit rows say what is escaped and what is not. The branch has merged origin/main (53bee5b, which carries #390); the merge touched neither bootstrap file.Fix round 2 (CI and SonarCloud)
bootstrap.pythroughimportlib, which wrotetemplates/*/scripts/__pycache__/bootstrap.cpython-*.pyc, andtests/test_scaffold_leg_path_collisions.py, which reads every file of the assembly template as UTF-8, then failed on it.load()now setssys.dont_write_bytecodearound the load, and a regression test asserts that loading either root, from the tree and from a copy in an empty directory, leaves no file behind.spelled()andescape_unencodable_stdout(), with the two sets, were written out once in each root, and a test held the two copies identical. The premise for the copies, that a root cannot import anything of this repository's, was false: both bootstraps already import fromscripts/repo_shape.py, which every project and every family holder carries. Brett Heap decided to move the two helpers intorepo_shape.pyand have both roots import them, with no Sonar exclusion. The code moved byte for byte (the AST of each function and set, docstrings aside, equals both old copies); only the docstring sentences the move made untrue are reworded. The new section,import unicodedataafterimport sys, and the docstring bullet afterRefusalsit where none of the open pull requests that also editrepo_shape.pyedits: scaffold-project.py, validate-pins.py, repo_shape.py:open-glassandopen_glassshare oneSHAPE_PIN_SOURCE_<PRODUCT>, so--pinrefuses the pair andvalidate-pins.pynotes a declared one and reads the variable for neither (#275) #409, validate-repository-naming.py, repo_shape.py: a--link-sourcevalue isPRODUCT=PATHonly when its key names a chain link, and a link source that cannot be entered is a WARNING with the system's reason, not a traceback (#338) #453 and adopt-project.py:executerecords inproject.yamlwhich leg the plan gave each submodule a leg keeps, and shape-doctor.py holds the submodule to that row, so one in the other leg is MISPLACED and a real adoption's placement readsok(#189) #394 each merge with this head with no conflict, in that file or any other (git merge-tree --write-treeagainst each branch's head, 2026-10-12). family.py:bumpandremoveput the holder back when a hook, a lock or git refuses them after they write, and say what they put back or what is left with the commands that put it back; they ask first what their dry runs ask first; andremoverefuses a heldindex.lockbefore it unmounts and a.gitmodulesthat gives the member's path to two submodules, of whichgit rmtook out one (#365, #360) #416 landed as 580d7df without touchingrepo_shape.py.git merge-tree --write-treeof this head with current main is clean (2026-10-12).How it was tested
tests/test_bootstrap_spelling.py(new, 205 tests at this head, local bare remotes, nogh). Answered in bytes, stdout decoded strictly in the encoding the run was given (stderr withreplace), underutf-8:strict,utf-8:surrogateescape,cp1252:surrogateescape,cp1252:strict,ascii:strictandutf-8:replace:日本, newline/tab/DEL, U+2028, the right-to-left override (U+202E) with the right-to-left mark (U+200F), an emoji, a quote and a backslash, two lone surrogates;山田U+3000太郎, a soft hyphen, a zero-width space with a byte-order mark. For each, the first line is compared in BYTES with what the plainprintit replaced wrote (assembly and family), and a full run with the validators shows the validator's ownmanifest ok: <name>line raw beside it;--members-make(withMAKEthe interpreter, so a member runs only if the RAW target reached it);FINDING manifest-name,make bootstrapexits 2 andbootstrap FAILED: manifest (exit 1), with noTracebackand no frame inbootstrap.py;spelledandescape_unencodable_stdoutdirectly, inrepo_shapeitself, once rather than once per root: every code point swept once against a set written from the specification (explicit ranges), which is itself checked againstunicodedata.category; that each root'sfrom repo_shape importnames both and that neither root defines or assigns either function or either set (anastcheck of the file); that the name each root's module exposes ISrepo_shape's object; that the reconfigure ismain's first statement; and that loading a root writes no bytecode undertemplates/.The end-to-end rows run a real scaffolded project and a real family holder, and each carries
scripts/repo_shape.py, so they run the moved code where it ships.Results (
Python 3.12.3, Linux; head d0aea3e = the move, on e4b0386 = the merge of origin/main ec8e09a):test_bootstrap_spelling.py,test_scaffold_leg_path_collisions.py,test_stdout_encoding.py,test_template_quoted_values.py,test_template_values_single_pass.py,test_repo_hygiene.py,test_agents_md_codes.pytests/test_stdout_encoding.py:286, Windows-only)test_yaml_*.py, the sixtest_update_shape*.py,test_scaffold_e2e.py,test_family.py,test_validate_family_unnamed_mounts.py,test_cli_reference.py,test_windows_paths.pytests/test_update_shape_add_symlink.py:831, NTFS junctions)repo_shape.spelledescapes nothingtest_bootstrap_spelling.pyagain, the mutation revertedEarlier heads, not re-run on this one: on af26b19 the then 260-test file gave 197 failed, 63 passed against the base (cd1fc79) templates, 59 failed with both copies'
spelledreverted tostr.isprintable, and caught all 12 mutations of the category sets; reverting each hunk of the first round's fix, one at a time, still failed a new test (34 mutations, listed in the report); on 118c85a the existing bootstrap, scaffold, adopt, update-shape and doctor tests gave 496 passed, 1 skipped, and 305 passed. macOS and Windows have not run here; CI runs them on this head. Every new test reads and writes bytes and compares againstproject.resolve().What the rest of the run says (the lone-surrogate and
make bootstraprows assert it at this head): a lone-surrogate name printsbootstrap: a\ud83db (<root>), reaches step (b), and endsexit 1 (
make bootstrap2), with noTracebackon a pipe, a TTY ormake.What is left out
validate-manifest.py'smanifest ok: <name>still prints through a stdout with no handler: a valid日本underasciiends in ITS traceback (the issue's "Not in this issue"), and an ESC or CR name is accepted and the line writes the raw control.tracking_branch: "a\ud83db"(or a leg'sbranch:) raises fromsubprocessinside step (a), before the validators; a NUL does too (ValueError: embedded null byte). Reproduced on origin/main 53bee5b.validate-manifest.pynever reads either key, so it accepts the manifest (manifest ok, exit 0) and NO tool names the value. Not aprint; follow-up below.print(str(exc), file=sys.stderr)(Refusal text built byrepo_shape, multi-line, manifest textrepr-escaped; stderr cannot raise) and the family'sFETCH INCOMPLETEtext (git's own words).templates/family-root/scripts/siblings.py, which carries the SAME first line twice (:676indispatch_all,:906in_run_placement) with the same plainprint, and prints member rows raw.make siblings,make parkandmake resumeonname: "a\ud83db"still end in aUnicodeEncodeErrortraceback (reproduced on origin/main), and underPYTHONIOENCODING=asciithey end so for every name (the literal dash). It already imports fromrepo_shape, where both helpers now are, so they are two more names on its existing import line. Follow-up below.\u200ffor it;\udcXXwhere the rootbootstrapshim's adopt-project.pyplanraisesUnicodeEncodeErrorin its terminal report, after the plan file is written, for a printable name stdout cannot encode (日本.mdunder cp1252, as on a redirected Windows stdout), soplanexits 1 beforeplan written to#214 handler wrote the raw byte back;\x1bprints as one holding ESC does;\u2014and\xb7on a stdout that cannot hold them, instead of raising.repo_shape.pyis recorded in this repository. The scaffold,family.py initand adopt compute eachfiles:row from the bytes they write, andupdate-shape.pycompares upstream blobs at two commits. A project scaffolded from origin/main 580d7df, checked against this head (d0aea3e), readsupstream-changedforscripts/bootstrap.pyandscripts/repo_shape.pyand nothing else (assembly root: 2 changed, 9 unchanged; family holder: 2 changed, 8 unchanged).applycopies both and re-pins, the project's own validators say11 copied shape file(s) match their digests(the holder's, 10), each copy equals its upstream blob, andscripts/bootstrap.pyruns. Run for real, local remotes only.scripts/repo_shape.py(checkreads itboth, andupdate-shape.py apply --accept-local scripts/repo_shape.pykeeps the project's bytes) while taking the newbootstrap.pymeets anImportErrorthe next time it bootstraps:scripts/bootstrap.pyends incannot import name 'escape_unencodable_stdout' from 'repo_shape', exit 1. Followingcheck's own instruction avoids it:checksays to merge upstream's bytes into the edited file by hand, commit it, and only thenapply --accept-local, and a merge that takes the new section brings both names. A project that skips the merge gets the error and nothing reports the break afterapply: its validators still pass, because they import neither name;checkthen reads everything unchanged; and the project'svalidate.ymlnever runsbootstrap.py, so its CI stays green. The first command to fail is the one run aftergit clone. Guarding it is an item on Tracking: update-shape.py and bump-leg.py, keeping a project's shape copies and leg pins current #458's checklist (Tracking: update-shape.py and bump-leg.py, keeping a project's shape copies and leg pins current #458 (comment)):applywould refuse a kept--accept-localcopy that lacks a name another copied shape file imports from it. It is the same coupling as the names the bootstraps already import from that file. Run for real on a project scaffolded from 580d7df.Follow-ups found by lane 5's review of this branch, each reproduced on origin/main and filed separately:
siblings.py's two first-line prints (and its member rows):make siblings,make parkandmake resumeon a lone-surrogatefamily.yamlname end in aUnicodeEncodeErrortraceback (family-root siblings.py:make siblings,parkandresumeprint the family'snamewith a plainprint, soname: "a\ud83db"ends each in aUnicodeEncodeError, and the literal dash ends every name underPYTHONIOENCODING=ascii#454).tracking_branch(or a leg'sbranch:) endsmake bootstrapin a subprocess traceback in step (a), andvalidate-manifest.pyaccepts the manifest (assembly-root bootstrap.py: a lone surrogate or NUL intracking_branch(or a leg'sbranch:) endsmake bootstrapin a subprocess traceback in step (a), andvalidate-manifest.pysaysmanifest ok, so no tool names it #455).make siblings,parkandresumeprint the family'snamewith a plainprint, soname: "a\ud83db"ends each in aUnicodeEncodeError, and the literal dash ends every name underPYTHONIOENCODING=ascii#454 and assembly-root bootstrap.py: a lone surrogate or NUL intracking_branch(or a leg'sbranch:) endsmake bootstrapin a subprocess traceback in step (a), andvalidate-manifest.pysaysmanifest ok, so no tool names it #455 (open, off the milestone) and the two Tracking: setup, scaffold, family and the shared helpers, creating projects and families #459 checklist items moved from bootstrap.py, the assembly root's template and the family holder's, ends in aFileNotFoundErrortraceback, exit 1, when git is not on PATH, because the root's step (a) callsrecorded_gitlink, its offline upstream notice calls_try_gitand the holder's step (a) runsgit submodule update, and nothing catchesOSError, where the docstrings say step (a) "never raises" and the notice "is silent when it cannot answer", so the first command a new engineer runs after cloning answers a missing git with a stack trace and the code its ownbootstrap FAILEDline uses: the third sibling of #368 and #412 #463 and scripts/validate-repository-naming.py, a pinned copy in every project, ends in aUnicodeEncodeErrortraceback, exit 1, when it prints a name that is not UTF-8 (--explainand the plain listing, under a strict UTF-8 stdout such as the one the doctor gives its children) and in aUnicodeDecodeErrorwhen it reads one from stdin, though--quietanswers the same name withFINDING naming-unclassified, so a directory made on a Latin-1 machine gets a crash where the script's own exit codes give 1 to a name that matches no family and 2 to a question that could not be asked #464 can importspelledandescape_unencodable_stdoutfromrepo_shape.py, wherever they print a value.Closes #391
🤖 Generated with Claude Code