Repository navigation
tests/conftest.py: the session turns git's automatic maintenance off, in command scope and in a global config that includes the person's own, so a copytree of a fixture no longer meets a detached repack deleting its loose objects (#645) - #713
Conversation
… in command scope and in a global config that includes the person's own, so a `copytree` of a fixture no longer meets a detached repack deleting its loose objects (#645) `shutil.copytree` walks a repository's `objects/` one file at a time, and the suite copies a repository a fixture has just committed in, fetched into or pushed to in 139 places: `project`, `test_family.py`'s `holder`, ten copies of bare remotes and two of a `.git` directory. Every `git commit`, `merge` and `fetch`, and the `receive-pack` a push starts in the remote, runs `git maintenance run --auto` as it finishes. Since git 2.47 that run detaches and holds `objects/maintenance.lock`; since git 2.54 its default `geometric` strategy repacks a small repository and deletes the loose objects it packed. A copy taken in that moment dies with `[Errno 2]`, as `holder` did on the Linux runner. A session autouse fixture, `no_auto_maintenance`, sets `gc.auto=0` and `maintenance.auto=false` in two scopes: - command scope (`GIT_CONFIG_COUNT`), appended after the entries the environment already holds: every git the suite or a tool under test starts inherits it, a `git submodule`'s children too, and it outranks every config file; - a global config file (`GIT_CONFIG_GLOBAL`) that includes the global config git would read without it and sets the two keys after it, because git starts the `receive-pack` of a push to a local path or a `file://` URL with `GIT_CONFIG_COUNT` and `GIT_CONFIG_PARAMETERS` removed. `tests/test_no_auto_maintenance.py` asks git what it starts and reads, through trace2 events and a `post-receive` hook: a commit, a fetch and a merge start no maintenance, with the session's global file and with a test's own; the scaffold and the remotes it pushes to start none; a remote's `receive-pack` reads both settings off; the entries are appended; and a person's own global config is still read beneath them. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es `test_doctor_pin_restore_merges.py`'s `the_standard_moved` beside `holder`, and says a `git config --global` read without `--includes` sees the two alone (#645) Lane 2 met the same race on #623's Linux run (run 38895011713): a setup error in `test_copies_neither_side_edited_are_taken_whole_with_the_pin`, `shutil.Error ... standard5/openRepoShape/.git/objects/1e ... [Errno 2]`. `the_standard_moved` (`tests/test_doctor_pin_restore_merges.py:99`) copies the session's `standard`, which `test_shape_doctor.py` commits in, and at :102 copies `moved` just after `moved_on` commits in it. Both commits run under `{**os.environ, ...}`, so the session's command scope already turns their maintenance off; the note now names the site. `git config --global <key>` reads the global file without following its includes unless `--includes` is given, so under the session's file it sees `gc.auto` and `maintenance.auto` and nothing of `~/.gitconfig`. The one such read in this repository is `setup-project.py`'s Windows-only `core.autocrlf` warning, and no test reads that from a person's own config. A read with no scope, the kind git itself makes, follows the include. Comment only. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
(lane openRepoShape-5, 2026-10-16T13:54Z) @codex review |
Reviewer's GuideThe PR hardens the test suite against Git 2.47+ background maintenance races during repository copies by applying File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: accc7d2afe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if named: | ||
| included = [Path(os.path.abspath(named)).as_posix()] | ||
| else: |
There was a problem hiding this comment.
Preserve an explicitly empty global-config override
When the test process starts with GIT_CONFIG_GLOBAL explicitly set to an empty string, Git treats that as an override that reads no user-global configuration, but this truthiness check treats it as unset and includes ~/.config/git/config and ~/.gitconfig. This can unexpectedly reactivate settings such as commit.gpgSign=true, causing fixture commits to fail for users or CI that deliberately use GIT_CONFIG_GLOBAL= to isolate the suite; distinguish None from the empty-string override and include no original global file in the latter case.
Useful? React with 👍 / 👎.
…s global file includes a `~/` path only where `HOME` is set and includes nothing over an empty `GIT_CONFIG_GLOBAL`, a test holds the fixture to session scope, and the receive-pack test names its own hooks, so a runner with no `HOME` no longer dies on every git (#645) The review of #713 (rv645) found two things worth fixing and three small ones: - With `HOME` unset, the include of `~/.config/git/config` and `~/.gitconfig` made every git the session started exit 128 ("could not expand include path"). `session_global_config` now writes a `~/` path only where `home_is_known()` (`HOME` is set, or this is Windows), and leaves out the `[include]` header when nothing is included. An `XDG_CONFIG_HOME` that is set is still included. The two `test_a_persons_own_global_config` cases skip where `HOME` was unset. - Nothing held `no_auto_maintenance` to session scope: at function scope all seven tests still passed. A new test reads the two variables from a session fixture built first, and compares them with what the test sees. - An empty `GIT_CONFIG_GLOBAL=` is git's "no global config"; the fixture treated it as unset and included `~/.gitconfig` again. It now includes nothing. - The receive-pack test set no `core.hooksPath` of its own, so a person's global one hid the remote's `post-receive` hook. It now sets the remote's own `hooks` directory in the remote's local config, which outranks a global one. Two tests are new for the `HOME` and empty-value cases, with an XDG variant. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e one helper that writes the session file, reads both settings back and returns the probe, so the fix round adds no duplicated test body (#645) The `HOME`-unset test and the empty-`GIT_CONFIG_GLOBAL` test each wrote `session_global_config(...)` to a file, set `GIT_CONFIG_GLOBAL` and looped over the two settings. That block is now `probe_read_from_the_session_file`, so SonarCloud's duplication gate has nothing to count. No assertion changes: the new file still gives 11 passed, and the same probes fail. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…nce-race) into fix/662-insteadof-helper No conflicts: #713 changed session_global_config, added home_is_known() above it, the comment block and tests/test_no_auto_maintenance.py; this branch's hunks are the insteadof helpers above clone_just_pushed, the wrapper, read_through_git and the sites. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
|
Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days. You can request another review in 12 hours and 31 minutes by commenting |
Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ewrite that `clone_just_pushed`, `read_through_git` and the adopt tests' `pasted` each built, and the empty-mount `submodule update --init` sites use it (#662) (#748) Stacked on #713 (`fix/645-fixture-copy-maintenance-race`, approved head `979bb394`). The branch head is `deb029c9`. Its history is my commit `4149cbb5`, a merge of `979bb394` (`5c06c115`, no conflicts), and a fix round after rv662's review (`deb029c9`). Against `979bb394` the diff is the two commits' files: `tests/conftest.py`, `tests/test_adopt_e2e.py`, `tests/test_clone_just_pushed.py`, `tests/test_doctor_legs_members_next.py` and `tests/test_family_bump_remove_rollback.py`. Retarget to main once #713 lands. The change is test only. - `tests/conftest.py` has one helper, `insteadof_prefixes(remotes, typed=False)`: the directory resolved, in both of a Windows path's spellings, each with a separator after it, and the directory as typed first when asked. `insteadof_entries` pairs each prefix with the one `url.<file URI>.insteadOf` key, and `insteadof_args` gives the same as `-c` arguments. - `clone_just_pushed` passes the arguments. `read_through_git` appends the entries to the `GIT_CONFIG_*` the environment already holds. It used to overwrite the count, which dropped #713's two settings from a pasted line's command scope. Its result is only right laid over `os.environ` at the same moment, as `pasted` does, and the docstring says so. - `pasted()` in `tests/test_adopt_e2e.py` was a third copy of the rule and now uses the helper. - `submodule_update_through_git(root, remotes, *options)` sits beside `clone_just_pushed` and runs `git submodule update` with the file protocol allowed and the helper's `-c` arguments. I traced every `git submodule update` the tests make (GIT_TRACE), before and after. A local clone shows as a `git clone` of a plain path with no `pack-objects`; the transport shows `pack-objects` and `index-pack` once per clone. - `test_family_bump_remove_rollback.py` :337 and :908: both legs of a member are cloned. Now the wrapper. - `test_doctor_legs_members_next.py` :631 (the `code` leg, store removed by `unbootstrap`), :987 (the member and its two legs) and :1054 (the member's two legs, `with_its_own_legs`): all clone. Now the wrapper. - `test_doctor_gitlink_repairs.py:343` and `test_doctor_unfinished_merge.py:145,171`: no clone and no fetch (the store exists). Unchanged. - `test_doctor_legs_members_next.py:607`: the deliberate failure probe. Unchanged. No rule separates a clone of the session's remotes from a clone of a test's own `source`. - What makes a clone racy is how the repository got its objects (a push, a commit, a copy), not how the call reads. The 55 `clone` call sites in tests/ name their source `source`, `bare`, `remote`, `origin`, `seed`, `work`. - In the adopt tests `source` is cloned plain before `execute` (a legal clone) and is the target of `execute`'s push after it (cloned then with `clone_just_pushed`). Only the order tells them apart. - The one spelling a name rule would catch, `remotes / f"{SPARE}.git"`, clones a `shutil.copytree` copy the test made itself. - The three `--recurse-submodules` hits assert on printed text; no recursive plain clone of a session remote is left. The real sites were `submodule update --init`, which the wrapper covers. - After #713 auto maintenance is off in every repository the suite makes. A limit: no test fails if a site goes back to a plain `submodule update --init`. rv662 notes a narrower test is possible (refuse any `"submodule", "update"` call outside conftest and a four-entry allowlist: the sites above that are left alone, and the probe at :607). It would guard six adoptions and was left out as a v1.1.1 choice. New cases in `tests/test_clone_just_pushed.py` (19 tests in the file now, 18 before the fix round): the prefix list for a typed and a resolved directory, each given with no trailing separator, "/" and the system separator; a symlinked directory; the entries and the `-c` form; `git ls-remote --get-url` through both forms (a plain path goes to its file URI, a neighbour called `remotes-other` is left alone); the environment form keeps the entries the environment holds; the environment form reads a path typed through a symlink (added in the fix round, `a_link_to` makes the link); an `--init` through the wrapper arrives as one pack, and the same `--init` plain leaves the remote's loose objects loose. - On `deb029c9`, after the merge and the fix round: `tests/test_clone_just_pushed.py` and `tests/test_no_auto_maintenance.py`: 30 passed; `tests/test_repo_hygiene.py`: 77 passed. - On `4149cbb5`, before the merge (the merge changes none of my files except the hunk positions in `conftest.py`): - `tests/test_family_bump_remove_rollback.py`, `test_doctor_legs_members_next.py`, `test_doctor_gitlink_repairs.py`, `test_clone_just_pushed.py`, `test_no_auto_maintenance.py`, `test_bump_leg_paste_lines.py`, `test_adopt_e2e.py`: 259 passed; - `test_scaffold_e2e.py`, `test_setup_project_py.py`, `test_setup_sh.py` (the `read_through_git` users): 279 passed; - `test_shape_doctor.py`: 405 passed, 3 skipped (the PowerShell line, Windows only); - the other 48 files that use `clone_just_pushed` or `cloned_just_pushed`, in two runs: 1980 passed, 9 skipped, and 1194 passed. - Mutants of the helper, the wrapper and the two users, each run against `tests/test_clone_just_pushed.py`. Each fails at least one new test: - no trailing separator on the prefixes: 9 failed (the separator cases, the `ls-remote --get-url` case and the held-entries case); - typed adds nothing, typed after resolved, and the key built from the typed URI: 1 failed each (the symlink case); - `read_through_git` overwrites the count: 1 failed (held entries); - the wrapper without the rewrite: 1 failed (the wrapper's one-pack case, `8 loose objects`); - `clone_just_pushed` without the rewrite: 2 failed (existing cases); - `read_through_git` with `typed=False` (rv662's N3, which survived at `5c06c115`): 1 failed on `deb029c9`, 18 passed, the new symlink case; unmutated 19 passed. `git merge-tree` is clean for `deb029c9` against origin/main `3726ada2` (and earlier `55c6ba7d` and `20c8873b`) and against #713's head `979bb394`. Checked at `4149cbb5`, before the merge, which changed none of my files: `fix/663-non-utf8-name-tests` `0b1d1ddb` (#723) and every `fix/*` and `l3-*` head on origin (35 refs). - Five tests build the same append-to-`GIT_CONFIG_COUNT` arithmetic by hand beside conftest's `appended_to_command_scope`: `command_scope` in `test_adopt_git_reads.py:91`, `test_adopt_ignored_shape_files.py:579` and `test_adopt_nested_attributes.py:377`, and inline in `test_bootstrap_after_bump.py:151` and `test_clone_just_pushed.py:96`. A follow-up. - Six `git clone --no-local` calls in tests build `clone_just_pushed`'s options by hand (`test_bump_leg_dry_run_truth.py:98,110`, `test_doctor_legs_members_next.py:1824,1925`, `test_doctor_pin_restore_merges.py:714`, `test_family_bump_to_checks.py:235`). rv662 checked that the ones it read are non-recursive. A follow-up. - `PASTE_ENV` (rv662's candidate, v1.1.1): `tests/test_doctor_legs_members_next.py:64` is a module-level snapshot of `os.environ` taken at import time with `GIT_CONFIG_COUNT` set to `"1"`, which overwrites. The 34 `paste(...)` calls run under neither #713's maintenance settings nor the `insteadOf` rewrite, and a pasted `git submodule update --init --recursive --checkout` still local-clones (`test_a_plain_clones_empty_mounts_are_cured_by_the_pasted_line` shows `git clone --no-checkout … remotes/Atlas-code.git` and no `pack-objects`). The same class as this issue, and not on its list. The fix is to build the paste environment with `read_through_git(remotes)` appended to `os.environ` at call time. **Review.** rv662 re-verified the fix round: APPROVE at `deb029c9`. The diff is the two asked-for changes only, mutant N3 now fails the new symlink test, and `merge-tree` is clean against `3726ada2` and `979bb394`. Review: rv662, lane 5's independent review, approved deb029c once its SHOULD-1, SHOULD-2 and NIT-1 were done. CI (Brett Heap's ruling 27): H0 deb029c passed tests, tests-git-bash and SonarCloud; the head 8368d1f merges #713's final head and then main after #713 landed as 741679e, taking the branch's own side for #713's files, with tests, tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree bc67aa8 over main 64ec47c: the adopt_e2e, adopt_plan_yaml_quoting, bootstrap_after_bump, bootstrap_leg_moves, bootstrap_without_git, clone_just_pushed, clone_url_bytes, doctor_legs_members_next, family, family_add_member_variant, family_add_spellings_remedies, family_bootstrap_gate_order, family_bump_remove_rollback, family_head_lost_gitmodules, family_members_case_variants, family_switch_branch_exists, local_remote_dir_urls, reader_refusals_named, shape_pin_row_path, siblings_inputs, validate_family_branch_step, validate_family_remedies_typed, yaml_indicator_openers, repo_hygiene, agents_md_codes, cli_reference sets, 1748 passed. Landed on Brett Heap's word of 2026-10-17T10:59:39Z, relayed by lane 3 on #473: "merge lane 5's approved v1.1.0 drafts". Closes #662 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mand-scope, and tests/test_local_remote_dir_urls.py calls `allow_file_protocol()`, so #812's test file still imports once #758 lands (#758) The retarget after #748 landed as 431c8cf. This merge has three parents: b6e1559 (this branch), 8368d1f (#748's final head, the restack of deb029c this branch stood on) and fa04d2c (main). Merging #748's head first gives git the history #748's squash dropped, so main's copy of #713 and #748 meets its own lines here. Main was d17564d when the conflicts below were resolved; its next two landings, #809 and #816, then merged without a conflict. Conflicts, and how each was resolved: - tests/test_adopt_empty_symlinks.py, against 8368d1f, which carries #388 through main 741679e. Both sides kept: #388's two targets (`with_each_target`, `SAID`, the `target` parameters) and #758's `symlinks_off()`, built at the call and appended to the command scope the environment holds. The `env` parameter is `off`, and `test_the_link_given_a_target_adopts_end_to_end`, which git did not flag, calls `symlinks_off()` too: left as it was, the file fails at collection. - tests/test_clone_just_pushed.py, tests/test_doctor_legs_members_next.py and tests/test_family_bump_remove_rollback.py, against main d17564d. Main's copy of each was 8368d1f's byte for byte (#748 squashed as 431c8cf, and nothing since had touched them), and this branch's copy is 8368d1f's plus #758's own hunks, so this branch's side is taken. Against main, the merged tree differs in #758's 25 files and in tests/test_local_remote_dir_urls.py. Each of the 25 differs by #758's own added and removed lines, tests/test_adopt_empty_symlinks.py aside. rv818's MUST 1 (lane 2): tests/test_local_remote_dir_urls.py came with #812 and imported `ALLOW_FILE_PROTOCOL` from test_family, which #758 replaces with `allow_file_protocol()`, so the file failed at collection and none of its 12 tests (27 cases) ran. It now imports `allow_file_protocol` and calls it at the paste. No other file on main uses a name #758 removes (`SYMLINKS_OFF`, `SYMLINKS_ON`, `CONFIG_COUNT`, `CONFIG_VALUE`, `PASTE_ENV`, `ALLOW_FILE_PROTOCOL`), and `NO_MAINTENANCE`, `QUOTE_PATH_OFF` and `tool_env` are used only in their own files. Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main gained #713 (#645), #812 (#693), #767 (#685), #745 (#704, #694) and #805 (#690) since 2000488. No conflict, and none of them touches README.md, the handbook, AGENTS.md, docs/ or the README cap's ledger, so every handbook marker reads as before. README.md is 1475 lines, its cap; AGENTS.md 494, its cap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…notes entries for the five pull requests landed since 2000488, current to main 40e678e (#563, #683) RV630's review of rounds A3, A4 and B1 (#630 issuecomment-6233641684, READY), its four optional nits: - 1: README.md's drift paragraph ended on the one-word line "finished.". "exactly " and "about the shape " are cut, and the paragraph re-wraps to seven lines with both bold leads whole. README.md is 1474 lines, its cap (1422 + 2 + 5 + 12 + 22 + 10 + 1); tests/readme-cap.d/683.txt is +1, and its text says why. Both `Keeping a project's shape current` markers move to df780eb5; the other 21 read current. - 2: #682's lead reads "A holder's `make bootstrap` that left a member unchecked-out". - 3: #730's entry says "Comments and docstrings", not "Four": its squash makes three claims, comments only. - 4: the doctor's `members` row has one bullet, #705's sentence and #624's together. Wave B2 so far, `git log --first-parent 2000488..40e678e`, each entry from its squash diff: - #713 (#645): the test session turns git's automatic maintenance off (Install, CI and the docs). - #812 (#693): `add --local-remote-dir`'s member clone URL, the remedies and the READMEs' quoted clone URL (the `--local-remote-dir` bullet). - #767 (#685): `members/` renamed with `git mv` is renamed back in two steps, and the git a printed `--sparse` needs (family). - #745 (#704, #694): lane 5's line (#630 issuecomment-6227129879), with v1.0.0's known limit at v1.0.0.md:463 (no git on `PATH`) said closed (Bootstrap). - #805 (#690): `make siblings`'s inputs (family). The counts (111 pull requests; 122 of 135 milestone issues closed at 16:14Z), "current to main" and the open list (13, each with a pull request open) are at main 40e678e, still placeholders. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#813 and #723, v1.0.0's quoted-header and plain-path-clone limits closed and the counts current to main bad05d2 (#563) Each entry is the one RV630 corrected in its pre-review of the B2 stage, put at its anchor, and checked against its squash on main: the `git merge-tree --write-tree` of main before the squash with the head the entry was written from equals the squash's tree for all seven. - #810 (#674, `d17564d9`): a new bullet, "`apply`'s lines, its branch and its headers.", after "Headers, branches and messages.". It closes v1.0.0's quoted `files:`/`shape:` header (v1.0.0.md:439) and `apply`'s half of the paths listed as staged (:433); `bump-leg.py`'s half waits for #746. - #731 (#692, `d66a9054`): appended to "Folders, asked before they are made.". - #813 (#689, `f6d17bc5`): appended to "`--tracking-branch`.". - #809 (#688, `42aed788`): a new bullet, "What `family.py init` records and commits as.", after "Names that are not UTF-8.". - #816 (part of #684, `fa04d2cd`): a new bullet, "More holder states, named.", after "family.py refuses before it writes.". #816 closes no issue; #684 keeps its reftable case and is on v1.1.1. - #748 (#662, `431c8cf8`), then #723 (#663, `bad05d21`): in the CI bullet, after #713's sentence. With #554 and #578, #748 closes v1.0.0's limit on fixtures that clone a just-pushed remote by plain path (:530). The counts, in the header comment, the counts paragraph, "What changed" and the open list: main `bad05d2`; 120 pull requests since `bfed143`; at 18:02Z, 3 of the `v1.1.0` milestone's 133 issues open and 130 closed. The open list is #563 (#630), #672 (#746) and #686 (#795); #684 and #695 (#738) are on v1.1.1. #746 and #795 have not landed, so neither has an entry yet. Nothing since 2000488 touches pyproject.toml, src/openreposhape/, install.sh or .github/workflows/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…holding a control character before they name it, and install.sh refuses a `:` before its fetch, so a carriage return or an escape is no longer echoed as typed (#760) (#804) Against main. #726 landed as `dcf62ed`, and this branch is restacked onto main as `7e4181c0`: it merges #726's final head `a22b94c4`, then main, taking this branch's side in the four files #726 also changed. Over main it carries exactly its own two commits: `git diff a3cde99 efd061a` and `git diff origin/main 7e4181c` have the same patch-id. #667's two checklist candidates, with rv667's NIT 2. Two commits on a3cde99: **7569ffa8**, and the fix round's **efd061a7**. Against a3cde99, `git diff --numstat`: `install.sh` +40/-3, `openRepoShape` +30/-16, `tests/test_install_sh.py` +132/-2, `tests/test_openreposhape_command.py` +176/-21, and nothing else. README, the handbook, `AGENTS.md` and `docs/cli.md` are unchanged (`render-cli-reference.py --check`: ok): no documented sentence names these refusals, and neither file is a pinned shape copy, so no `upstream-changed` row follows. One new commit, **efd061a7**, on 7569ffa, pushed to `fix/760-bin-dir-control-chars` (a fast-forward). The PR is now two commits on a3cde99. `git diff --numstat 7569ffa efd061a`: `install.sh` +12/-7, `openRepoShape` +14/-11, `tests/test_install_sh.py` +34/-29, `tests/test_openreposhape_command.py` +61/-35, and nothing else. Against a3cde99 the PR is `install.sh` +40/-3, `openRepoShape` +30/-16, `tests/test_install_sh.py` +132/-2, `tests/test_openreposhape_command.py` +176/-21. README, the handbook, `AGENTS.md` and `docs/cli.md` are still unchanged. The red Linux run on 7569ffa was #645's `copytree` race (`shutil.Error` in `test_doctor_pin_restore_merges.py`'s setup), which #713 fixed on main. It is not this PR's fault, and nothing here touches it. ### Finding 1 (should-fix): taken. A C1 control passed the arm under a UTF-8 locale when the value also held a byte that was not UTF-8 **Reproduced first, on 7569ffa, with bash 5.2.21.** `LC_ALL=C.UTF-8`, `$OPENREPOSHAPE_BIN_DIR` set to `<dir>/caf`, the byte 0xE9, U+009B, `31mFORGED/bin`, and `openRepoShape --install`: exit 0, the command placed, and the value printed raw in four lines (`od -c` of the output shows `c a f 351 302 233 3 1 m F O R G E D`). It is lane 1's reproduction, and it is the failure the finding names. Cause: for a value that is not valid in the locale's multibyte encoding, bash matches byte by byte, and read one byte at a time U+009B's two bytes (`0xC2 0x9B`) are not controls, as under `C`. **The fix is lane 1's, tested by them, in both files** (`install.sh`'s `main` and `openRepoShape`'s `install_commands`), in place of the `case`: ```sh if (LC_ALL=C; case "$bin_dir" in *[[:cntrl:]]* | *$'\302'[$'\200'-$'\237']*) exit 0 ;; *) exit 1 ;; esac); then die "…unchanged…" fi ``` The command's has `"$dir"`. Under `LC_ALL=C` the class is the ASCII controls and DEL, byte by byte, and the second pattern names the two bytes of each of U+0080 to U+009F (`0xC2`, then `0x80` to `0x9F`). The arm is the same in every locale. The subshell leaves the locale of every later line as it was. I added a `*) exit 1 ;;` arm so the construct has a default case (finding 2). Both files' comments now say what the check does, and no longer say which C1 controls match is the locale's. **Measured, command `--install` (bash 5.2.21; each cell is exit, whether the command was placed, whether the value reached the output):** | value | locale | 7569ffa | efd061a | |---|---|---|---| | U+009B | `C`, `POSIX` | 0, placed, in 4 lines | 2, nothing placed, nothing echoed | | U+009B | `C.UTF-8` | 2 | 2 | | U+0080, U+009F | `C`, `POSIX` | 0, placed | 2, nothing placed | | U+0080, U+009F | `C.UTF-8` | 2 | 2 | | 0xE9 then U+009B then `FORGED` | `C`, `POSIX`, `C.UTF-8` | 0, placed, in 4 lines | 2, nothing placed, nothing echoed | | a `:`, 0xE9, U+009B | all three | 2, the colon refusal names the directory, the C1 control in its line | 2, the control-character refusal, nothing of the value | | an e with an acute accent, U+00A0 | all three | placed | placed | | a lone 0x9B byte | all three | placed | placed (see the checklist) | | U+2028 | `C`, `POSIX` | placed | placed | | U+2028 | `C.UTF-8` | **2** | **placed** (see below) | **Measured, install.sh with a fake `curl` that logs each request and exits 22, a version named:** | value | locale | 7569ffa | efd061a | |---|---|---|---| | U+009B, U+0080, U+009F | `C` | exit 2 after 1 request | exit 2, 0 requests | | 0xE9 then U+009B | `C`, `C.UTF-8` | exit 2 after 1 request | exit 2, 0 requests | | a `:`, 0xE9, U+009B | `C`, `C.UTF-8` | exit 2, 0 requests, the colon refusal holds the C1 control | exit 2, 0 requests, nothing of the value | | an e with an acute accent, U+00A0 | `C`, `C.UTF-8` | 1 request (the value is accepted) | 1 request (the value is accepted) | **Side effect, as lane 1 said: U+2028 and U+2029 are refused in no locale now.** Only a UTF-8 locale's `[[:cntrl:]]` matched them, and no one chose that. They are not controls a terminal acts on, and a line that holds one prints as it is. `NOT_CONTROLS` places both under `C` and `C.UTF-8`, through both files, so the decision is held and not left to the locale. If a ruling wants them refused, `*$'\342\200'[$'\250\251']*` joins the pattern in both files and the two rows turn into refusals. **What the locale still does, measured:** over every byte string of up to three bytes from a set of fourteen (2955 strings: ASCII, ESC, DEL, 0x80, 0x85, 0x9B, 0x9F, 0xA0, 0xA9, 0xC2, 0xC3, 0xE2, 0xE9), under `C`, `POSIX` and `C.UTF-8`, the new arm gives the same answer in all three locales, and the arm without `LC_ALL=C;` differs from it on exactly one string: U+2029 (0xE2 0x80 0xA9) under `C.UTF-8`. So on glibc the locale's one observable effect is those two characters, and that is what pins `LC_ALL=C` in the tests (see the mutants). It stays in the code as lane 1 specified it, so the match is byte by byte by construction on a libc whose single-byte `iscntrl` says otherwise for 0x80 to 0x9F in a UTF-8 locale. macOS's was not measured. ### Finding 2 (nit, two SonarCloud S131 issues): not taken, partly cleared The colon `case` in `install.sh` has no `*)` arm and keeps its S131, as nine others open in that file do. The control check's `case` now has one, so its S131 should go. That is expected from the code, not measured: SonarCloud's run on this push says. ### Finding 3 (info): #804 depends on #726 #726 has landed on main as dcf62ed. This branch still descends from a3cde99, so `git merge-tree --write-tree origin/main efd061a` (origin/main 40e678e) conflicts in `install.sh`, `openRepoShape` and both test files, as lane 1 found for a cherry-pick. I did not restack: the coordinator retargets and restacks by the stacked-PR procedure. ### Ruling on the colon refusal naming the directory (lane 1 agreed; record it when #760 closes) Lane 1 agreed that the colon refusal names the directory, and asked that it be recorded on #760 at close: #760's Verify line says "neither stream holds the value" for all four values, colon included, and the colon's is named, by a choice every `$HOME` default needs. With this commit the premise is whole: a value that reaches the colon refusal holds no ASCII control character, DEL or C1 control, in any locale and whatever else it holds, so naming it cannot forge a line or reach the terminal. ### Tests at efd061a (every run through `run9.sh`: gh stub first on `PATH`, bogus token, empty `GH_CONFIG_DIR`; dirty=0) Logs are in `$Q/v116/f804/` (`$Q` is the lane's scratchpad). | Run | Result | Log | |---|---|---| | The new and changed cases (`-k "c1_control or unprinted_letter"`) | **36 passed** (18 in each file) | `new-final.log` | | `test_openreposhape_command.py`, `test_install_sh.py`, `test_install_sh_filter_repo.py`, `test_openreposhape_package.py`, `test_repo_hygiene.py`, `test_agents_md_codes.py`, `test_cli_reference.py` | **478 passed**, 103.92s | `runA-final.log` | | the three files `tests-git-bash` runs, with every `Path.write_text` writing CRLF (the plugin that wave 7 used) | **274 passed**, 65.38s | `runC-final.log` | | `test_release_workflow.py`, `test_doctor_validator_rows.py`, `test_doctor_machine_row.py` | **93 passed**, 50.72s | `runB-final.log` | | `test_shape_doctor.py -k "shim or installed_doctor or shipped_doctor or refused_doctor or signal"` (it runs the launcher's `--doctor`) | **20 passed**, 388 deselected | `runD-final.log` | - **What changed in the tests.** - The C1 case, which skipped where no UTF-8 locale read U+009B as a control, is gone with its helper `utf8_locale_with_c1_controls`. In its place a table of five values (`C1_BIN_DIRS`) runs under both `C` and `C.UTF-8`, through the command and through install.sh. The values are U+009B, U+0080, U+009F, 0xE9 then U+009B, and a `:` beside that. Nothing skips for want of a locale. - The placed-name test takes four names (`NOT_CONTROLS`): an e with an acute accent, U+00A0 (the byte after the range's last), U+2028 and U+2029, under both locales, through both files. The installer's version looks for its `preflight:` line among lines split at a newline only, because `str.splitlines` also splits at U+2028 and U+2029. - A lone 0xE9 is `chr(0xDCE9)`, which reaches the child as that byte through `surrogateescape`. No escape is typed into any file, and `body-guard.py` is clean on the four files. - **On the base.** The new and changed cases are 36: **18 fail on 7569ffa's code and 18 pass** (`new-base2.log`). The 18 that pass are guards: U+009B, U+0080 and U+009F under `C.UTF-8` (6, which the head already refused), and the placed names under `C` and the e and U+00A0 under both (12). Of the 18 that fail, 8 surface as a `UnicodeDecodeError` raised inside `subprocess.run` (the head echoes the lone 0xE9, and `run_cmd` and `Net.run` decode strictly) and 10 as assertions. That is a rough failure message and no more: decoding with `errors="surrogateescape"` in those two helpers would turn the 8 into assertions, and I left it, as the round is one commit and no mutant of the helpers can fail on the fixed code. Over a3cde99's code, the larger selection of #760's cases (`-k` of the refusal, installer, default-directory, control, letter and C1 tests, 62 cases) gives **42 failed, 20 passed** (`new-base-a3cde999.log`); over 7569ffa's it gives 18 failed, 44 passed (`new-base-7569ffa8.log`). ### Mutants of the two new hunks, and the earlier commit's rows against the new arm Each runs in a scratch clone against 95 selected cases (the refusal, installer, default-directory, control, letter, C1, Windows-spelling and symlink cases), one edit at a time, restored after. "Failed" is how many of the 95 failed. **All 28 are caught.** Logs: `mut2/` and `mut3/`; the runner is `mutate2.py`; the counts are in `mutants2.out` and `mutants3.out`. | Mutation | Command | install.sh | Where it fails | |---|---|---|---| | no C1 bytes in the pattern | 10 | 10 | every C1 value, under both locales, in the file mutated | | `LC_ALL=C;` dropped | 4 | 2 | U+2028 and U+2029 under `C.UTF-8` (the installer's test runs the command too) | | `[[:cntrl:]]` dropped, C1 bytes kept | 17 | 9 | carriage return, tab, DEL, escape and the drive and colon rows | | range starts one late (`\201`) | 2 | 2 | U+0080, both locales | | range ends one early (`\236`) | 2 | 2 | U+009F, both locales | | range ends one late (`\240`) | 4 | 2 | U+00A0 is refused (the placed-name test) | | lead byte `\303` for `\302` | 10 | 10 | every C1 value | | the whole arm removed | 27 | 19 | every control value | | arm narrowed to a newline | 23 | 17 | every control but a newline | | `[![:print:]]` for the pattern | 16 | 8 | the e with an acute accent under `C`, and more | | refusal echoes the value | 27 | 19 | every control value | | arm after the drive arm (command) | 4 | | the two drive rows (both files) | | arm after the colon arm | 8 | 5 | the colon and the drive rows, the colon beside a C1 control | | arm after the symlink loop (command) / check (install.sh) | 9 | 6 | the link case, the colon and drive rows | | arm after the release lookup (install.sh) | | 19 | a request was made, for every control value | ### Not checked, and known limits - **bash 3.2.57** is not here. Lane 1 built it and measured the same table on it, and the pattern is theirs. I did not run it. - **macOS and other libcs.** `LC_ALL=C` is what is meant to hold there. Not run. - **A GBK or GB18030 locale.** The bytes `0xC2 0x80` to `0xC2 0x9F` are 32 hanzi there (`0xC2 0x9B` decodes to one with Python's `gbk` codec), and the arm now refuses a directory that holds one as "a control character", where `[[:cntrl:]]` placed it. Measured with Python's codecs only: no such locale is installed here. Big5, Shift_JIS (`cp932`), EUC-KR and EUC-JP read those bytes as invalid. A candidate, not a fix: it needs the locale's charset asked first. - **Windows (Git Bash).** The C1 and placed-name tests skip there, as before; the three files `tests-git-bash` runs pass with CRLF `write_text` emulated (274). **1. The command refuses any control character in the directory, first** (`install_commands`, `openRepoShape`). - Before: only a newline was refused first. Measured on a3cde99: `$OPENREPOSHAPE_BIN_DIR` set to `<dir>/bin`, a carriage return, then `FORGED` placed the command, exit 0, and printed `…/bin^MFORGED is not on $PATH`. An escape sequence did the same (`…/bin^[[31mFORGED`). The value reached four lines: `installed at`, `placed in`, `is not on $PATH` and the `export PATH` line. - Now: the arm is `*[[:cntrl:]]*` and the two bytes of each C1 control (U+0080 to U+009F), run under `LC_ALL=C`, still first, before the Windows spelling and the colon. Its refusal says "holds a control character, such as a newline, a carriage return, a tab or an escape" and nothing of the value, so no refusal after it can echo an ASCII control character, DEL or a C1 control, in any locale and whatever else the value holds, that forges a line or reaches the terminal. Exit 2, nothing placed. **2. install.sh carries both arms and refuses before any request** (`main`, `install.sh`). - Before: a `:` or a newline reached the fetch. install.sh fetched the launcher, ran `--install`, and said only "the v9.9.9 openRepoShape's --install refused (exit 2)"; a release older than #667 had no such refusal and placed the command under the directory. Measured with a fake `curl` that logs its calls, version named: a3cde99's install.sh made one request for each of a colon, a newline, a carriage return and an escape; this head makes none. - Now, copied from the command as #667 copied the drive-letter `case` (install.sh arrives alone down a pipe): the control-character arm right after the variable is read, before the Windows respelling; the colon arm after the respelling, as the command orders them. Both are before the symlink check and every line that names the directory. - The control-character refusal never names the directory. The colon refusal names it, as the command's does (see decisions). - **`[[:cntrl:]]`, not `[![:print:]]`**, as the issue directs. `[![:print:]]` is what `checked_repo` uses for `$OPENREPOSHAPE_REPO`, but under `LC_ALL=C` the two bytes of an e with an acute accent are not printable, so it would refuse a directory called `caf` and that letter. A test places that directory under `C` and under `C.UTF-8`, through both files; the mutants "`[![:print:]]`" fail it. - **A C1 control is refused in every locale.** The check runs under `LC_ALL=C` and names U+0080 to U+009F's two bytes (`\302`, then `\200` to `\237`), whatever else the value holds; U+2028 and U+2029 are refused in none. Left to the locale, `[[:cntrl:]]` matches a C1 control only in a value that is valid UTF-8 (bash 5.2.21, Linux): U+009B alone is refused under `C.UTF-8` and passes under `C` and `POSIX`, and one byte that is not UTF-8 before it (`caf`, 0xE9, U+009B) makes bash match byte by byte, so it passes under `C.UTF-8` too (lane 1 measured the same on bash 3.2.57). The tests run five C1 values under `C` and `C.UTF-8`, through both files, and place U+00A0, U+2028 and U+2029 beside the e with an acute accent. - **A tab and DEL are refused too.** They are control characters, the issue says every one, and no `$PATH` line prints either as it is. - **The colon refusal names the directory; the control-character one never does.** The issue's Verify says "neither stream holds the value" for all four values, but the command's colon refusal (#726) names `'$dir'`, and a test of #726's pins that for the `$HOME` default, where the person typed nothing and needs the directory named. A colon value holds no ASCII control character, DEL or C1 control by then, in any locale, so naming it cannot forge a line or reach the terminal. Lane 1 agreed with this ruling (2026-10-17T11:06:51Z) and asked that it be recorded when #760 closes: the issue's Verify line says "neither stream holds the value" for all four values, and the colon's is named. install.sh's colon refusal does the same. The tests hold the control values to "no `forged` tail, no escape, DEL or C1 in either stream", and the colon values to "named, in quotes". If the coordinator rules that a colon must go unnamed too, it is one message in each file and the same assertion. - **The order is held, not just the width.** A drive spelling with a newline or an escape must be refused as a control character (not as "spelled for Windows" when no `cygpath` answers), a colon with an escape must be refused as a control character, and a link in the directory must not make the symlink refusal print the value. **The figures in this section are 7569ffa's.** At efd061a they are the fix round's table above: 36 new and changed cases, 478 passed in the seven files, 274 passed with CRLF `write_text`, and 93 and 20 passed in the others. The base figures are 18 failed and 18 passed over 7569ffa's code, and 42 failed and 20 passed over a3cde99's for the 62-case selection. 31 cases in the new and extended tests (26 are new; the command's five older refusal cases run again in the same test), run with the one existing test the selection matches: 32 passed in 1.75s on the head, dirty=0. - `tests/test_openreposhape_command.py` (15): the refusal over 11 values (a colon, two directories, a drive-relative name, a newline, a carriage return, an escape, a tab, DEL, a drive spelling with a newline, a drive spelling with an escape, a colon with an escape; the first five were 5 cases, six are new), a letter under `C` and `C.UTF-8`, U+009B under a UTF-8 locale, and a link in a directory holding a carriage return. A shared `assert_a_bin_dir_is_refused` holds exit 2, the wording, the ending, and that no `forged` tail, escape, DEL or C1 reaches either stream. - `tests/test_install_sh.py` (16): the same 11 values through install.sh and the command, in one test (exit 2 for both, the fake `curl` records no call, nothing placed or made in either world, `$TMPDIR` empty), the `$HOME` default with a colon, the symlink case, a letter under both locales (placed, named as typed), U+009B under a UTF-8 locale. - **On the base a3cde99** (the head's two test files over the base's code): `24 failed, 8 passed` in the 32. The 8 are the command's three colon cases and the `$HOME` default (the base already refuses them), and the letter under both locales through both files (the base places it; they guard the width). Of the 24, 22 fail on behaviour (a request was made, or the value was placed and echoed, or the refusal came from the wrong arm); the command's `a-newline` and `a-drive-spelling-and-a-newline` fail only on the refusal's new wording. - **On the head 7569ffa, dirty=0:** - `tests/test_openreposhape_command.py tests/test_install_sh.py tests/test_install_sh_filter_repo.py tests/test_openreposhape_package.py`: **302 passed** in 68.90s. - `tests/test_repo_hygiene.py tests/test_agents_md_codes.py tests/test_cli_reference.py tests/test_setup_by_hand_commands.py tests/test_readme_windows_quoting_sentence.py tests/test_gh_guard.py tests/test_workflow_concurrency.py tests/test_release_workflow.py tests/test_readme_sections.py`: **297 passed** in 38.97s. - `tests/test_shape_doctor.py -k "shim or installed_doctor or shipped_doctor or refused_doctor"` (it runs the launcher's `--doctor`): **20 passed**, 388 deselected. - **The three files `tests-git-bash` runs, with Windows' `write_text` emulated** (a plugin makes every `Path.write_text` write CRLF): **244 passed** in 52.64s. A probe test takes the plugin's effect: 1 passed with it, 1 failed without. - **install.sh, version named, each of a colon, a newline, a carriage return, an escape:** a3cde99: exit 2 after 1 request (the launcher's fetch), refused by the fetch's error or by the command. This head: exit 2, 0 requests, install.sh's own refusal (the colon one names the directory; the others name nothing of it). - **The command, `--install`:** a3cde99: a carriage return and an escape: exit 0, the command placed under the directory, the value in four lines. A newline: exit 2. This head: exit 2 for all three, nothing placed. - **Needs Windows or another system to be real:** none of these rows. See below for what was not run. The fix round above adds 14 mutants of the new pattern line and 4 order mutants, and re-ran this table's rows against the new arm (the whole arm removed, narrowed to a newline, `[![:print:]]`, the refusal echoing, and the two later placements). Each is still caught. Each row changes one arm or hunk in a throwaway clone and runs the new cases and #726's drive and symlink cases (65 selected). "Failed" is how many of the 65 failed. | Mutation | Failed | Where | |---|---|---| | command: no control-character arm | 18 | the command's cases and install.sh's, which runs it | | command: arm narrowed back to a newline | 14 | carriage return, escape, tab, DEL (both files) | | command: `[![:print:]]` for `[[:cntrl:]]` | 2 | the letter under `C` (both files) | | command: arm after the drive arm | 4 | the two drive spelling cases (both files) | | command: arm after the colon arm | 6 | colon with an escape, the two drive cases | | command: arm after the symlink loop | 7 | the link case, colon with an escape, the two drive cases | | command: refusal echoes the value | 18 | every control value | | install.sh: no control-character arm | 10 | the link case, U+009B, the control values | | install.sh: arm narrowed to a newline | 8 | carriage return, escape, tab, DEL, the link case, U+009B | | install.sh: `[![:print:]]` | 1 | the letter under `C` | | install.sh: arm after the drive block | 2 | the two drive spelling cases | | install.sh: arm after the symlink check | 4 | the link case, colon with an escape, the two drive cases | | install.sh: arm after the release lookup | 10 | a request was made, for every control value; and the link refusal named it | | install.sh: refusal echoes the value | 10 | every control value | | install.sh: no colon arm | 4 | the three colon values, the `$HOME` default | | install.sh: colon arm before the drive block | 17 | #726's drive cases (the respelling never runs) | | install.sh: colon arm after the release lookup | 4 | a request was made, for the colon values | | install.sh: colon refusal names nothing | 4 | the three colon values, the `$HOME` default | All 18 are killed, on the final head. A first run on the pre-reorder tests gave the same counts. - origin/main bf1f099: clean. The parent a3cde99: clean (this head descends from it). - All 47 `refs/heads/fix/*` and `l3-*` branches on origin (none of `l2-*` or `l4-*` exists; mine excluded): clean. Only #726's branch edits `install.sh`, `openRepoShape`, `tests/test_install_sh.py` or `tests/test_openreposhape_command.py`, as the issue said. - Lane 5's other wave-7 branches (`fix/757` through `fix/777`, local heads at the time): clean; none edits these four files. - **bash 3.2** is not here; lane 1 measured the table and the fix on 3.2.57 built from GNU source. - **Windows.** A control character, or a non-ASCII value, crossing the environment into Git Bash is not run here: the letter and U+009B tests skip on Windows, and the link and `$HOME` tests do too (no file there holds a control character or a `:`). The 11-value test runs there as #726's newline case did, and its result is `tests-git-bash`'s to give. CRLF `write_text` was emulated (244 passed). - **macOS** and a locale other than `C`, `POSIX` and `C.UTF-8`: not run. No test skips for want of a locale. - **A GBK or GB18030 locale.** The bytes `0xC2 0x80` to `0xC2 0x9F` are 32 hanzi there, and the arm now refuses a directory that holds one as "a control character". Measured with Python's codecs only: no such locale is installed here. - **Release notes.** `--install` and install.sh now refuse a control character in `$OPENREPOSHAPE_BIN_DIR` where `--install` placed the command and printed the value as typed, and install.sh refuses a `:` before its fetch. Lane 3's #630 drafts `docs/release-notes/v1.1.0.md`; a line for it: "`install.sh` and `openRepoShape --install` refuse a `$OPENREPOSHAPE_BIN_DIR` holding a control character (a newline, a carriage return, a tab, an escape or a C1 control), and `install.sh` a `:`, with exit 2 before it makes a request." Not edited here. - README, the handbook, `AGENTS.md`, `docs/cli.md`: no sentence became false. 1. **U+202E, U+200B, U+2028 and U+2029 in `$OPENREPOSHAPE_BIN_DIR` are echoed as typed** (the body's candidate 1, now also the two separators). None is a control a terminal acts on; U+202E can make a directory read as another. 2. **A lone 0x9B byte** (an 8-bit C1 control, not UTF-8) matches in no locale, before and after this commit. Only an 8-bit terminal with C1 controls on acts on it. (Lane 1's candidate 2.) 3. **A GBK or GB18030 locale reads `0xC2 0x80` to `0xC2 0x9F` as hanzi**, so the arm refuses a directory holding one. Measured with codecs only. 4. **Lane 1's other candidates stand as they listed them:** a leading `-` makes `--install` run `mkdir -p "$dir"` with no `--`; U+202E and U+200B; a `$HOME` holding a control character is refused without saying the directory came from `$HOME`; install.sh's Windows-spelling refusal names nothing where the command's names `'$dir'`; macOS's `[[:cntrl:]]` for C1 under `en_US.UTF-8` is unmeasured. Review: lane 1's re-verification of #804 is PR-READY (#804 issuecomment-6238973516). CI (Brett Heap's ruling 27): the head 7e4181c passed tests, tests-git-bash and SonarCloud; main has touched none of its files since 6626a3c; and the local gate passed on its predicted squash tree cbc85da over main a9d1579: the install_sh, openreposhape_command, repo_hygiene, agents_md_codes and cli_reference sets, 404 passed. Landed on Brett Heap's word of 2026-10-17T21:47:28Z, in lane 5's session: "merge lane 5's v1.1.1 PRs once reviewed and green". Closes #760 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd what they tell git to the `GIT_CONFIG_COUNT` entries the environment holds, so a pasted line keeps the session's maintenance settings and the Windows job's two (#758) (#818) Against main. #748 landed as `431c8cf8`; this branch merged #748's final head and main (see Fix round 1). After that merge, #723 (`bad05d21`) and #746 (`fc867207`) landed. The branch is refreshed over main `fc867207` as `4a553e17`, one merge commit. Its only conflict was `tests/test_non_utf8_git_reads.py`'s `from conftest import` block, resolved by keeping both sides' names, as rv818 asked. The merged tree differs from main in #818's own 27 files, and #818's count scan flags no file. Tests only: 25 files under `tests/`, no shipped code, no pinned file, no README, handbook, `AGENTS.md` or `docs/cli.md` change. Two new commits on `b6e15591`: the retarget merge `2908e46f`, then the fix commit `eaa72d9e`. Tests only, as before. ### The retarget: merge commit `2908e46f` #748 landed as `431c8cf8` from its final head `8368d1f8`, and GitHub moved this PR's base to main. The merge commit has three parents: `b6e15591`, `8368d1f8` and main `fa04d2cd`. #748's head goes in first, so git sees the history the squash dropped. - **#748's head did not merge clean**, as rv818 found. `tests/test_adopt_empty_symlinks.py` conflicts, because `8368d1f8` holds main `741679e7`, which carries #388. - I took w758's earlier resolution of that file, which rv818 checked. Main's side of the file is the same at `974d4827`, `8368d1f8` and `fa04d2cd`, so it still fits. - Both sides stay: #388's two targets (`with_each_target`, `SAID`, the `target` parameters) and #758's `symlinks_off()`, built at the call. - `test_the_link_given_a_target_adopts_end_to_end` calls `symlinks_off()` too. Git does not flag it, and left as it was the file fails at collection. - **Main then conflicted in three files** (main was `d17564d9` when I resolved them): `tests/test_clone_just_pushed.py`, `tests/test_doctor_legs_members_next.py` and `tests/test_family_bump_remove_rollback.py`. - Main's copy of each was `8368d1f8`'s byte for byte. - This branch's copy is `8368d1f8`'s plus #758's own hunks, so this branch's side is taken. - Main's next two landings, #809 and #816, merged with no conflict. - **rv818 MUST 1.** `tests/test_local_remote_dir_urls.py` came with #812 and imported `ALLOW_FILE_PROTOCOL` from `test_family`. - Without a change, the file fails at collection: `ImportError: cannot import name 'ALLOW_FILE_PROTOCOL' from 'test_family'`, and none of its 27 cases run. - It now imports `allow_file_protocol` and calls `pasted(lines, paste, allow_file_protocol())`. - `git grep` finds no other file on main that uses a name #758 removes (`SYMLINKS_OFF`, `SYMLINKS_ON`, `CONFIG_COUNT`, `CONFIG_VALUE`, `PASTE_ENV`, `ALLOW_FILE_PROTOCOL`). `NO_MAINTENANCE`, `QUOTE_PATH_OFF` and `tool_env` change shape, and each is used only in its own file. - **Checked:** - Against main, the merged tree differs in #758's 25 files and in `tests/test_local_remote_dir_urls.py`. - Each of the 25 differs by exactly #758's own added and removed lines, `tests/test_adopt_empty_symlinks.py` aside. - Collecting the whole suite at the head finds 12,320 tests and no error. ### The fix commit `eaa72d9e` - **SHOULD 2:** `configure` in `tests/test_adopt_reencoded_files.py`, the seventh append from #758's line 2, now sets what `appended_to_command_scope(os.environ, pairs)` returns. It gives the same entries in the same order, so no test can fail on the old form. - **SHOULD 3:** the scan also reads a name the module binds to `"GIT_CONFIG_COUNT"` (`CONFIG_COUNT = "GIT_CONFIG_COUNT"`, annotated or not) as the count. It does so in a dict key, a setter's first argument and a subscript. - rv818's repro: append the base's two lines to `tests/test_adopt_root_symlinks.py`. - The old scan passes. The new one names line 2137. - **SHOULD 4:** `counts_set_to_a_number` is split into one reader per node type: `in_a_dict`, `in_a_call` and `by_subscript`. The walk takes each node's reader from `READERS`. By hand, no function scores more than 5 on S3776. I could not run Sonar. - **NIT 5:** `conftest.py` is no longer skipped. It scans clean, at the head and on today's main. - **NIT 6:** - New "sees" cases: `os.putenv("GIT_CONFIG_COUNT", "1")` and a bare `setenv("GIT_CONFIG_COUNT", "1")`. - The walk is now `files_off_their_count(directory)`. A new test runs it over a folder of its own, and checks three things: - `conftest.py` and a helper module are read; - a `.txt` file is not; - a file `OWN_COUNTS` allows is named for fewer counts as well as for more. - Four of rv818's six surviving mutants now fail a case: `putenv` dropped, the bare-name setter dropped, `!=` changed to `>`, and `glob("test_*.py")`. - The fifth, `conftest.py` scanned, is now how the code works, and putting the skip back fails the new test. - The sixth, the count's value taking any string, still survives: it is a stricter scan. Git refuses a count that is not a number (`bogus count in GIT_CONFIG_COUNT`, exit 128), so a test that holds one fails on its own. - **NIT 7, in part:** - An empty count (`""`) is now a count of its own. Git 2.43 and 2.47.1 read it as no entries at all. Measured: `GIT_CONFIG_COUNT= GIT_CONFIG_KEY_0=a.b GIT_CONFIG_VALUE_0=c git config --get a.b` exits 1. - The docstring says the scan reads the count's value as a constant only. - Left out: - a value built by a call (`str(len(pairs))`) or held in a variable; - an int; - `dict()`, or `environ={}` by keyword, handed to the helper; - `setenv` with keyword arguments; - an annotated subscript. rv818 judged only `str(len(entries))` likely, and telling it from a count worked out needs what the call means, not how it is spelled. - **NIT 8:** the paste case deletes `GIT_CONFIG_PARAMETERS` first. Measured on the merge before the fix commit, with `GIT_CONFIG_PARAMETERS="'probe.param'='p'"`, the case failed (`Left contains one more item: 'probe.param=p'`). It passes now. - **NIT 9, left as it is:** main's three appends built by hand each work the count out right: `tests/test_family_head_lost_gitmodules.py:97`, `tests/test_family_bootstrap_gate_order.py:135` and `tests/test_family_add_spellings_remedies.py:137`. Each of those files is also changed by another open PR (#828 `fix/761-add-move-back-gitmodules`, #820 `fix/762-validate-family-remedies-texts`, `fix/763-holder-docs-agree`), so converting them here would only add conflicts. - **NIT 10:** the comment on `NO_MAINTENANCE` now says its `maintenance.auto` and `gc.auto` repeat the session's two, which reach these gits in the command scope. Only `gc.autoDetach` adds a setting. The two stay, so the constant alone still keeps maintenance off for these gits. - **NIT 11:** the count was in lane 5's #473 summary, not in this body. The seven sets held 2,401 distinct passes, not 2,609: the 208-test run repeats three files of the 256-test run. ### Tests at `eaa72d9e` All runs went through `run9.sh`, with the real `gh` unreachable. All passed, none skipped: - **The 26 test files the PR changes against main**, in one full-file run each: 2,206 passed (1,108 + 1,098). - These are #758's 24 test files, `tests/test_local_remote_dir_urls.py` and `tests/test_adopt_reencoded_files.py`. - The count matches the collection exactly. - **The 43 files that import any of those 26, with `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py` and `tests/test_cli_reference.py`:** 1,646 passed (825 + 821). - **The same merge made on main `d17564d9`** (a local commit, `c16a7e1a`, never pushed), before the fix commit: `tests/test_local_remote_dir_urls.py` and `tests/test_no_auto_maintenance.py`, 55 passed. ### Fails on the base The new tests were run against the base's scan and walk, as `b6e15591` has them, with `GIT_CONFIG_PARAMETERS` set: 6 failed, 30 passed. The six: - the paste case; - the `nothing`, `a-name-bound-to-it`, `an-annotated-name` and `a-name-by-subscript` cases; - the walk test. The three other new cases (`a-bare-setenv`, `putenv`, `a-name-bound-to-another-key`) are there to kill mutants, and they pass on any correct scan. MUST 1 fails on main's form at collection, as above. SHOULD 2 and NIT 10 change no behaviour. ### Mutants I made 26 mutants. 25 are caught, and one survives as rv818 said it would: - **22 on the scan and the paste case**, each run against `tests/test_no_auto_maintenance.py`: - the delenv dropped; - `putenv` dropped from `SETTERS`, or the bare-name setter; - the name branch of `is_the_count` dropped, or any name taken as the count; - `""` no longer a count; - the `Assign` names dropped, or the `AnnAssign` names; - no names collected; - each reader ignoring the names; - each of the three readers dropped from `READERS`; - `conftest.py` skipped; - `!=` changed to `>` or to `<`; - `glob("test_*.py")`, or `glob("*")`; - `by_subscript` taking any value; - the walk pointed at an empty folder. The last passes on a clean tree, like any change to an assertion that nothing is found. It is caught by rv818's SHOULD 3 repro, where the real walk fails and the mutant passes. - **The survivor:** the count's value taking any string. It makes the scan stricter (see NIT 6). - **`configure`:** with `appended_to_command_scope({}, pairs)`, the scan names the file. With the pairs dropped, three of its tests fail. - **MUST 1:** main's import put back fails collection with `ImportError`. ### Merge checks at `eaa72d9e` I ran `git merge-tree --write-tree` against `origin/main` (`fa04d2cd`) and the 53 other `fix/*` and `l3-*` heads on origin: - **Clean:** `origin/main` and 43 heads. - **#792 (`l3-785-non-ascii-paths`)** merges clean, but `tests/test_doctor_non_ascii_paths.py` imports `PASTE_ENV`. Whichever PR lands second changes `:50` (the import), the `:266` docstring and `:268` to `paste_env()`. - **#723 (`fix/663-non-utf8-name-tests`), #746 (`fix/672-bump-leg-branch-and-push-line`, which now holds #723) and #806 (`fix/757-non-utf8-test-dirs`)** conflict only in the `from conftest import` block of `tests/test_non_utf8_git_reads.py` (and of `tests/test_adopt_git_reads.py`, for #806). Both sides add names, so keep them all. - **Seven heads conflict with main itself, not with this PR:** - `l3-732-latin1-section` and `l3-733-ignored-sections` (`tests/test_doctor_legs_members_next.py`); - `fix/712-validate-pins-contracts`, `fix/766-spelled-paths` and `fix/798-adopt-legs-asked-first`; - `l3-677-pins-hand-step-wip` and `l3-678-naming-checkout-wip`. - I scanned each clean merged tree's changed test files for a name #758 removes, or a count the scan flags. Only #792 shows up. - **Main moved after the push.** #723 landed as `bad05d21`, and the two now conflict in one place: the `from conftest import` block of `tests/test_non_utf8_git_reads.py`. The fix is to keep both sides' names: `ORG, PROJECT, REPO, appended_to_command_scope, cloned_just_pushed, git, run_script, skip_message, skip_where_fsencode_gives_a_surrogate_no_byte, windows_fsencode`. Every other change merges in. On that merged tree, with the block resolved that way, the scan flags no file, and no file uses a name #758 removes. On `deb029c9` a probe test imported `PASTE_ENV` and pasted `git config --show-origin --list` through `paste()`: - `PASTE_ENV` held `GIT_CONFIG_COUNT=1` (`protocol.file.allow`), a stale `GIT_CONFIG_KEY_1=user.email` left from collection, and no `GIT_CONFIG_GLOBAL`. - `os.environ` at the call held a count of 4 and the session's global file. - The pasted git listed one command-line entry, `protocol.file.allow=always`, and `git config --get maintenance.auto` exited 1. So a pasted line ran with git's automatic maintenance on, in both scopes, as the issue said. `paste_env()` replaces it. It reads `os.environ` at each call (minus `PYTHONPATH`), adds the identity and `PYTHONDONTWRITEBYTECODE`, and appends `protocol.file.allow` with `appended_to_command_scope`. The six uses call it. The import-time copy also predated the session's `gh` guard on `PATH`, as `test_doctor_next_is_a_command.py`'s docstring says of its own environment; a pasted line now gets the `PATH` of the call. That docstring now says `paste_env()` reads at the call too. - **The seven.** Six are on this base, each now one call to `appended_to_command_scope`: `test_adopt_git_reads.py`, `test_adopt_nested_attributes.py`, `test_adopt_ignored_shape_files.py`, `test_adopt_root_symlinks.py`, `test_bootstrap_after_bump.py`, `test_family_bump_remove_rollback.py`. The seventh, `test_adopt_reencoded_files.py`, came with #743 and is converted in fix round 1. `test_clone_just_pushed.py:96` stays, as the issue says: it is the helper's own test. - **The other literals.** Each sets a count of its own and is laid over `os.environ`, where the session's two settings and the Windows job's two sit, so each replaced them (`NO_MAINTENANCE` put the two maintenance keys back and dropped the job's two). I read each; each is converted, none left: - `test_family.py`: `ALLOW_FILE_PROTOCOL` is now `allow_file_protocol()`; its two importers (`test_siblings_family_name.py`, `test_family_siblings_placement.py`) and the four uses in `test_family.py` call it. - `test_family_dry_run.py`: `NO_MAINTENANCE` is a tuple of entries, appended by `isolated()`. - `test_non_utf8_git_reads.py`: `QUOTE_PATH_OFF` is a tuple of entries, which `tool_env()` appends; `user_name()` appends too. - `test_adopt_empty_symlinks.py`: `SYMLINKS_OFF` is `symlinks_off()`. - `test_adopt_root_symlinks.py`: `SYMLINKS_OFF` and `SYMLINKS_ON` are gone; the four tests that set the first call `links_as_files` and the two that set the second call `told_git`, which already appended. - one-off environments: `test_scaffold_reuse_seeded_legs.py`, `test_adopt_split_branch.py`, `test_scaffold_free_text_values.py`, `test_adopt_plan_yaml_quoting.py`, `test_setup_project_py.py`, `test_setup_sh.py`, `test_doctor_next_is_a_command.py`. - The two tests that build a held environment on purpose to see it kept (`test_adopt_git_reads.py`, `test_clone_just_pushed.py`) now build it with `appended_to_command_scope({}, ...)`. Measured with a probe on a throwaway bare remote after a push (18 loose objects): the hand-built options and `clone_just_pushed`'s give the same clone, one pack and no loose object, with `upload-pack` and `pack-objects` in `GIT_TRACE`, and `origin` keeps the plain path. A plain local clone of the same remote keeps all 18 loose objects. For a clone that is not recursive the helper's `insteadOf` rewrite is inert, because no submodule is cloned; the helper adds one place for the options, not a different read. - `test_bump_leg_dry_run_truth.py:98`, the private bare copy of the session's spec remote: a session remote. `cloned_just_pushed(..., bare=True)`; the fixture now asserts the copy is bare. - `test_bump_leg_dry_run_truth.py:110`, `private_work`, a clone of that private remote, which tests push tags into: `cloned_just_pushed(..., recurse=False)`. - `test_doctor_legs_members_next.py:1828` and `:1929`, `Ghost`, a clone of the first member's checkout in the test's own holder copy: not a remote, and the hand-built `-c protocol.file.allow=always` was dead weight on a direct clone. Both call `clone_member`, one helper over `cloned_just_pushed(..., recurse=False)` that asserts the legs stay empty. - `test_family_bump_to_checks.py:235`, a clone of `a_private_remote`, a bare repository filled by a push: `cloned_just_pushed(remote, work, "main", recurse=False, env=env)`. - `test_doctor_pin_restore_merges.py:714` is **left as it is**: a `--filter=blob:none` clone of a file URI. The helper takes a path, and has no way to pass `--filter`. - `test_family_dry_run.py`: the comment on `NO_MAINTENANCE` says what the entries are for, the gits `isolated()` hands an environment to, whose global file `excluding()` replaces, and that the session fixture covers the module's own setup commits, which pass no `env=`. - `tests/conftest.py`: the two i645 replacements, "repacks once its estimate, taken from `objects/17` alone, is above 256 loose objects, so two loose objects there are enough", and the `objects changed` sentence. **#713's body holds the matching sentence and needs the same two replacements; I cannot edit a body.** `tests/test_no_auto_maintenance.py` has 17 new cases (28 in the file, 11 before): - `test_a_pasted_line_keeps_the_entries_the_environment_holds_at_the_call`: the environment holds two entries as the Windows job does, then the two settings as the session fixture appends them, set after the doctor tests were imported. `git config --show-origin --list` through `paste()` lists the four, in order, then `protocol.file.allow`, and reads both settings from the session's global file. - `test_no_test_sets_a_git_config_count_of_its_own`: reads every `tests/*.py` but `conftest.py` with `ast` and fails a `GIT_CONFIG_COUNT` set to a number (dict, keyword, `setenv`/`setdefault`/`putenv`, subscript) and an `appended_to_command_scope` call handed a dict written out. Three files are allowed the ones that build a held environment on purpose, with a count each in `OWN_COUNTS`. - 15 cases that hold the scan itself: seven spellings it must see, eight it must leave (`env.get("GIT_CONFIG_COUNT", "0")` is a read). Runs at the head `b6e15591` through `run9.sh` (the real `gh` unreachable), all passed, none skipped: - `test_no_auto_maintenance.py`, `test_doctor_legs_members_next.py`, `test_doctor_next_is_a_command.py`: 208 passed. - `test_clone_just_pushed.py` and `test_doctor_validator_rows.py` (it imports the doctor file), with those three: 256 passed, run one commit earlier (the later commit only moved two blank lines and reworded one failure message). - The sweep files: `test_adopt_git_reads.py`, `test_adopt_nested_attributes.py`, `test_adopt_root_symlinks.py`, `test_adopt_ignored_shape_files.py`, `test_bootstrap_after_bump.py`, `test_family_bump_remove_rollback.py`, `test_adopt_empty_symlinks.py`, `test_adopt_split_branch.py`: 445 passed. - `test_non_utf8_git_reads.py`, `test_family.py`, `test_siblings_family_name.py`, `test_family_siblings_placement.py`, `test_scaffold_reuse_seeded_legs.py`, `test_scaffold_free_text_values.py`, `test_adopt_plan_yaml_quoting.py`: 999 passed. - `test_family_dry_run.py`, the five files that import its `isolated` (`test_family_bump_to_a_tag.py`, `test_family_bump_to_checks.py`, `test_family_bump_to_local_only.py`, `test_family_refuse_up_front.py`, `test_family_remove_gitmodules.py`) and `test_bump_leg_dry_run_truth.py`: 216 passed. - `test_setup_project_py.py`, `test_setup_sh.py` and their importers (`test_adopt_shape_pin_and_work_dir.py`, `test_dry_run_asks_remotes.py`, `test_clone_url_bytes.py`, `test_setup_by_hand_commands.py`): 313 passed. - `test_repo_hygiene.py`, `test_agents_md_codes.py`, `test_cli_reference.py`, `test_doctor_pin_restore_merges.py`, `test_git_helper_identity.py`: 172 passed. `render-cli-reference.py --check` is ok. **On the base (`deb029c9`)**, with the new `test_no_auto_maintenance.py` copied in: 2 of the 17 fail, the paste case and the scan (it finds 12 files). The other 15 hold the scan itself and pass on any tree. Each of the new hunks was broken in a throwaway and run against its tests: - `paste_env`: a literal count of 1, a snapshot taken at import, and `appended_to_command_scope({}, ...)`: each fails the paste case, and the first and third the scan too. - The 18 sites that call `appended_to_command_scope(os.environ, ...)` (or `env`), each with `{}` in place of the environment: each fails the scan. Sites: the six of line 2, `allow_file_protocol`, `isolated`, `tool_env`, `user_name` (non-UTF-8), `symlinks_off`, `git_reads_github_from`, `github_at`, `user_name` (free text), `test_adopt_plan_yaml_quoting.py`, the two setup tests, `test_doctor_next_is_a_command.py`. - The scan: eight changes (ignore dicts, keywords, `setenv` calls, subscripts, `appended_to_command_scope({}`; flag `os.environ`; flag a read; allow fewer in `OWN_COUNTS`) each fail a scan case. - The clone sites: dropping `bare=True` from the private remote, `recurse=False` from `clone_member`, and `"main"` from the checks clone each fail. Two **survive as equivalents**: `recurse=False` in `private_work` (the spec leg has no `.gitmodules`, so a recursive clone of it is the same clone) and `env=env` in the checks clone (a clone reads no config the isolated environment changes). - The two comments cannot be mutated. `git merge-tree --write-tree` of this head against `origin/main` (`974d4827`), #748 (`deb029c9`), #713 (`979bb394`) and every `fix/*` and `l3-*` head on origin (46 refs): - **Clean:** #748, #713, and 24 of the 46 heads: 21 `fix/*` heads (every lane-5 PR but #723 and #756, and most of lane 4's) and `l3-677`, `l3-678`, `l3-rehearsal-v1.1.0`. - **`origin/main`: one file, `tests/test_adopt_empty_symlinks.py`** (four hunks), against #388's landing (`5c990708`). The parent #748 merges clean, so this arrives with the retarget. Resolve by keeping main's lines and my `symlinks_off()`. Two more spots need the same change and git does not flag them: - `tests/test_adopt_empty_symlinks.py::test_the_link_given_a_target_adopts_end_to_end` uses `SYMLINKS_OFF`, which this change removes: give it the `off` parametrization. - `tests/test_adopt_reencoded_files.py::configure`, the seventh hand-built append, goes to `appended_to_command_scope(os.environ, pairs)`. - `tests/test_family_head_lost_gitmodules.py::on_disk` (on main) is one more hand-built append: `{**env, **appended_to_command_scope(env, [("core.ignorecase", ignorecase)])}`. The scan does not read for hand-built appends, so it is not required, but it is the same thing. I made that merge in a scratch tree (main at `bf1f099b`), resolved it as above, and ran the files this change touches that main also changed, with `test_adopt_empty_symlinks.py`, `test_adopt_reencoded_files.py`, `test_family_head_lost_gitmodules.py`, `test_no_auto_maintenance.py` and the files that import `isolated` or the doctor helpers: 653 and 245 passed, in two runs. The scan finds no other file of main's with a count set by hand. - **#723 (`fix/663-non-utf8-name-tests`):** `tests/test_non_utf8_git_reads.py`, the `from conftest import` block, where both add names. - **`l3-732`, `l3-733`:** `tests/test_doctor_legs_members_next.py` (the `from conftest import` block) and `tests/test_adopt_empty_symlinks.py`. - **19 more** (`fix/664`, `fix/729` (#756) and 17 `l3-*` heads): `tests/test_adopt_empty_symlinks.py` only, because they carry main's #388 as main does. - **Not a textual conflict, but it breaks on merge:** `l3-785-non-ascii-paths` imports `PASTE_ENV` (`tests/test_doctor_non_ascii_paths.py:47` and `:223`); it must call `paste_env()`. #812 (`fix/693-local-remote-dir`) landed first, with `tests/test_local_remote_dir_urls.py` importing `ALLOW_FILE_PROTOCOL` (`:51`, `:349`). This PR's retarget merge makes it call `allow_file_protocol()` (rv818 MUST 1). - A pasted `git submodule update --init` still clones a local path: `paste()` takes no remotes, so no `insteadOf` rewrite reaches it (rv662's "build it from `read_through_git`"). It runs under the session's maintenance settings now, which is what the issue asks. - Three more appends built by hand beside `appended_to_command_scope`, in files this base does not hold: `tests/test_family_head_lost_gitmodules.py:97` (main), `tests/test_family_bootstrap_gate_order.py:135` (#682), `tests/test_family_add_spellings_remedies.py:137` (#767). The scan reads for counts set to a number, not for appends worked out by hand. Review: lane 2's independent review (issuecomment-6236874291) and its re-verification of eaa72d9, READY (issuecomment-6238781272); its new SHOULD is taken in #848, and 4a553e1 merges main over eaa72d9. CI (Brett Heap's ruling 27): H0 is the head 4a553e1 itself, which passed tests, tests-git-bash and SonarCloud; main has touched none of its files since its base fc86720; and the local gate passed on its predicted squash tree 9d6c099 over main d8b8eb3: the adopt_empty_symlinks, adopt_git_reads, adopt_ignored_shape_files, adopt_nested_attributes, adopt_plan_yaml_quoting, adopt_reencoded_files, adopt_root_symlinks, adopt_split_branch, bootstrap_after_bump, bump_leg_dry_run_truth, clone_just_pushed, doctor_legs_members_next, doctor_next_is_a_command, family, family_bump_remove_rollback, family_bump_to_checks, family_dry_run, family_siblings_placement, local_remote_dir_urls, no_auto_maintenance, non_utf8_git_reads, scaffold_free_text_values, scaffold_reuse_seeded_legs, setup_project_py, setup_sh, siblings_family_name, repo_hygiene, agents_md_codes, cli_reference sets, 2359 passed. Landed on Brett Heap's word of 2026-10-17T21:47:28Z, in lane 5's session (posted on #473 as issuecomment-6241758069): "merge lane 5's v1.1.1 PRs once reviewed and green". Closes #758 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lper whether a name that is not UTF-8 can be made and skip on its two answers only, and three tests skip where an ASCII file name encoding cannot carry their text (#757) (#806) Against main. #723 landed as `bad05d2`, and this branch is restacked onto main as `76ae6419`: it merges #723's final head `4cdb427c`, then main, taking this branch's side in `tests/test_non_utf8_git_reads.py`, the one file #723 also changed. Over main it carries exactly its own change: `git diff 6ba8c5d 6b5e907` and `git diff origin/main 76ae641` add and remove the same lines (7 files, +408/-45). Tests only. No tool, template, pinned file, README, handbook, `AGENTS.md` or `docs/cli.md` changes (`render-cli-reference.py --check`: `[ok] docs/cli.md is what the generator produces`). Seven test files change, one of them new. **No Windows run and no macOS run was made.** CI is Linux only since #597. What Windows and macOS answer is emulated (see the tests below), and the emulation holds the skip decisions, not a real disk. **Line 1: the callers ask the helper, and every other disk error is the test's own.** Four tests and `directory_named` each made the directory themselves and turned any `OSError` into `pytest.skip`, beside `skip_where_a_name_cannot_be_not_utf_8` from #723. That helper skips on the two answers that mean "this system cannot carry the name" (Windows' `os.fsencode` first, macOS's EILSEQ second) and raises the rest. So a full disk or a `tmp_path` this user cannot write in read as SKIPPED in those five and as a failure in the helper's own tests, and no run fails on a skip. - `tests/test_stdout_encoding.py`, `tests/test_adopt_git_reads.py`, `tests/test_adopt_template_values.py` and the directory test of `tests/test_adopt_plan_escapes.py` now call the helper first. The directory they make after the probe (`b"b\xffd"`, `caf` and byte 0xE9) is no longer guarded. - Each test keeps its own second skip: `git will not work under <name>` in the three adopt tests, and `this filesystem rewrote a name that is not UTF-8` in the stdout test. - The file twin, `test_a_leg_extracts_a_path_that_is_not_utf8` in `tests/test_adopt_plan_escapes.py`, asked the same question of a FILE (`caf` and byte 0xE9 plus `.txt`, `write_bytes`, `unlink`). It calls the helper too and no longer writes a probe file of its own. I chose the helper over `make_file`'s skip because `make_file` also skips on any `OSError` of the write (see the candidates), and because the helper's probe is a name that is not UTF-8 under the same directory, which macOS refuses for a file and a directory alike. The file itself is made later, by git, in `filter-repo`'s clone. - `directory_named` in `tests/test_clone_url_bytes.py` is split. `skip_where_this_system_cannot_name(tmp_path, raw)` asks the helper for a name that is not UTF-8 (`raw.decode("utf-8")` fails) and keeps `directory_named`'s own Windows skip for a control character (a byte below 0x20), which Windows refuses in any file name. `directory_named` then makes the directory with an unguarded `os.mkdir`. **The Windows half of #459's line (R433 O5, issuecomment 6214823552), decided with line 1.** The old `directory_named` skipped every row on Windows, the `bidi-override` row (the name U+202E plus `evil`, as UTF-8) included, though NTFS holds U+202E. Now the bidi row is not skipped on Windows: the name is valid UTF-8 and holds no control character. **This is reasoned and emulated, not run.** If a Windows run finds the row's other assertions (a refusal text, `str(bad.resolve())`, `os.listdir`) fail there for a reason that is not a defect, the fix is one more skip in `skip_where_this_system_cannot_name`. **Line 2: the `make_file` test under an ASCII file name encoding.** `test_make_file_writes_a_name_that_is_utf8_where_fsencode_makes_no_byte` asserted `skip_message(make_file, ...) is None`, and `make_file` cannot write `café.yml` where the encoding is ASCII, so the assert failed. The test now asks `skip_where_the_environment_cannot_carry(UTF8)` first, before it swaps in the Windows `os.fsencode`, so it skips for the encoding and for nothing else. The assert is unchanged, so a write refused with EACCES or ENOSPC still fails the test. **Beyond the issue's measured set.** The issue measured `1 failed, 6 passed, 2 skipped` under `PYTHONUTF8=0 LC_ALL=C PYTHONCOERCECLOCALE=0` on the `make_file` tests. Its Verify asks that the WHOLE file `tests/test_non_utf8_git_reads.py` pass under that environment, and on `6ba8c5d2` it gives `6 failed, 40 passed, 6 skipped`. The other five failures hold no defect either: `subprocess` puts each environment value through `os.fsencode`, which raises `UnicodeEncodeError` for `git config user.name` of an accented name, a U+200C name or a U+202E name, in the test and before the tool runs. They are `test_a_user_name_that_is_text_still_initialises[accent-dry-run]`, `[accent-real]`, `[zwnj-dry-run]`, `[zwnj-real]` and `test_every_other_name_no_file_can_hold_is_refused_the_same_way[u202e]`. Both tests now call `skip_where_the_environment_cannot_carry(value)`, which skips where `os.fsencode(value)` raises `UnicodeEncodeError` (the `make_file` test above calls it too, with the file name), and says so as "this system's file name encoding (...) cannot carry '<value>'". ASCII names, and a lone surrogate that `os.fsencode` gives back as its byte, are not skipped. On Windows and under a UTF-8 file name encoding nothing skips that did not before. Lane 1's review of `41849725` asked for three fixes and a nit. All four are in one commit, in `tests/test_non_utf8_git_reads.py` and `tests/test_non_utf8_name_callers.py`. 1. **The `make_file` test skipped on a write the disk refused.** `make_file` skips on any `OSError` of its write with a reason that contains "filesystem", so the first version of the test, which skipped on that reason, ended SKIPPED when `write_bytes` raised EACCES or ENOSPC for the plain UTF-8 name, and the reason it gave was false. The test now asks `skip_where_the_environment_cannot_carry(UTF8)` first and keeps its strict assert. The helper's reason no longer ends "in an environment", and the callers file reads "file name encoding". The ASCII row patches `os.fsencode`, and two rows make `Path.write_bytes` raise EACCES and ENOSPC and expect the test to FAIL. The broader narrowing of `make_file` stays a candidate below. 2. **Five own-directory rows failed on a disk that refuses a name that is not UTF-8** (macOS's APFS, a ZFS with `utf8only`): they let the probe through to the real disk, and the helper skipped. `a_disk_that_takes_the_probe` asks the real disk first and only those rows use it. Measured with a plugin that makes `os.mkdir` answer EILSEQ: `31 passed, 5 skipped` (without the fixture, `5 failed, 31 passed`). 3. **S5778 (two places).** `fail_where_the_caller_ends` holds the `pytest.fail`, so each `pytest.raises` block holds one call, as does the new disk-error row. 4. **The Windows rows' names.** The byte and control-character rows now call `directory_named` itself (chosen over renaming), so its order, asking before it makes anything, is held. The bidi row asks the decision function and is named for it. A new file, so it stays clear of the open branches that edit the five. Each caller is called as a plain function with the disk answering as it would on a system CI does not have: | group | what the disk does | the caller must | tests | |---|---|---|---| | macOS | the probe `probe` + U+DCFF is refused with EILSEQ | skip, on the helper's reason | 6 | | a folder nobody can write in | the probe is refused with EACCES | fail with `PermissionError`, never skip | 6 | | the test's own directory | the probe passes, the test's own directory is refused with EACCES. These rows are skipped where the real disk refuses the probe (EILSEQ): a fixture asks it first | fail with `PermissionError`, where it skipped | 5 + 1 (the fixture asked of a disk made to answer EILSEQ) | | Windows (`os.name` is `nt`, `os.fsencode` gives a surrogate three bytes) | nothing is made | `directory_named` skips for byte 0xE9 (reason names `os.fsencode`) and for a line feed and an escape (names the control character), and does not skip for the bidi override | 4 | | a system that is not Windows | the disk takes it | `directory_named` makes a directory named with a line feed or an escape | 2 | | ASCII file name encoding | `os.fsencode` is ASCII with `surrogateescape` | the `make_file` test and the two environment tests skip, on "file name encoding"; the helper skips no text that is carried | 1 + 3 + 4 + 2 | | a write the disk refuses | `Path.write_bytes` raises EACCES or ENOSPC for the plain UTF-8 name | the `make_file` test still FAILS on its assert, and does not end SKIPPED | 2 | The six callers are the four tests, `directory_named` and the file twin (the last has no directory of its own, so it is not in the own-directory group). The Windows rows for a byte and a control character call `directory_named` itself, so that it is held to asking before it makes anything; the bidi row asks `skip_where_this_system_cannot_name`, because `directory_named` goes on to make a `Path`, which under an emulated `os.name` of `nt` is a `WindowsPath` that Linux cannot make. The "carried" rows (ASCII text, a lone surrogate that is a byte, a line feed, an escape) hold that the environment skip is not wider than it should be. All with a stub `gh` first on PATH, `GH_TOKEN=bogus`, an empty `GH_CONFIG_DIR` and `/usr/bin/python3`, at head `6b5e9074`. Per file: `test_stdout_encoding.py` 48, `test_adopt_git_reads.py` 21, `test_adopt_template_values.py` 78, `test_adopt_plan_escapes.py` 59, `test_clone_url_bytes.py` 30, `test_non_utf8_git_reads.py` 52, `test_non_utf8_name_helpers.py` 18, `test_non_utf8_name_callers.py` 36, and the four files that import `test_stdout_encoding.py` (`test_bootstrap_branch_bytes.py` 61, `test_bootstrap_spelling.py` 205, `test_siblings_family_name.py` 150, `test_validator_child_encoding.py` 55). - **The twelve files above:** `811 passed, 2 skipped in 295.42s`. Both skips are the same two as on the base and have nothing to do with this change: `test_stdout_encoding.py:285` ("the default this is about is Windows' own") and `test_adopt_template_values.py:224` (`git config user.name` here is not read from `GIT_CONFIG_GLOBAL`). - **The Verify's C locale**, `PYTHONUTF8=0 LC_ALL=C PYTHONCOERCECLOCALE=0`, `tests/test_non_utf8_git_reads.py` and the new file: `74 passed, 14 skipped in 20.01s`. The 14 are the 6 that skipped on the base, the 6 this change turns from failures into skips (the `make_file` test, two accent rows, two U+200C rows and the U+202E row), and the 2 disk-error rows of the new file, which need an encoding that can write the name. On the base the first file gives `6 failed, 40 passed, 6 skipped`. - **A disk that answers EILSEQ to a name that is not UTF-8** (a test plugin that makes `os.mkdir` answer so, as macOS's APFS does), the new file: `31 passed, 5 skipped`. The 5 are the own-directory rows, skipped by the fixture; without it they fail (mutant M9b). - **`tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py` and `tests/test_cli_reference.py`:** `146 passed in 31.39s`. The new file copied onto `6ba8c5d2` fails 33 of its 36 tests: `33 failed, 3 passed in 27.36s`. The decisive lines: the probe-refusal rows end `Failed: the caller ended without an error` (the base never asks the probe); the own-directory rows end `Failed: the caller ended SKIPPED: no directory here is named b'caf\xe9': PermissionError(13, 'Permission denied')` (stdout: `this filesystem holds no name that is not UTF-8: [Errno 13] Permission denied`; `directory_named`: `this disk cannot hold the name b'caf\xe9': [Errno 13] Permission denied`); the C-locale rows fail on `UnicodeEncodeError: 'ascii' codec can't encode character ...` and on the `make_file` assert. The 3 that pass on the base are the two disk-error rows (the base's `make_file` test is strict, so they guard against the first version of this change, which skipped: on `41849725` they and the ASCII row fail, `3 failed, 33 passed`) and the fixture row, which holds a fixture the new file defines. The seven existing files (the five, `test_non_utf8_git_reads.py` and `test_non_utf8_name_helpers.py`) pass on the base too (`304 passed, 2 skipped`). Each hunk reverted alone, in a throwaway clone at `6b5e9074`, new file only (36 tests). Every one is caught, by the test it is meant for, except where it says otherwise. | mutant | result | caught by | |---|---|---| | M1 / M2 / M3 / M4: the helper call removed from the stdout, git-reads, template-values and plan-escapes directory test | `2 failed, 34 passed` each | the EILSEQ row and the EACCES-on-the-probe row of that caller | | M1b / M2b / M3b / M4b: the same four with their own `mkdir` guarded by `except OSError: pytest.skip` again | `1 failed, 35 passed` each | the own-directory row of that caller | | M5: the file twin's helper call removed | `2 failed, 34 passed` | the EILSEQ and EACCES rows of `adopt-plan-escapes-file` | | M5b: the file twin's old broad write-and-unlink probe put back | `2 failed, 34 passed` | the same two | | M6a: the helper call removed from `skip_where_this_system_cannot_name` | `3 failed, 33 passed` | EILSEQ, EACCES and the Windows byte row | | M6b: `directory_named`'s own `os.mkdir` guarded again | `1 failed, 35 passed` | the own-directory row of `clone-url-bytes` | | M6c: the old blanket `os.name == "nt"` skip back | `1 failed, 35 passed` | `test_where_windows_holds_a_bidi_override_the_name_check_does_not_skip` | | M6d: the control-character test made `False` | `2 failed, 34 passed` | both Windows control-character rows | | M6e: `directory_named` asks after it makes the directory | `3 failed, 33 passed` | the Windows byte row and both control-character rows (new in the fix round: they call `directory_named` itself) | | M7: the `skip_where_the_environment_cannot_carry(UTF8)` line removed from the `make_file` test | `1 failed, 35 passed` | `test_make_files_utf8_name_test_skips_where_the_encoding_is_ascii` | | M7b: the broad `"filesystem" in reason` skip back in the `make_file` test, the encoding question kept | `2 failed, 34 passed` | `test_a_write_the_disk_refuses_still_fails_the_make_file_test[EACCES]` and `[ENOSPC]` (new) | | M8a: `skip_where_the_environment_cannot_carry` returns instead of skipping | `6 failed, 30 passed` | the `make_file` ASCII row, the three "cannot carry" rows and the two-test rows | | M8b / M8c: its call removed from the text test, or from the every-other-name test | `1 failed, 35 passed` each | the matching row of the two-test group | | M8d: its reason no longer says "file name encoding" | `6 failed, 30 passed` | the same six, on the new `"file name encoding" in reason` asserts | | M9a: the probe fixture `a_disk_that_takes_the_probe` asks nothing | `1 failed, 35 passed` | `test_the_fixture_that_guards_those_rows_asks_the_real_disk` (new) | | M9b: the own-directory rows use the system-only fixture again | `36 passed` on this disk; `5 failed, 31 passed` under a plugin that makes `os.mkdir` answer EILSEQ to a name that is not UTF-8 | **no committed test**: Linux's disk never answers EILSEQ. The probe run is the evidence, the five own-directory rows | | M10: `fail_where_the_caller_ends` never fails, with the stdout helper call also removed | `2 failed, 34 passed` | the same two rows as M1 (`DID NOT RAISE PermissionError` for the EACCES row). The helper is a refactor of what each row did inline; it has no behaviour of its own that Linux can change | `git merge-tree --write-tree HEAD <ref>` at `6b5e9074`, against origin/main (`bf1f099b`), against #723 (`6ba8c5d2`, my parent), against `fix/693-local-remote-dir` (`831d76c6`, the one open branch that edits `tests/test_clone_url_bytes.py`) and against every other wave-7 branch (local refs): all clean except two, which are inherited. At the first push (`41849725`) every `fix/*` and `l3-*` head on origin (24 + 22) was checked too, with the same result. - **`fix/672-bump-leg-branch-and-push-line` (#746, `a5d473ee`) conflicts in `tests/test_non_utf8_git_reads.py`, one region (lines 316-324 of the merged file). It is inherited, not mine:** `merge-tree 6ba8c5d a5d473e` (#723 alone against #746) conflicts in the same file and the same region, and my hunks are at lines 123-127, 497-513, 647 and 668. Whichever of #723 and #746 lands second needs the one conflict resolved by hand, and this branch follows #723. - **`fix/758-git-command-scope` (local `ceca042f`) conflicts in the same file, in the import block (lines 54-62 of the merged file).** Also inherited: `merge-tree 6ba8c5d ceca042f` conflicts in the same lines. - `fix/663-...` (#723), `fix/645-...` (#713) and `fix/662-...` (#748) touch `tests/conftest.py`, which this branch does not edit. `l3-785-non-ascii-paths` (`a1e2c9a8`) touches `tests/test_stdout_encoding.py` and merges clean. - **`make_file`'s own skip** (`tests/test_non_utf8_git_reads.py:100`) turns any `OSError` or `UnicodeError` of the write into a skip. It is the same defect as line 1, in the helper that #723 added, and the issue names it only as the model for the file twin and for line 2. See the candidates. - **A real Windows or macOS run.** Not possible here. - **Two more tests make their own "a name that is not UTF-8, or skip" on any `OSError`:** `tests/test_cli_reference.py:632` (`test_a_checkout_whose_path_is_not_utf8_is_a_checkout`, `parent.mkdir()`, then `pytest.skip`) and `tests/test_doctor_validator_verdicts.py:317` (`here.mkdir(parents=True)`, then `pytest.skip`), at `6ba8c5d2`. Same fix as line 1: `skip_where_a_name_cannot_be_not_utf_8(tmp_path)`, then an unguarded `mkdir`. Neither is in #757's lines. - **`make_file` (`tests/test_non_utf8_git_reads.py:100`) skips on any `OSError` of `path.write_bytes`,** where the helper it calls first raises all but EILSEQ. Only the encoding error (a `UnicodeError` under an ASCII file name encoding) and EILSEQ are the system's answer; a full disk should fail the test. - **`tests/test_adopt_template_values.py:314`** (`test_execute_refuses_a_local_source_directory_holding_a_bidi_control`) makes a directory `U+202E` plus `caf` and skips on any `OSError` ("no directory here is named ..."). The name is text, not a byte that is not UTF-8, so the helper is not its question, but the broad skip hides a full disk the same way. Review: lane 1's re-verification of 6b5e907 is PR-READY (issuecomment-6233426675); 76ae641 restacks it onto main after #723 with the same changed lines. `a1498632` merges main `e62b5e17` after #818: its one conflict, `tests/test_adopt_git_reads.py`'s `from conftest import` line, takes both sides' names, and its -U0 changed lines against main equal the reviewed delta but for that line. CI (Brett Heap's ruling 27 and his fast-refresh ruling of 2026-10-15T22:48:35Z): H0 a149863 passed tests, tests-git-bash and SonarCloud; the head 22238b4 is a clean update-branch merge of main d6475f9, with tests-git-bash and SonarCloud green; and the local gate passed on its predicted squash tree 9f94a18 over main d6475f9: the adopt_git_reads, adopt_plan_escapes, adopt_template_values, clone_url_bytes, non_utf8_git_reads, non_utf8_name_callers, stdout_encoding, repo_hygiene, agents_md_codes, cli_reference sets, 486 passed, 2 skipped. Landed on Brett Heap's word of 2026-10-17T21:47:28Z, in lane 5's session (posted on #473 as issuecomment-6241758069): "merge lane 5's v1.1.1 PRs once reviewed and green". Closes #757 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n as it reads the file fails `make bootstrap` naming the key, and the family gate names it in every mode, so `update = None` no longer ends `family bootstrap ok` (#768) (#819) Written on #682 (`fix/648-bootstrap-and-gate-order`, `f74f640c`), which landed as `dc7d3fbb` while this was written. The commit is `fa69ead9`. The branch then merged #682's final head `5afb24b9` and main, taking its own side for #682's files: those files are the same at `5afb24b9` and `dc7d3fbb`, and main changed none of them after the squash. The head is `c12b5f8a`, and its diff against main is this PR's four files. **Pinned copies.** This changes the family root's `templates/family-root/scripts/bootstrap.py` and `templates/family-root/scripts/validate-family.py`. Each holder sees them as `upstream-changed` on its next `update-shape.py check`, and they are re-pinned in the next shape bump; this PR re-pins nothing. No test pins either file's bytes or digest. README, the handbook, `AGENTS.md` and `docs/cli.md` are unchanged: no `--help` text changes, and no sentence in them is made false. CI's run 39031235254 (git 2.55.0) failed one row of the new test file, and lane 2's review rv819 of `c12b5f8a` (NOT READY) found one MUST, four SHOULDs and five NITs. This commit answers each of them. It changes `templates/family-root/scripts/bootstrap.py` and `templates/family-root/scripts/validate-family.py`, both pinned and both already changed by this PR, and `tests/test_family_gitmodules_values.py`. No other pinned file changes. ### What git does with a `branch` that has no value Measured on git 2.43.0 (the host's) and 2.47.1 (a local build). The probe adds a `branch` line with no value after a submodule's url, commits it, then runs `git submodule update --init --recursive` in a plain clone: | `.gitmodules` read from | git 2.43.0 | git 2.47.1 | |---|---|---| | the work tree's file | crash, exit 139 (`Segmentation fault`) | `error: missing value`, `fatal: bad config line 4 in file`, exit 128 | | the index's (work tree file lost) | crash, exit 139 | `error: bad config line 4 in blob`; it reads on, checks the submodule out, exit 0 | | HEAD's (index entry lost too) | crash, exit 139 | the same as the index's | A `url` with no value gives 2.47.1's answers on both gits. `branch =`, `branch = .` and `branch = -x` are taken by both gits from every source (exit 0), as both tools read them. CI's log shows git 2.55.0 giving 2.47.1's answer for the work tree's file. From git's source, `submodule-config.c`: the `branch` case has `if (!value) ret = config_error_nonbool(var);` at v2.43.1 and v2.44.0. v2.43.0, v2.42.4 and v2.39.5 go straight to `xstrdup(value)`, which crashes. ### Each finding 1. **MUST: the row pinned git 2.43's crash. Fixed.** Reproduced on git 2.47.1 at `c12b5f8a`: `assert 128 == 139`, as on CI (1 failed, 27 passed). The table's exit column is gone. The premise is now that git's update died and checked nothing out; every other assertion is about the tools' own output. The two `shutil.Error` setup errors in that run are #645's copy race, which #713 fixed on main. 2. **SHOULD: the index and HEAD rule fixed `branch` to git 2.43's crash. Fixed, by measurement.** On 2.47.1 at `c12b5f8a`, with the work tree's file lost and no credential, bootstrap said `NOT CHECKED OUT, and not for want of a credential: ... git cannot read .gitmodules`, right under git's own `bad config line 7 in blob`, and failed. New `crashes_on_branch` reads `git version`. Before 2.43.1, or where it cannot read a version, the key is refused as before. From 2.43.1 on, the line ends the reading of a blob, as a `url` with no value does (`ends_blob_reading`). The gate needs no change: it reads the work tree's file and HEAD's, which a clone checks out as its work tree's file, and every git dies on the key there. The comments that said "git crashes" now say which git does. 3. **SHOULD: with every member checked out, a `.gitmodules` git cannot parse ended `family bootstrap ok`. Fixed.** `Gitmodules` now carries `unreadable`. It is set where the work tree's file is one `git config --list` cannot parse, which git dies reading. With every member checked out, the run prints `NO MEMBER UPDATED, and not for want of a credential: git cannot parse .gitmodules. Mend .gitmodules by hand (`git config -f .gitmodules --list` says where it breaks), and commit it.` and fails on `.gitmodules (git cannot read it)`, as it does for a refused value. A row left not checked out now gets that reason too, where before it was "no section of .gitmodules gives that path". Tested with a conflict marker and with an open quote. A blob is not marked: see the checklist candidates. 4. **SHOULD: mutants survived. Each now fails a test:** - Reading HEAD's file in place of the index's: `test_the_index_s_file_is_read_where_it_differs_from_head_s` stages `update = None` without committing it. - `update` without `merge` in bootstrap, or without `rebase` in the gate: `update = rebase` and `update = merge` are now in `TAKEN`. - The stop-reading rule applied inside ignored sections: `test_a_key_with_no_value_in_a_section_git_ignores_reads_on`. - `pattern_names` matching a prefix without the `/`: a new `a-string-prefix` row, with two `submodule.active` patterns, `members/IRRS` and `members/IRS`. - `active` reading the first value: OTHER's own `active` is now set twice, `true` then `false`. - `_written_back` keeping a url that begins with `-`: a new `url-after-a-dashed-one` row. - The `<url>` placeholder: `test_a_url_no_row_gives_one_for_is_written_back_as_a_placeholder`. - `_submodule_entries` without the backslash rule: the ignored-section test now runs on `..\x` too. - `if refused_here:` made `if refused:`: `test_a_value_head_s_file_alone_gives_leaves_the_work_tree_s_read`. 5. **SHOULD: cognitive complexity. Split.** These are estimates from an AST count of Sonar's rules. At `c12b5f8a` it gives the same figures rv819 gave. - `gitmodules_sections`: 33 → 5. Its parts are `gitmodules_source` (8), `gitmodules_entries` (9), `ends_blob_reading` (2) and `read_sections` (13). - `unfetchable`: 16 → 10, with `row_reason` (8). - `cures`: 24 → 22. It was 23 on main, so this PR no longer raises it; its `.gitmodules` sentences moved to `gitmodules_cures` (3). - `_check_gitmodules_values`: 22 → 10, with `_value_said` (2) and `_value_taken_out` (5). - `fetch_members`: 10 → 13. 6. **NIT: duplicated test code. Done in this file.** The gate-in-three-modes block is now `found_in_every_mode_and_typed`, and the good-run block is `assert_a_good_holder_s_run`. The 4-line `EXITED_0` assertion it shares with `test_family_bootstrap_gate_order.py` stays: it is shorter than a duplicated block, and sharing it means editing a file #682 landed. 7. **NIT: the rules are written twice. Left.** One copy in `repo_shape.py` would change the module every root imports, and every pinned copy of it, which is more than a fix round should carry. Filed below as a candidate. 8. **NIT: a false sentence in the HEAD-only case. Fixed.** Where HEAD's file alone gives the value, the finding now ends "and git dies reading a file that does, so in a clone neither `git submodule update` nor make bootstrap checks out any member." 9. **NIT: the person's own config could change the boolean answer. Fixed.** Measured: with `probe.value = notabool` in a global config, in `GIT_CONFIG_COUNT`, or given with `-c`, the old probe dies (`bad boolean config value 'notabool'`). Both tools now ask `git config --file <null device> --type=bool --default=<value> --get probe.value`, which reads no config scope (measured on 2.43.0 and 2.47.1). New test: `test_a_probe_value_in_the_person_s_own_config_changes_no_answer`. 10. **NIT: a count in this body. Corrected here.** Under "Fail on base", seven of the 28 tests pass on the base, not six. The seventh is the control `[a-leading-directory]`. The totals, 24 failed and 18 passed, were right. ### Tests `tests/test_family_gitmodules_values.py` now has 52 tests, up from 28. Every run used `run9.sh`: a `gh` stub, a bogus token and an empty `GH_CONFIG_DIR`. | run | tree | git | result | |---|---|---|---| | the file | e1379a9's | 2.47.1 | `52 passed in 67.32s` | | the file, `test_repo_hygiene.py`, `test_agents_md_codes.py`, `test_cli_reference.py`, `test_pinned_copy_text.py`, `test_bootstrap_spelling.py`, `test_validate_family_without_git.py`, `test_family_bootstrap_gate_order.py`, `test_family_head_lost_gitmodules.py` | e1379a9's | 2.43.0 | `573 passed in 252.33s` | | wide half A: 26 files that load or run the family root's `bootstrap.py` or `validate-family.py`, this file among them | e1379a9's but for three docstrings and one byte match in a test | 2.43.0 | `1474 passed, 22 skipped, 1 xfailed in 1321.42s` | | wide half B: 29 more such files, the three above among them | the same | 2.43.0 | `1933 passed, 1 skipped in 1097.64s` | | the file, `test_family_bootstrap_gate_order.py`, `test_bootstrap_after_bump.py`, `test_family_head_lost_gitmodules.py`, `test_repo_hygiene.py` | the predicted merge with main `64ec47c3` | 2.43.0 | `250 passed in 229.25s` | The wide set is the 49 files w768 found with `git grep`, plus 6 more that name the family root's `bootstrap.py` (`test_family_siblings_placement.py`, `test_park_resume_targets.py`, `test_pins_child_kind_and_siblings_root_spelling.py`, `test_readme_windows_quoting_sentence.py`, `test_setup_project_py.py`, `test_windows_paths.py`). Both halves ran in a clone with that tree committed, since the `update-shape` tests fail on a work tree whose copies are not at HEAD. ### Fails on base The new file was run against `c12b5f8a`'s scripts: - on git 2.47.1: 18 failed, 34 passed (`r2-base247.log`); - on git 2.43.0: 16 failed, 36 passed (`r2-base243.log`). The tests that fail there: - the three unparseable-file tests; - the two blob `branch` tests (2.47.1 only); - the nine `git version` rows; - the three HEAD-only texts; - the probe test. The tests added for finding 4 pass on the base, as they should: each catches a mutant, not a defect of the base. The original file fails on git 2.47.1 at `c12b5f8a` with CI's assertion: 1 failed, 27 passed. ### Mutants Each mutant was applied alone to the final tree and caught (`mutants2.log`): | id | change | caught by | |---|---|---| | M1 | a blob's `branch` never ends the reading | blob `branch` tests, git 2.47.1 (2.43.0 passes, as it should) | | M2 | a blob's `branch` always ends the reading | blob `branch` tests, git 2.43.0 (2.47.1 passes, as it should) | | M3 | `BRANCH_READ_SINCE` is (2, 44, 0) | `which_git[2.43.1]` | | M4 | `<` made `<=` | `which_git[2.43.1]` | | M5 | an unreadable version taken for one that does not crash | `which_git[unread]` | | M6 | the version read with two numbers | `which_git[2.43.1]` | | M7 | `unreadable` never set | the three unparseable-file tests | | M9 | `fetch_members` asks `refused` alone | the every-member unparseable tests | | M10 | no mend sentence | the three unparseable-file tests | | M11 | `row_reason` without `unreadable` | the per-row unparseable test | | M12 | bootstrap's old boolean probe | the probe test | | M13 | a blob never ends the reading | the `url` index test and the blob `branch` tests, git 2.47.1 | | M14 | ignored sections asked for refused values | the ignored-section tests | | M15 | HEAD's file read in place of the index's | the staged-index test | | M16 | the stop rule inside ignored sections | the ignored-section blob test | | M17 | `pattern_names` matches a bare prefix | `[a-string-prefix]` | | M18 | `active` reads the first value | the `active` test | | M19 | bootstrap's `update` without `merge` | `TAKEN[update-merge]` | | V1 | the HEAD-only sentence never used | the three HEAD-only texts | | V2 | the gate's old boolean probe | the probe test | | V3 | `_written_back` keeps a url that begins with `-` | `[url-after-a-dashed-one]` | | V4 | no `<url>` placeholder | the placeholder test | | V5 | `_submodule_entries` without the backslash rule | `[backslash]` | | V6 | `if refused:` for `if refused_here:` | the HEAD-only `gitmodules-path-twice` test | | V7 | the gate's `update` without `rebase` | `TAKEN[update-rebase]` | ### Merge checks `git merge-tree --write-tree` of this head with: - main `64ec47c3`: clean, and its tests pass (see Tests); - #747 `ff7c59b8`: clean; - #822 `ea86cb60`: clean; - #815 `f6f318d1`: clean; - #820 `9e2dc7d9`: conflicts in `validate-family.py`, in the same three regions as before. The conflict blocks are byte for byte the ones `c12b5f8a` gives, so the resolution rv819 checked still applies. ### Checklist candidates - **A blob git cannot fully parse is read as no section at all.** Measured on 2.43.0 and 2.47.1: - Where the work tree has no `.gitmodules` and the index's file holds a bad line, git reads the sections before that line, says `error: bad config line 4 in blob`, and checks the members out. - `git config --blob <oid> --list` prints those entries but exits 128. So `gitmodules_sections` reads nothing. - With no credential, every row is then named "no section of .gitmodules gives that path" and the run fails, where the members are only the credential's. - This came with #648's blob reading. This round leaves it as it was. - **One copy of the `.gitmodules` value rules** (rv819's NIT 7). Bootstrap's `NO_VALUE_REFUSED`, `UPDATE_VALUES`, `refused_value`, `git_bool` and `TAKES` and the gate's `_values_refused`, `_git_takes_bool` and `GIT_TAKES` could live in `repo_shape.py`, so a change to git's rules is made once. Git 2.43 dies reading a family holder's `.gitmodules` before `git submodule update` checks out any member when a section gives: - a `path`, `url`, `update`, `ignore` or `branch` with no value (`bad config line`; a `branch` with none crashes git, exit 139); - an `update` other than `none`, `checkout`, `rebase` or `merge`, a `!command` included (`invalid value`); - a `shallow` that is no boolean (`bad boolean config value`); - a `fetchRecurseSubmodules` that is neither a boolean nor `on-demand`. The holder's bootstrap listed the file with `git config --list`, which reads all of these, and ended `family bootstrap ok`, exit 0. The gate refused `member-uninitialized` members checked and under `--pins`, naming the update that dies, and said `family ok` under `--no-members`. ### Line 1: bootstrap `gitmodules_sections` checks each `submodule.*` entry against those rules (`refused_value`). Where one would make git die, it returns those entries and no sections, as it returns `unmerged`, and `unfetchable` asks that second, after `unmerged`. Every row not checked out gets one reason: ``` NOT CHECKED OUT, and not for want of a credential: members/Atlas (git cannot read .gitmodules: `submodule.members/Nova.update` is `None`), members/Nova (git cannot read .gitmodules: `submodule.members/Nova.update` is `None`). Give `submodule.members/Nova.update` a value git takes, one of `none`, `checkout`, `rebase` or `merge`, or take that line out, and commit .gitmodules. `python3 scripts/validate-family.py --no-members` prints the commands. family bootstrap FAILED: Atlas (not checked out), Nova (not checked out) ``` A key with no value says "has no value". As for every other reason, the holder's records decide, never git's stderr. **Where every member is checked out**, as in the holder itself, git dies before it updates any of them, so no row is left to name, and the run used to say ok over git's `fatal:` line. It now prints `NO MEMBER UPDATED, and not for want of a credential: git cannot read .gitmodules: ...` with the same cure, and fails on `.gitmodules (git cannot read it)`. That is my reading of the goal, "so neither one says ok over a file git refuses". The issue's direction named only the rows not checked out. ### Line 2: the gate A new finding, `gitmodules-value-refused`, for each such key in the work tree's `.gitmodules` and in HEAD's (HEAD's as `member-gitmodules-absent` reads it). It is asked before any member check, so `--no-members` makes it too. Where either file gives such a value, a member that is not checked out is not refused `member-uninitialized`: git checked none out, and the finding fails the run, exit 1, in all three modes. The remedy prints one command to a line, as `gitmodules-path-unnamed` does, says which values git takes, and ends with the commit: ``` FINDING gitmodules-value-refused: .gitmodules gives `submodule.members/Nova.update` the value `None` here and in HEAD, which a clone reads, and git dies reading a file that does, before `git submodule update` checks out any member, so neither that nor make bootstrap checks one out. Git takes one of `none`, `checkout`, `rebase` or `merge` for that key. Take the key out by hand, in a POSIX shell, then stage and commit .gitmodules: git config -f .gitmodules --unset-all submodule.members/Nova.update ``` Decisions: - **A member's `path` or `url` is written back.** `--unset-all` takes out every line of the key. For `update`, `ignore`, `shallow`, `fetchRecurseSubmodules` and `branch`, git reads a section without one, so that is the cure. In the issue's `url` state, Nova's section has its url and one more `url` line with no value, and `--unset-all` alone would leave no url (git: `No url found`). So for a member's own section the finding writes the key back on a second line: the value the section last gave that git takes, or, where it gave none, what `family.py add` writes (the section's name for `path`, `https://github.com/<repository>.git` for `url`, `<url>` where the row's `repository:` is not `<org>/<Name>`). - **Which file gives it.** The finding says "here and in HEAD", "in the work tree, not in HEAD" (nothing to commit), or "in HEAD, not in the work tree" (it says to stage and commit the work tree's file, and prints no command). Where the work tree has no `.gitmodules`, the first such finding prints `git checkout HEAD -- .gitmodules` first, and only the first: a second would put back the key the first took out. - **The other `.gitmodules` checks wait.** Where the work tree's file gives such a value, `gitmodules-path-twice` and `gitmodules-path-unnamed` read nothing from it, as git reads nothing. Without that, a `path` with no value was read as the empty path, and `gitmodules-path-unnamed` printed `--unset-all` for every key of the member's own section (measured on `f74f640c`, probe c8). Those checks run again once the file is mended. - **A boolean is git's own.** `shallow` and `fetchRecurseSubmodules` are read by `git -c probe.value=<value> config --type=bool --get probe.value`, which uses the same `git_parse_maybe_bool` the submodule parser does. That covers `0x10`, `010`, `1k` and the int range (`2097152k` and `-2097152k` die, `2097151k` does not) with no copy of git's number parsing. ### Line 3: what #682 left approximate - **Where git reads the file from.** Where the work tree has no `.gitmodules`, `gitmodules_sections` reads the index's stage-0 blob, then `HEAD:.gitmodules`, with `git config --blob`, as `config_from_gitmodules` does. The unmerged check now uses `git ls-files -s`, which gives both the conflict stages and the stage-0 blob. One difference from a file is measured and kept: in a blob, a key with no value is an error, not a die. Git stops reading at that line and goes on, so `gitmodules_sections` stops there too (probe b2: the last line, git exits 0 and checks both out; b2b: the first section, git dies `No url found` at the second). - **`submodule.<name>.active`.** `submodule_config` now returns each key's values. Where `submodule.active` is set, `init_stop` asks the section's own `active` first, its last value read as a boolean, as `is_tree_submodule_active` does. - **`submodule.active`'s patterns.** A plain path pattern, as `git clone --recurse-submodules=<path>` writes one, names that path and what is under it, and `.` names every path. A leading `./` and a trailing `/` are read past. A pattern with pathspec magic (a leading `:`), a glob character or a backslash, or an empty, `.` or `..` component, keeps the old reading: it names every path. Measured with git 2.43.0 in plain clones of a holder built by this branch's `scaffold-project.py` and `family.py` (Atlas and Nova), each line added to Nova's section and committed (`p1.sh`; 32 states). On `f74f640c` every state git's update dies on ended `family bootstrap ok` with `--no-members` saying `family ok`; on this head, bootstrap fails and the gate finds `gitmodules-value-refused` in all three modes on exactly the states where git's update exits non-zero, and on none of the others: | added to Nova's section | git's update | `f74f640c` bootstrap / `--no-members` | this head | |---|---|---|---| | `update = None`, `update = !true`, `update = Merge` | 128, `invalid value` | ok / `family ok` | exit 1 / finding | | `url`, `update`, `ignore` (no value) | 128, `bad config line 7` | ok / `family ok` | exit 1 / finding | | `path` (no value) | 128, `bad config line 7` | exit 1, wrong reasons / `gitmodules-path-unnamed` for the path "" | exit 1 / finding | | `branch` (no value) | 139, `Segmentation fault` | ok / `family ok` | exit 1 / finding | | `shallow = maybe`, `08`, `3000000000`, `2097152k`, `-2097152k`, `-2097153k` | 128, `bad boolean config value` | ok / `family ok` | exit 1 / finding | | `fetchRecurseSubmodules = bogus`, `ON-DEMAND` | 128, `bad ... argument` | ok / `family ok` | exit 1 / finding | | `update = none` | 0, Nova skipped | exit-0 skip | the same exit-0 skip, no finding | | `update = checkout`, `ignore = bogus`, `shallow` (no value), `shallow = true`, `""`, `" 1"`, `TRUE`, `1 `, `0x10`, `010`, `1k`, `2097151k`, `fetchRecurseSubmodules` (no value), `= on-demand`, `= 0` | 0 | ok / `family ok` | ok / `family ok` | `p2.sh` (index and HEAD fallback, `active`), on `f74f640c` then this head: | state | git | `f74f640c` | this head | |---|---|---|---| | work tree's file deleted, not staged, no credential | reads the index's; fails on the fetch | exit 1, "no section of .gitmodules gives that path" | the credential's skip, ok | | the index's taken out too | reads HEAD's; fails on the fetch | exit 1, the same | the credential's skip, ok | | `update = None` committed, work tree's file deleted | 128, dies on the index's blob | exit 1, "no section" | exit 1, "git cannot read .gitmodules" | | Nova's url taken out, subset `members/Atlas`, no credential | inits Atlas alone; fails on its fetch | Atlas "git stopped at members/Nova" | Atlas the credential's; Nova "git has no url" | | the same with `submodule.active=.` and `submodule.members/Nova.active=false` | inits Atlas alone; fails on its fetch | Atlas "git stopped at members/Nova" | Atlas the credential's; Nova "git has no url" | A section under a name git ignores whole (`../x`) with `update = None` is ignored by git (exit 0, a warning) and by both tools (`p3.sh`). `tests/test_family_gitmodules_values.py`, new, beside #682's `tests/test_family_bootstrap_gate_order.py`, whose helpers it imports (28 tests): - `test_a_value_git_dies_on_fails_bootstrap_and_every_mode_of_the_gate`, one case for each row of the issue's table, `update = None` first, and `branch` with no value: git's update exits 128 (139 for `branch`) and checks nothing out; bootstrap exits 1 with its last two stderr lines exact; the gate finds the whole finding in all three modes, exit 1; its commands are typed as printed and committed; HEAD's `.gitmodules` is then byte for byte the one `family.py add` wrote; the same bootstrap is a good holder's, and the gate passes in every mode. - No misfire: `update = checkout`, `shallow = true`, `fetchRecurseSubmodules = on-demand`, `ignore = bogus` (a good holder's run and gate); `update = none` (the exit-0 skip as before, no finding); a section git ignores whole. - Every member checked out: the holder itself fails on `.gitmodules`, and mended as the gate prints it, passes. - The index's file and HEAD's (no credential: the skip, ok); a key with no value at the end of the index's file (read past, the skip); `update = None` in the index's file (named, and the gate's `git checkout HEAD -- .gitmodules` first); two keys there (one checkout line). - `submodule.members/<P>.active=false` with `submodule.active=.`; the subset's pattern as `members/<P>`, `./members/<P>/` and `members`. - Which file gives it (HEAD's alone; the work tree's alone); a `url` or `path` left with no value, written back as `family.py add` writes it. **Fail on base.** The file run against `f74f640c`'s two scripts: 21 of 28 fail, and the three changed cases of `tests/test_validate_family_without_git.py` fail too: 24 failed, 18 passed (`base1.log`). The six that pass there are the no-misfire cases, which must pass on both. **Runs on this head** (Linux, git 2.43.0; through `run9.sh`, the real `gh` unreachable): - Every test file that loads or runs the family root's `bootstrap.py` or `validate-family.py`, found with `git grep` (49 files, the issue's Verify set, `test_family.py` whole, `tests/test_repo_hygiene.py`, `tests/test_agents_md_codes.py`, `tests/test_cli_reference.py` and `tests/test_pinned_copy_text.py` among them), in two halves on `fa69ead9` with a clean tree: `1309 passed, 22 skipped in 1431.96s` (`fa.log`) and `1644 passed, 1 skipped in 1258.67s` (`fb.log`). - `render-cli-reference.py --check`: `[ok] docs/cli.md is what the generator produces`. **One existing test changed.** `tests/test_validate_family_without_git.py::test_with_no_gitmodules_the_gitlink_listing_is_named` pins which check meets a missing git first in a holder whose work tree lost `.gitmodules`. That is now the reading of HEAD's `.gitmodules` for a value git dies on, so the `git-not-runnable` refusal says "HEAD's .gitmodules could not be read as git reads it" where it named the gitlink listing. Exit 2, the code and the remedy are as they were. No open branch edits that file. **A dirty-tree artifact, not a failure.** Run on the uncommitted tree first, `tests/test_update_shape.py`, `tests/test_update_shape_header_comments.py` and `tests/test_update_shape_pr_base.py` failed (`update-conflict`: the copy is not the template at the pinned commit), because their fixtures pin HEAD while the copies came from the edited templates. On a committed tree they pass: `171 passed` (`c1.log`), and in the halves above. **Mutants.** One for each hunk that changes behaviour, each run against the new file's tests that name it: 30 mutants, all killed (`mutants.log`, `mutants-2.log`). Three survived the first round; each showed a test that did not reach the hunk, and the test was made to reach it: | file | hunk | mutant | killed by | |---|---|---|---| | bootstrap.py | `NO_VALUE_REFUSED` | `branch` dropped | `[branch-no-value]` | | bootstrap.py | `fetch_members` | the every-member-checked-out branch never taken | `test_a_holder_with_every_member_checked_out_fails_on_it_too` | | bootstrap.py | `reason` | "has no value" reworded | `[url-no-value]` | | bootstrap.py | `cures` | the refused sentence dropped | `[update-None]` | | bootstrap.py | `gitmodules_sections`, source | the work tree's file always read | the index's and HEAD's tests | | bootstrap.py | `gitmodules_sections`, blob stop | no `break` | `test_a_key_with_no_value_ends_git_s_reading_of_the_index_s_file` | | bootstrap.py | `gitmodules_sections`, refused | never recorded | `[update-None]` | | bootstrap.py | `gitmodules_sections`, ignored sections | asked for refused values | `test_a_value_in_a_section_git_ignores_whole_is_no_finding` (survived until it ran bootstrap with no credential first) | | bootstrap.py | `refused_value`, `update` | every value taken | `[update-None]` | | bootstrap.py | `refused_value`, `on-demand` | not exempt | `[fetch-on-demand]` (survived until it ran with no credential first) | | bootstrap.py | `git_bool` | every value a boolean | `[shallow-maybe]` | | bootstrap.py | `submodule_config` | values dropped | the `active` and pattern tests | | bootstrap.py | `init_stop` | the old condition | the `active` and pattern tests | | bootstrap.py | `active` | own key not read | `test_a_member_whose_own_active_is_false_is_not_where_git_stops` | | bootstrap.py | `pattern_names` | every pattern names every path | `[a-path]` | | bootstrap.py | `unfetchable`, read with no row left | not read | `test_a_holder_with_every_member_checked_out_fails_on_it_too` | | bootstrap.py | `unfetchable`, the reason | not given | `[update-None]` | | validate-family.py | `_gitmodules_listed` | returns nothing | `[update-None]` | | validate-family.py | `_head_listed` | HEAD's listing empty | `[head-only]` (the first, badly written mutant still read it) | | validate-family.py | `GIT_TAKES` | `update`'s values reworded | `[update-None]` | | validate-family.py | HEAD-only branch | never taken | `[head-only]` | | validate-family.py | `git checkout HEAD -- .gitmodules` | never printed | `test_a_value_git_dies_on_in_the_index_s_file_is_named_and_mended` | | validate-family.py | the checkout printed once | printed for each key | `test_two_keys_with_no_gitmodules_in_the_work_tree_put_head_s_back_once` | | validate-family.py | the commit clause | always said | `[work-tree-only]` | | validate-family.py | `_submodule_entries` | ignored sections kept | `test_a_value_in_a_section_git_ignores_whole_is_no_finding` | | validate-family.py | `_values_refused` | `branch` dropped | `[branch-no-value]` | | validate-family.py | `_git_takes_bool` | every value a boolean | `[shallow-maybe]` | | validate-family.py | `_written_back` | never writes back | `[url-no-value]` | | validate-family.py | `_check_members`, blanking | sections read all the same | `[path]` of the written-back test (`gitmodules-path-unnamed` for the path "") | | validate-family.py | `_check_members`, skipped checkouts | `member-uninitialized` again | `[update-None]` (members mode exits 2) | `git merge-tree --write-tree` of `fa69ead9`: | against | result | |---|---| | `origin/main` `dc7d3fbb` | conflicts in both scripts, because #682 is squashed into main; with `--merge-base=f74f640c`, clean | | #682's last head `5afb24b9` (`fix/648-bootstrap-and-gate-order`, deleted on merge) | clean | | #747 `fix/681-validate-family-remedies` `58d6c50f` | clean | | #745 `fix/704-bootstrap-leg-moves` `f4935e11` | clean (a first draft conflicted twice with it, in `fetch_members`' docstring and where it adds `git_not_started`; both are moved out of its way) | | w764 `fix/764-bootstrap-notice-exit-texts` `271710ba` | clean | | w761 `bbb961f8`, w757 `6b5e9074`, w758 `b6e15591`, w759 `052410b6`, w760 `7569ffa8`, w777 `854f6ed9` | clean | | w762 `fix/762-validate-family-remedies-texts` `1038d7fa` (local) | **conflicts, three regions in `validate-family.py`**: the EXIT CODES sentence (each adds its exit-1 findings), the end of `_check_members`' docstring (each adds a paragraph), and the `_check_member_checkout` call (this adds `set(gitlinks) if refused else unregistered`, w762 adds `stop`). Each resolves by keeping both: `_check_member_checkout(root, asked, report, set(gitlinks) if refused else unregistered, stop)` | The other `fix/*` branches that touch these files and conflict (`fix/539-...`, `fix/607-...`, `fix/610-...`, `fix/bootstrap-print-name-escaped`, `fix/family-add-ghost-section`, `fix/family-bump-remove-rollback`, `fix/pin-files-not-a-list`, `fix/validate-family-member-duplicate`, `fix/validate-family-unnamed-mounts`) are old branches whose PRs landed long ago; `fix/611-...` is clean. No `l3-*` branch touches them. - Values in `siblings.py`'s `submodule_urls` (lane 4's `fix/690-siblings-inputs`), which the issue leaves out. - A `submodule.<name>.active` that is no boolean makes git die (`bad boolean config value`); `init_stop` reads past it to the patterns. - A blob `.gitmodules` with a config syntax error: git reads the entries before the bad line; `gitmodules_sections` reads none, as before. 1. **The credential advice prints where the line says no credential would help.** Whenever git's update exits non-zero, `fetch_members` prints "If the members are PRIVATE, set SHAPE_LEGS_TOKEN ... then re-run", and then, for a reason found in the holder's records (this PR's `.gitmodules` value, and #648's unmerged file, missing gitlink, missing section, missing url and stopped init), the line "NOT CHECKED OUT, and not for want of a credential". The first sends a person to a token that cannot help. Measured in every refused state above. 2. **`_sections_listed` reads a key with no value as the empty string.** In HEAD's file, Nova's `path` with no value reads as the path "", so `member-gitmodules-absent` also fires for Nova beside `gitmodules-value-refused`. Its line writes the same path the new finding writes back, so typing both is harmless, but it is a second finding for one fault (the `[path]` case of the written-back test pins it as it is). In the work tree's file the new blanking keeps it out. Review: lane 2's independent review (issuecomment-6236559204) and its re-verification of e1379a9, READY (issuecomment-6237963983). `233a334e` is its merge round after #820 (`e32ed55a`, both sides kept, `stop` only where `not refused`, `as_read` for `_check_removals_staged`) with a regression test; lane 2's re-check found it READY (#819 issuecomment-6244287619). CI (Brett Heap's ruling 27): H0 is the head 233a334 itself, which passed tests, tests-git-bash and SonarCloud; main has touched none of its files since its base b9b7069; and the local gate passed on its predicted squash tree caf6706 over main fe99b8b: the adopt_execute_refusals, adopt_filter_repo_failure, adopt_github_exits, adopt_late_failure, adopt_leg_keys, adopt_pins, adopt_split_branch, adopt_temp_work_roots, family_gitmodules_values, gh_guard, pin_annotated_tag, pin_source_top_level, template_values_single_pass, validate_family_without_git, repo_hygiene, agents_md_codes, cli_reference sets, 1025 passed. Landed on Brett Heap's word of 2026-10-17T21:47:28Z, in lane 5's session (posted on #473 as issuecomment-6241758069): "merge lane 5's v1.1.1 PRs once reviewed and green". Closes #768 Lane: openreposhape-5 (openRepoShape-5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>



Lane: openreposhape-5 (openRepoShape-5)
Claim: #645 (comment)
What changed. One session fixture in
tests/conftest.py,no_auto_maintenance(autouse), and one new test file,tests/test_no_auto_maintenance.py. No shipped file and no other test file changes. Four commits: the fix and its tests (e2110f8), a comment (accc7d2), and the fix round below (8cfc8eb, and 979bb39, which only shares a test helper).The fixture sets
gc.auto=0andmaintenance.auto=falsefor every git the session starts, in two scopes:GIT_CONFIG_COUNT), appended after the entries the environment already holds (the Windows job holds two). Every git the suite starts, every git a tool under test starts, and every git agit submodulestarts inherits it. It outranks every config file.GIT_CONFIG_GLOBAL, in the basetemp). It includes the global config git would read without it (the fileGIT_CONFIG_GLOBALalready named, or$XDG_CONFIG_HOME/git/configand then~/.gitconfig) and sets the two keys after it. This one is for thereceive-packa push starts in a local remote: git starts it withGIT_CONFIG_COUNTandGIT_CONFIG_PARAMETERSremoved, so command scope never reaches it. WhereHOMEis unset the~/paths are left out, and an emptyGIT_CONFIG_GLOBALincludes nothing (fix round, below).A comment beside #440's note in
clone_just_pushedsays why, names both sites, and says what is left.Why.
shutil.copytreereads a repository'sobjects/one file at a time. The suite copies a repository a fixture has just committed in, fetched into or pushed to in 139 places. Eachgit commit,mergeandfetch, and thereceive-packin a remote after a push, runsgit maintenance run --autoas it finishes. From git's source:--detach, takesobjects/maintenance.lockand then detaches, so the command returns while it runs.gc, which does nothing below 6700 loose objects.geometric. Its repack runs when packs are out of geometric order, or once its estimate, taken fromobjects/17alone, is above 256 loose objects, so two loose objects there are enough (maintenance.geometric-repack.auto, 100, rounded up). It deletes the loose objects it packed.So on a current git, a copy taken just after a commit can find a loose object gone. It happened twice on the Linux runner:
holder(tests/test_family.py:129), intest_family_member_name.py(lane 1's report on docs/release-notes/v1.0.0.md: the v1.0.0 release notes, a draft for Brett to amend before he tags, saying what 1.0 contains and promises, how to pin it by commit, what changed on the way grouped by the five tracking areas, and every known limit as the input that triggers it and what the person sees (#472) #473);the_standard_moved(tests/test_doctor_pin_restore_merges.py:99), a copy of the session'sstandardwith its.git, intest_copies_neither_side_edited_are_taken_whole_with_the_pinon setup-project.py: step (3) refuses a parent nobody can write in and a drive that is not there, step (4) verifies a chain from--pin-source,--pinno longer takes a flag as its value, and step (5) says a printed plan was refused (#608) #623's run 38895011713:shutil.Error ... standard5/openRepoShape/.git/objects/1e ... [Errno 2](lane 2's docs/release-notes/v1.0.0.md: the v1.0.0 release notes, a draft for Brett to amend before he tags, saying what 1.0 contains and promises, how to pin it by commit, what changed on the way grouped by the five tracking areas, and every known limit as the input that triggers it and what the person sees (#472) #473 comment 6215422231). The same function copiesmovedat :102 just after a commit in it.Both are covered by the same session setting: the commits before each copy run under
os.environ(commit_allpasses{**os.environ, ...}).A listing of
.git/objectstaken before and after such a repack also differs: lane 3'sobjects changedintests/test_family_bump_remove_rollback.py(#736's Linux run 38981191418), 37 loose objects packed into two. The same setting covers it (i645 on #645).Decisions and evidence.
Option A (the suite's environment), not B or C.
scaffold-project.py --local-remote-dirmakes and pushes into before a test sees them. A config written after that first push is too late.objects/pack/before the new pack lands, and then a loose-object directory after it was emptied. That copy succeeds and is missing objects.Two scopes, not one. Probe on git 2.43, pushing into a local bare remote with
receive.denyDeletes=truegiven each way:GIT_CONFIG_COUNT, the branch was deleted;-c, deleted;file://URL withGIT_CONFIG_COUNT, deleted;GIT_CONFIG_GLOBAL,remote: error: denying ref deletion(refused).So command scope alone misses the remotes, and ten copies are of bare remotes. A global file alone would be dropped wherever a test gives git a global file of its own (14 test files set
GIT_CONFIG_GLOBAL, five more throughexcluding()). No test file replaces both scopes and also copies a repository. rv645, the reviewer, repeated thereceive.denyDeletesprobe and got the same answers, and instrumented the git calls of the batches it finished before the checkpoint: the only receive-packs that ran without both settings were intest_family_add_excludes.pyandtest_adopt_clone_template.py, neither of which copies a remote.Which verbs start auto maintenance (git 2.43 trace2):
commit,fetch,pullandsubmodule update --remote(its fetch in the submodule) startgit maintenance run --auto. A push'sreceive-packstartsgit gc --auto --quiet.clone,submodule addandsubmodule update --initstart none.Behaviour no test checks is unchanged. The person's own global config is still read: it is included, and it comes first. A test of that is part of the new file. The tests that read or set a global config (
test_git_helper_identity.py,test_adopt_git_reads.py,test_tracking_branch_ref.pyand others) set their own, which replaces the session's.Fix round (8cfc8eb), on rv645's review (no MUST; two SHOULD, five NIT).
HOMEunset every git was fatal. The file's include of~/.config/git/configand~/.gitconfigmade git exit 128,error: could not expand include path '~/.config/git/config', in every command the session started. The estate runs withHOMEset, so nothing failed here; anenv -ior hermetic runner would have lost the whole suite.session_global_confignow writes a~/path only wherehome_is_known()(HOMEis set, oros.name == "nt", where Git for Windows finds a home itself), and writes no[include]header when nothing is included. AnXDG_CONFIG_HOMEthat is set is still included, because it needs noHOME.The two
test_a_persons_own_global_config...cases skip whereHOMEwas unset. They skip on the environment and not on the file's bytes, so a fixture that dropped its includes still fails them (probe M5 below).A new test,
test_where_home_is_unset_the_file_is_one_git_can_read(with and withoutXDG_CONFIG_HOME), reads the file back throughgit config --global --includes. It is skipped on Windows. It and the empty-value test share one helper,probe_read_from_the_session_file, so the new tests do not repeat a body for SonarCloud's duplication gate.Measured under
env -u HOME -u XDG_CONFIG_HOME,run9.sh, git 2.43.0:tests/test_clone_just_pushed.pytests/test_no_auto_maintenance.pytests/test_family_member_name.pyThe 2 skips are the two
test_a_persons_own_global_config...cases.Where
HOMEis set and the file is written as before, nothing changes: across 27 combinations ofHOME(set, unset, empty),XDG_CONFIG_HOME(set, unset, empty) andGIT_CONFIG_GLOBAL(unset, a file, empty), the file's bytes are the same as accc7d2's in the 12 whereGIT_CONFIG_GLOBALnames a file orHOMEis set withGIT_CONFIG_GLOBALunset, and differ in the 15 whereHOMEis not set orGIT_CONFIG_GLOBALis empty.scaffoldedandfamilywould have been built before the setting existed. A new test,test_the_settings_are_in_place_before_a_session_fixture_builds_a_repo, takes a session fixture as its FIRST argument, which readsGIT_CONFIG_GLOBALandGIT_CONFIG_COUNTas that fixture sees them, and compares them with what the test sees and with the fixture's file. Pytest builds the higher scope first, and then an autouse fixture, so the fixture is only in place early enough if it is session scope and autouse.core.hooksPath, which hid the remote's ownpost-receivehook (FileNotFoundErroronread-by-receive-pack). The test now setscore.hooksPathto the remote's ownhooksdirectory in the remote's local config, which outranks a global one. Measured withGIT_CONFIG_GLOBALnaming a file that holdscore.hooksPath: before,1 failed, 4 passed, 2 skipped; after,9 passed, 2 skipped(11 tests).GIT_CONFIG_GLOBAL=was read as unset. Git 2.43 reads it as "no global config" (checked here: withHOMEholding a.gitconfigprobe,GIT_CONFIG_GLOBAL=does not read it). The fixture now includes nothing for an empty value:if named is not None: included = [...] if named else []. A new test,test_an_empty_GIT_CONFIG_GLOBAL_is_no_global_config_and_includes_nothing, fails on the oldif named:.test_family_dry_run.py's ownNO_MAINTENANCEis redundant now. It lives in another test file that this PR leaves alone, and it is harmless.git config --globalread without--includessees only the session's two keys. The only reader issetup-project.py:1105, behindos.name == "nt", and no test reads it._credential_helper_configureduses--get-allwith no scope, so it still sees the person's config. The conftest comment already says so.Tests (
run9.sh: gh stub first on PATH, identity blanked; git 2.43.0, Python 3.12, pytest 9.1.1).tests/test_no_auto_maintenance.py: 11 passed (7 from the first commit, 4 new: theHOMEtest twice, the empty value, the scope).tests/test_no_auto_maintenance.py,tests/test_clone_just_pushed.py,tests/test_repo_hygiene.pyandtests/test_git_helper_identity.pytogether, on 979bb39: 105 passed. Withtests/test_family_dry_run.pyadded, on 8cfc8eb: 166 passed.tests/test_adopt_git_reads.py,tests/test_tracking_branch_ref.py,tests/test_adopt_late_failure.py,tests/test_adopt_github_exits.pyandtests/test_family_add_excludes.py(the tests that read or set a global config or a hooks path): 314 passed.HOMEis set andGIT_CONFIG_GLOBALis not empty (the byte comparison above), so these were not run again:tests/test_family*.py(20 files): 823 passed, 19 skipped.tests/test_adopt*.py(46 files): 3463 passed, 2 skipped.tests/test_shape_doctor.pyandtests/test_doctor*.py(12 files): 741 passed, 3 skipped.tests/test_scaffold*.py: 634 passed.tests/test_update_shape*.pyandtests/test_bump_leg*.py(24 files): 857 passed, 4 skipped.test_bootstrap_after_bump.py,test_non_list_project_fields.py,test_pin_source_env_collisions.py,test_pin_source_unreadable.py,test_shape_pin_files_not_a_list.py,test_validator_child_encoding.py,test_siblings_family_name.py,test_bootstrap_spelling.py,test_non_utf8_git_reads.py), withtest_repo_hygiene.py,test_git_helper_identity.py,test_clone_just_pushed.pyand the new file: 924 passed.test_no_auto_maintenance.py,test_git_helper_identity.py,test_repo_hygiene.py,test_clone_just_pushed.pyandtest_family_member_name.py, 114 passed.Reproductions.
The mechanism, on git 2.43. A repository of 300 files plus two loose objects named
17…, committed undergc.auto=1, then copied at once withshutil.copytree. This is what git 2.54 does to such a repository by default. With the base's settings, 10 of 10 copies failed, then 9 of 10 ([Errno 2]on a loose object directory or file). With the fix's two entries appended after it, 0 of 10 failed.The emulation of the race (rv645's recipe, run on 8cfc8eb: a global
gc.auto=1that includes the person's~/.gitconfig, plus acore.hooksPathpre-commit hook that leaves two17…objects and dropsgc.log, so git 2.43 repacks in the background after each commit;GIT_TRACE2_EVENTover the whole run). Base figures are rv645's, on a524542; head figures are the fix round's own, on 8cfc8eb:test_doctor_pin_restore_merges.pymaintenance run --auto, 77gc --autogc --auto(fromreceive-pack, inert undergc.auto=0)test_family_member_name.pymaintenance run --auto, 29gc --autogc --auto(inert)On the base, rv645 reproduced the symptom itself:
shutil.Error ... moved/openRepoShape/.git/objects/pack/tmp_rev_..., atthe_standard_moved's copy ofmoved(tests/test_doctor_pin_restore_merges.py:102). On head, 0 maintenance runs and 0 repacks in both files.The first writer's runs on the issue's file (
test_family_member_name.py, base then head, 19 maintenance and 15 repacks against 0 and 0) agree.Mutation probes. Each reverts one hunk of the fix in a throwaway clone; the new file then fails. First round (against the first seven tests):
test_a_commit_a_fetch_and_a_merge_start_no_maintenance[a-tests-own-global]maintenance.autogc.autoFix round (against the eleven tests):
no_auto_maintenanceat function scopehome_is_known()always truetest_where_home_is_unset_the_file_is_one_git_can_read, both variantsHOMExdgvariantif named:)test_a_persons_own_global_config...cases, and thexdgvariant of theHOMEtestcore.hooksPath, under a globalcore.hooksPathS2 is caught by tests that ask for the setting by itself; the scope test does not catch it, because earlier tests in the file build the fixture on request. K1 only fails where a global
core.hooksPathis set, as NIT 1 says.On the base itself the new file does not import (
cannot import name 'NO_AUTO_MAINTENANCE').Merge checks (
git merge-tree --write-treeon 979bb39): clean against origin/main 2fac979 and against lane 5'sfix/662-insteadof-helper(4149cbb; #662 is stacked on accc7d2 and also editstests/conftest.py: the fix round's hunks are insession_global_config, its helperhome_is_known, and the comment above them). Earlier, on e2110f8 and accc7d2: clean against origin/main d4a7fab and c6122e8, #621, #622, #630, #631, #634, #635, #636, #637, #638 and #639, and the sibling branches for #646, #647, #648 and #649.What it leaves.
GIT_CONFIG_COUNToverwritten, not appended to, together with a test's ownGIT_CONFIG_GLOBAL, or an environment built from nothing), and a push made under a test's own global file. No repository the suite copies is made or pushed to that way today.GIT_CONFIG_GLOBAL. Itsreceive-packrunsgc --auto, which does not detach below 6700 loose objects.clone_just_pushedkeeps reading through git. A test with its own global file still needs it.test_family_dry_run.py's ownNO_MAINTENANCEis now redundant, and is left alone (NIT 3).HOMEtest is skipped on Windows), and a git newer than 2.43.Copies found (
git grep -n 'shutil.copytree(' -- tests, 152 calls):.git/modules):conftest.py1,test_family.py18,test_shape_doctor.py32,test_doctor_legs_members_next.py8, thetest_update_shape*files 30, and 38 more in 28 files. Four of them copy a subtree that holds no store:standard / nametwice, and a member's work tree twice.test_bump_leg_paste_lines.py:95,test_doctor_unfinished_merge.py:166,test_family_add_member_id.py:239,test_family_add_refusals.py:487,:788and:1983,test_family_dry_run.py:1446,test_family_tracking_branch.py:279,test_update_shape_landing.py:156andtest_update_shape_next_quoting.py:116..gitdirectory:test_adopt_clone_template.py:190andtest_scaffold_mount_ignored.py:381..gitignored.test_adopt_plan.py:449copies a clone named bySHAPE_MEDXEHR_CLONE. It is outside the suite and skipped without the variable.No shipped script copies a repository.
Checklist candidates (rv645's; the coordinator places them):
tests/test_family_refuse_up_front.py:75-96runs in a throwaway directory before any session setting. It is harmless today.GIT_CONFIG_COUNT(14 files) lose command scope and rely on the global file alone. That holds today, and nothing guards it.Closes #645
🤖 Generated with Claude Code
Summary by Sourcery
Prevent Git’s automatic maintenance from corrupting repository copies made during the test suite.
Bug Fixes:
Enhancements:
Tests: