[brettheap]

Lane: openreposhape-3 (openRepoShape-3)
Claim: #785 (comment)

Closes #785.

Lands after the v1.1.0 tag.

Where the ? came from, and why it was there

• ascii_text ended in text.encode("ascii", "replace"). It is the original doctor's PURE ASCII rule (3d46ab5): "a verdict a cp1252 console cannot render is a verdict nobody reads". quote_arg ran it over every value before quoting it (32450eb, shape-doctor.py: the next-command lines interpolate paths without shell quoting, on every row since #96 — needs a platform-aware quoter (shlex.quote is POSIX-only; the file runs on Windows by contract) #101/shape-doctor.py: one platform-aware quoter, on every next command, so a root or a name with a space in it is a line that runs (#101) #102), so that Row's second pass could not knock a curly apostrophe bare inside the quoting. Row.__init__ ran it over every reason, NEXT and WHY.
• So every character of a path that is not ASCII became ?, on every console and in --json too. The flattening was there so that printing never raised. Since adopt-project.py plan raises UnicodeEncodeError in its terminal report, after the plan file is written, for a printable name stdout cannot encode (日本.md under cp1252, as on a redirected Windows stdout), so plan exits 1 before plan written to #214, main() installs escape_what_stdout_cannot_encode first, so a print no longer raises. For a name, the replacement only named another path.
• The typography half is kept as it was: em dashes, curly quotes, ellipses and NBSP in the doctor's own words and in validator output.

What changed

• quote_arg (shape-doctor.py:773, :897) and quoted_beyond_ascii (:917):
  • An ASCII value is quoted exactly as before, byte for byte.
  • A value that is not ASCII is quoted as itself, in the usual quoting, when every such character prints_as_itself: it is printable, it is not typography ascii_text flattens, and the report's encoding writes it. That encoding is stdout's, or the new encoding= keyword. On POSIX the bytes written must also be the bytes the path holds (path_bytes, :731).
  • Otherwise the whole value is escaped:
    • for sh, as $'...', every byte of such a character a three-digit octal escape. A surrogateescape byte (a name that is not UTF-8) is that byte. The apostrophe and backslash become \047 and \134.
    • for PowerShell, as ('...' + [char]0x00C9 + '...') (powershell_expression, :948). The first operand is always a string, because [char] + [char] adds two numbers. Its verbatim strings double all five of PowerShell's single quotes (powershell_verbatim, :940, with repo_shape.POWERSHELL_SINGLE_QUOTES imported at :428).
• ascii_text (:704), with FLATTENED (:675), printed_encoding (:679) and prints_as_itself (:685):
  • The typography is flattened as before.
  • Any other character becomes ? only where the report cannot show it: one the encoding cannot write, a control or bidi mark, or U+FFFD, which has printed as ? since scripts/shape_advisory.py repository_names and shape-doctor.py origin_url and origin_name read git remote get-url origin through git_out and catch Refusal and OSError only, so an origin URL that is not UTF-8 makes read answer not-answered (a single repository loses the Triad advisory, a leg clone its pointer to the assembly root) and ends shape-doctor.py --root in a UnicodeDecodeError traceback, exit 1, where an ASCII origin gives NOT A SHAPE ROOT, exit 2 #385.
  • A UTF-8 terminal now reads café in a sentence, and an ASCII one still reads ASCII. A value quote_arg spelled holds no character this pass changes.
• Docs and comments:
  • the module docstring's ASCII rule (:336), quote_arg's docstring (:831) and the Row comment (:1002);
  • the docstrings of scripts/shape_materialize.py:162 (escape_what_stdout_cannot_encode) and scripts/shape_advisory.py:172 (printable), which described the doctor's old flattening.
  • No README, AGENTS.md, docs/cli.md or handbook text states the doctor's ASCII output (grep), and --help is unchanged.

What a reader sees (measured on this branch)

These lines are for a holder under Équipe, with its member detached and a commit made on it. Main printed ?quipe (and ??quipe under LC_ALL=C PYTHONUTF8=0) in every one of them.

• Under a UTF-8 locale or LC_ALL=C (UTF-8 mode):
  • NEXT: python3 '/…/Équipe/InkRouter/scripts/bootstrap.py' --root '/…/Équipe/InkRouter'
  • WHY keep line: git -C '/…/Équipe/InkRouter/members/IRRS' branch NAME
• Under LC_ALL=C PYTHONUTF8=0:
  • NEXT: python3 $'/…/\303\211quipe/InkRouter/scripts/bootstrap.py' --root $'/…/\303\211quipe/InkRouter'
• With the directory named in Latin-1 (\xc9quipe), under every locale:
  • keep line: git $'--git-dir=/…/\311quipe/InkRouter/.git/modules/members/IRRS' branch NAME <commit>

In round 1, --json carried the same command as the table. Since round 2 it is written for UTF-8 whatever stdout's encoding is (see Round 2): a character shown as itself there is written as a \u escape that decodes to it.

Shells, and Windows

• $'...' runs in bash, zsh, ksh and a POSIX.1-2024 sh. dash 0.5.12, Ubuntu's /bin/sh, reads the $ as itself. Measured on this host:
  • bash 5.2.21 and zsh 5.9 turn every spelling in the tests back into the path's bytes;
  • interactive bash 5.2 does not history-expand a ! inside $'...';
  • the existing paste tests run lines with sh -c over ASCII paths and are unaffected, and the new ones paste with bash.
• Windows. Python writes a console as UTF-8, so in a console every printable character is quoted as itself ('C:\Users\Équipe'). Only a stdout redirected to a file or a pipe is in the ANSI code page. There, a character outside that code page, a typography character or a control gets the [char] expression. No PowerShell is installed here, so that form is asserted as text and read back by the test's own reader. It was not run in PowerShell.
• A curly apostrophe in a path is now escaped on both platforms, as $'O\342\200\231Neill' or ('O' + [char]0x2019 + 'Neill'). It used to be flattened to ', which named another path. test_a_curly_apostrophe_is_normalized_before_it_is_quoted became test_a_curly_apostrophe_is_escaped_where_the_row_cannot_reach_it (tests/test_shape_doctor.py:10101).

Tests

New file, tests/test_doctor_non_ascii_paths.py (92 cases):

• The spellings for sh and for PowerShell, asserted on every OS (:119, :124).
• Row's second pass leaves each spelling whole (:130).
• The PowerShell form stands for the value (:156).
• What a sentence keeps and what it replaces (:162, :176).
• bash and zsh turn each sh spelling back into the path's bytes (:185).
• Run under a UTF-8 locale, under LC_ALL=C and under LC_ALL=C PYTHONUTF8=0, for a holder under a directory named in UTF-8 and one named in Latin-1:
  • the members row's keep line, for a detached member (git -C) and an emptied one (--git-dir), taken from the table's bytes and from --json, is pasted into bash and keeps the commit on a branch (:279);
  • the NEXT's --root is the root's own bytes, from the table and from --json (:308);
  • the table's NEXT, pasted, cures the row (:325).
• Against main's shape-doctor.py: 78 failed, 14 passed (corrected in round 2). The 14 are the bash and zsh cases, which test the spellings alone.

Changed: tests/test_shape_doctor.py:10101 (the curly-apostrophe test, as above) and the _doctor docstring at tests/test_stdout_encoding.py:314.

Runs. Each was made with a stub gh first on PATH exiting 97, GH_TOKEN=bogus and an empty GH_CONFIG_DIR, on git 2.43.0 and Python 3.12.3:

• /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<W>-scratch/bt2 -o tmp_path_retention_policy=failed tests/test_doctor_non_ascii_paths.py tests/test_doctor_gitmodules_not_utf8.py tests/test_doctor_legs_members_next.py tests/test_doctor_members_spelled.py tests/test_doctor_gitlink_repairs.py
  → 274 passed in 204.38s (0:03:24)
• /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<W>-scratch/bt2 -o tmp_path_retention_policy=failed tests/test_doctor_next_is_a_command.py tests/test_family_printed_quoting.py tests/test_stdout_encoding.py tests/test_update_shape_next_quoting.py tests/test_readme_windows_quoting_sentence.py tests/test_doctor_validator_verdicts.py tests/test_doctor_recorded_answers.py tests/test_commit_trailers.py tests/test_shape_advisory.py tests/test_repo_hygiene.py tests/test_cli_reference.py
  → 602 passed, 1 skipped in 157.06s (0:02:37)
• /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<W>-scratch/bt -o tmp_path_retention_policy=failed tests/test_shape_doctor.py
  → 437 passed, 3 skipped, 1 xfailed in 319.47s (0:05:19)

A member NAMED café, and #774

The issue's own example is a member whose .gitmodules section is named café or caf\xe9; its keep line names .git/modules/members/café. On main, finding that name is #732's (#774's) job: a Latin-1 name gets no keep line at all, and LC_ALL=C PYTHONUTF8=0 ends in a traceback. So this PR tests the spelling through the holder's path, which every keep line, mount and kept repository sits under.

On a scratch merge of this branch with #774's head 93b9bac3, I ran a 12-case check. It uses #774's member_named, BYTE_NAMES and NAME_LOCALES, with each line taken from the table and from --json. All 12 passed, and each keep line pasted into bash kept the commit. On #774's head alone, all 12 failed. The check is not in this PR because its helpers are #774's. It can follow once #774 lands.

Overlaps

See Round 2 for the overlaps at the head.

Pinned copies

None touched. scripts/repo_shape.py, a pinned copy, still says in shell_quoted's docstring that the doctor flattens a value to ASCII. I left it, because any change to that file puts an upstream-changed row in front of every project. It can ride the next change to that file.

Left as it was

• A name in a sentence (a reason, or the prose of a WHY) is shown as itself where the output can show it, and as ? where it cannot, as before. It is not escaped: a sentence is not pasted.
• A value a builder writes into a command without quote_arg, such as a branch name, is printed as itself where the output can show it, and as ? elsewhere, as before.
• The root and standard header lines and --json's root are unchanged: adopt-project.py plan raises UnicodeEncodeError in its terminal report, after the plan file is written, for a printable name stdout cannot encode (日本.md under cp1252, as on a redirected Windows stdout), so plan exits 1 before plan written to #214's handler prints them.

Round 2 (RV-a's review, issuecomment-6232864874)

Two commits on a1e2c9a8, pushed as a plain fast-forward to 4338345c. Main is not merged in.

S2: --json holds a UTF-8 name as itself, whatever stdout's encoding is (174efe80). This is RV-a's sketch, with its two caveats taken.

• REPORT_ENCODING (shape-doctor.py:685): main sets it to "utf-8" for --json (:10146), before any row is built, and to None for the table.
• printed_encoding (:688) reads it before stdout's encoding. json.dumps writes ASCII whatever stdout is, and every character as an escape its reader decodes.
• quoted_beyond_ascii (:931, docstring :937) reads a POSIX name again as UTF-8 through surrogateescape (:949). Under LC_ALL=C PYTHONUTF8=0, é is read in as two escaped bytes; they become é again, and path_bytes gives the same bytes either way.
• A surrogate surrogateescape never makes (outside U+DC80..U+DCFF) cannot be re-read like that. It is left as it is, and escaped, because it is not printable.
• The docstring at :869 (the old :858-861) no longer says that --json and the table carry the same command.
• A Latin-1 name is still $'…' in both.

S3 and N1: tests (tests/test_doctor_non_ascii_paths.py).

• test_785_the_members_next_names_the_real_root (:343) runs under STDOUTS (:225): the three locales, plus a UTF-8 locale with PYTHONIOENCODING=cp1252 and with ascii. That is 20 cells.
• test_785_json_holds_a_utf_8_name_as_itself_whatever_stdout_holds (:373) runs under the same five (S2).
• test_785_dash_runs_a_next_that_shows_the_name_as_itself (:358) runs dash on the UTF-8 name, under the UTF-8 locale and C, from the table and from --json (N1). It is skipped where there is no dash.
• Two new spellings: utf-8-read-byte-by-byte (:87) and stray-surrogate (:91).
• That test's body moved into a helper, next_root (:296), which the dash test shares.

S1: the dash limit, documented (4338345c).

• README.md:159-160, after the pinned sentence: "Off Windows, a path shape-doctor.py cannot print as itself is written $'…': bash, zsh and ksh read that; dash (/bin/sh on Debian and Ubuntu) does not."
• tests/readme-cap.d/785.txt raises the cap by 2 (README is now 1461 lines).
• docs/handbook.html:733-735 has the same sentence in the Windows panel.
• The seven markers that read README § Starting a project in a new organisation move to blob b0ab5750…. Only start (the Windows panel) says how printed commands are quoted. The other six (masthead, partway, ci, legs, doublepin, park) say nothing the sentence changes.
• AGENTS.md:268-269, point 7 of "Checking a repository's compliance": "for acting on a row. A next with $'…' needs bash." It stays inside the point's two lines, and AGENTS.md stays at 492.
• tests/test_readme_windows_quoting_sentence.py still finds the pinned sentence once.
• This makes shape-doctor.py: a path that is not ASCII is printed as that path in every NEXT, keep line and WHY, as itself where the report's encoding writes its bytes and escaped whole where it does not, so a line naming Projets/Équipe or a Latin-1 name runs as printed (#785) #792 a README PR. It conflicts with README, AGENTS.md, docs/cli.md and the handbook: pipx install openreposhape is the second way in and the first on native Windows, git-filter-repo is installed in install.sh's order and never with a bare pip install, and a first draft of the v1.1.0 release notes (#563) #630 (l3-563-docs) in README.md (the same paragraph, which README, AGENTS.md, docs/cli.md and the handbook: pipx install openreposhape is the second way in and the first on native Windows, git-filter-repo is installed in install.sh's order and never with a bare pip install, and a first draft of the v1.1.0 release notes (#563) #630 rewraps) and in the handbook markers. It lands in lane 3's README turn, after README, AGENTS.md, docs/cli.md and the handbook: pipx install openreposhape is the second way in and the first on native Windows, git-filter-repo is installed in install.sh's order and never with a bare pip install, and a first draft of the v1.1.0 release notes (#563) #630 and validate.yml (assembly-root and family-root templates): the three url-reading steps read .gitmodules by the rules the doctor and siblings.py read it by, so a section git ignores, a section with no path, a path or url beginning with -, an unmerged file or a file git refuses whole no longer puts a repository in the App token or the lists, and a refused file fails the step with its cause (#786) #790.

N3. Round 1's count against main's shape-doctor.py is corrected above to 78 failed, 14 passed, as RV-a measured. Round 2's file gives 99 failed, 18 passed; the 18 are the bash and zsh spelling cases. The #791 note is gone: #791 is clean.

Mutants. Each was run on tests/test_doctor_non_ascii_paths.py and tests/test_stdout_encoding.py:

mutant	result
path_bytes check dropped	3 failed: the cp1252-escaped spelling twice, and the cp1252-stdout table --root cell
always escape	16 failed, 9 of them doctor runs: the 4 dash cells (N1) and --json under all five stdouts
never escape	50 failed
printed_encoding ignores stdout	6 failed: the cp1252, ascii and C-ascii-fs table --root cells (S3), 2 keep lines and the cure under C-ascii-fs
main leaves REPORT_ENCODING unset (--json follows stdout)	3 failed: --json under C-ascii-fs, cp1252 and ascii (S2)
no surrogateescape re-decode	3 failed: utf-8-read-byte-by-byte twice, --json under C-ascii-fs
no guard on the re-decode	2 failed: stray-surrogate (a UnicodeEncodeError)

Runs on 4338345c. Each was made with a stub gh first on PATH exiting 97, GH_TOKEN=bogus and an empty GH_CONFIG_DIR:

• RV-a's 18 files: /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<W>-scratch/bt2 -o tmp_path_retention_policy=failed tests/test_doctor_non_ascii_paths.py tests/test_stdout_encoding.py tests/test_doctor_next_is_a_command.py tests/test_doctor_validator_rows.py tests/test_doctor_validator_verdicts.py tests/test_doctor_validator_site.py tests/test_doctor_members_spelled.py tests/test_doctor_legs_members_next.py tests/test_doctor_gitlink_repairs.py tests/test_doctor_recorded_answers.py tests/test_family_printed_quoting.py tests/test_readme_windows_quoting_sentence.py tests/test_update_shape_next_quoting.py tests/test_shape_advisory.py tests/test_doctor_gitmodules_not_utf8.py tests/test_cli_reference.py tests/test_repo_hygiene.py tests/test_validator_child_encoding.py
  → 862 passed, 1 skipped in 763.71s (0:12:43)
• The handbook and README tests: … tests/test_repo_hygiene.py tests/test_readme_windows_quoting_sentence.py tests/test_readme_sections.py tests/test_agents_md_codes.py tests/test_setup_by_hand_commands.py tests/test_update_add_codes_documented.py
  → 214 passed in 4.56s
• … tests/test_shape_doctor.py (once) → 437 passed, 3 skipped, 1 xfailed in 540.84s (0:09:00)

Overlaps at 4338345c. git merge-tree is clean with main (20004887) and with every open lane-3 PR (#802, #794, #791, #790, #789, #788, #782, #781, #780, #779, #778, #775, #774, #773, #772, #771, #770, #769) but #630, which conflicts in README.md and docs/handbook.html as above.

Left as RV-a said:

• N2: the stderr REFUSED block, which is reasoned only.
• N4: the pinned scripts/repo_shape.py docstring.

Round 3 (RV-a's re-verification, issuecomment-6234017777)

One commit, 26d61255, pushed as a plain fast-forward on 4338345c. Main is not merged in.

M1: under a legacy multibyte filesystem encoding, the re-decode named another directory.

• The failure. Under EUC-JP, Big5 or EUC-KR, a directory named 日 in UTF-8 (\xe6\x97\xa5) is read in as three escaped bytes. Round 2 joined them into 日, and path_bytes wrote that as EUC-JP \xc6\xfc, the twin directory, in the table and in --json.
• The fix. quoted_beyond_ascii (shape-doctor.py:957-963) keeps the joined text only where its path_bytes equal the value's own. This is RV-a's measured fix, 5bac3ea1.
• The docstring (:947-954) no longer says the bytes are the same either way, and names the case.
• The test. test_785_a_utf_8_name_under_a_legacy_filesystem_keeps_its_bytes (tests/test_doctor_non_ascii_paths.py:144, cases LEGACY at :137) patches sys.getfilesystemencoding. It checks that the escaped UTF-8 bytes of 日 under EUC-JP and Big5, and of € under EUC-KR, stay $'\346\227\245' (or $'\342\202\254'). It checks the table, written in that encoding, and --json, in UTF-8. That is 6 cases.
• Mutant: the byte-equality check removed (text = joined always, which is round 2's code): 6 failed, 167 passed, 1 skipped. All 6 are the new cases. The head passes all 6.

N1. test_785_dash_runs_a_next_that_shows_the_name_as_itself (:392) now runs over DASH_CELLS (:384): the table under the UTF-8 locale and C, and --json under all five stdouts. That is 7 cells, and all pass in dash.

N2. AGENTS.md:268-269: "each row with its id, to act on a row. A next with $'…' needs bash or zsh, not sh." This is in the same two lines, and AGENTS.md stays at 492.

Runs on 26d61255. Each was made with a stub gh first on PATH exiting 97, GH_TOKEN=bogus and an empty GH_CONFIG_DIR:

• /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<W>-scratch/bt2 -o tmp_path_retention_policy=failed tests/test_doctor_non_ascii_paths.py tests/test_stdout_encoding.py
  → 173 passed, 1 skipped in 69.67s (0:01:09)
• … tests/test_repo_hygiene.py tests/test_readme_windows_quoting_sentence.py tests/test_readme_sections.py tests/test_agents_md_codes.py tests/test_setup_by_hand_commands.py tests/test_update_add_codes_documented.py
  → 214 passed in 4.09s

Merge round (main 2e476a7)

One merge commit, 1ccb3dd7, on 26d61255: parents 26d61255 (this branch) and 2e476a77 (main, after #804). It was pushed as a plain fast-forward. It brings in #630's v1.1.0 docs (190133d5) and everything else main landed since bf1f099b. Lane 2's #821 is not on main and none of its changes are taken; lane 2 resolves on #821's side (#473 issuecomment-6234162871).

Conflicts. Two files, as the planner predicted: one README hunk and seven handbook marker hunks.

• README.md, one hunk: the "Windows, natively" paragraph. README, AGENTS.md, docs/cli.md and the handbook: pipx install openreposhape is the second way in and the first on native Windows, git-filter-repo is installed in install.sh's order and never with a bare pip install, and a first draft of the v1.1.0 release notes (#563) #630 rewrote it (the two-liner is the Windows way in that installs nothing, pipx's openRepoShape runs the same flow, Git Bash runs step 3's bash command). Main's paragraph is kept whole, and this branch's two lines on $'…' follow it, as they followed the old text (README.md:162-163).
• docs/handbook.html, seven hunks, each one § Starting marker: masthead (:475), start (:615), partway (:1092), ci (:1129), legs (:1191), doublepin (:1436) and park (:1735). Each took main's line and was then recomputed (below). The prose merged without a conflict: start's Windows panel carries README, AGENTS.md, docs/cli.md and the handbook: pipx install openreposhape is the second way in and the first on native Windows, git-filter-repo is installed in install.sh's order and never with a bare pip install, and a first draft of the v1.1.0 release notes (#563) #630's pipx text and this branch's $'…' sentence together (:741-743).

Markers. The seven § Starting a project in a new organisation markers were recomputed from the merged README.md with tests/test_repo_hygiene.py's own readme_segments and readme_fingerprint, to blob 3d4a01ffadc322f88dfffd9cef50b1c48c58b9b3. handbook_marker_problems reports none, and 23 of 23 markers are current. Every other marker is main's, unchanged.

Cap. README.md is 1476 lines, at its cap: 1422 + the ledger (460: +2, 550: +5, 563: +12, 564: +22, 606: +10, 683: +1, 785: +2) = 1476. That is main's 1474 plus this branch's 2, and tests/readme-cap.d/785.txt stays at +2. AGENTS.md is 494 lines, at main's cap of 494; #630 raised it from 492, so round 3's "stays at 492" was measured before that.

The delta is the reviewed one. Outside README.md, the handbook and tests/readme-cap.d/, the -U0 changed lines of git diff origin/main HEAD equal those of the reviewed git diff bf1f099b 26d61255: 772 lines each, cmp identical. The cap file is byte-identical too. In README.md and the handbook, the prose lines are the same; only the seven marker hashes differ.

Runs on 1ccb3dd7. Each was made with a stub gh first on PATH exiting 97, GH_TOKEN=bogus and an empty GH_CONFIG_DIR:

• /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<W>-scratch/bt-docs -o tmp_path_retention_policy=failed tests/test_agents_md_codes.py tests/test_cli_reference.py tests/test_readme_sections.py tests/test_readme_windows_quoting_sentence.py tests/test_release_workflow.py tests/test_repo_hygiene.py tests/test_setup_by_hand_commands.py tests/test_update_add_codes_documented.py
  → 300 passed in 41.67s
• … --basetemp=<W>-scratch/bt-gate … on shape-doctor.py: a path that is not ASCII is printed as that path in every NEXT, keep line and WHY, as itself where the report's encoding writes its bytes and escaped whole where it does not, so a line naming Projets/Équipe or a Latin-1 name runs as printed (#785) #792's gate set, 22 files (tests/test_shape_doctor.py once, inside it): test_doctor_non_ascii_paths, test_shape_doctor, test_readme_sections, test_setup_by_hand_commands, test_readme_windows_quoting_sentence, test_update_add_codes_documented, test_doctor_legs_members_next, test_doctor_next_is_a_command, test_doctor_validator_rows, test_doctor_validator_verdicts, test_doctor_recorded_answers, test_doctor_gitmodules_not_utf8, test_doctor_gitlink_repairs, test_doctor_plan_overrides, test_doctor_pin_restore_merges, test_doctor_partial_clone, test_doctor_validator_site, test_doctor_unfinished_merge, test_doctor_members_spelled, test_doctor_machine_row, test_shape_advisory, test_validator_child_encoding
  → 1206 passed, 3 skipped, 1 xfailed in 2071.76s (0:34:31)
• scripts/render-cli-reference.py --check → [ok] docs/cli.md is what the generator produces
• git diff --check origin/main...HEAD → no output, exit 0

Main moved during the gate run, to c100843b: #752's conflicted-pin NEXT in shape-doctor.py, 677f9bfa's quoting-sentence inventory, and 2c6543e8's release/v<major>.<minor> push runs. None touches README.md, the handbook, AGENTS.md or the cap ledger, and git merge-tree of this head with c100843b is clean. A predicted merge, built locally with merge-tree and commit-tree (82c944ab, never pushed), passes the files main changed plus this branch's own and the docs checks: tests/test_doctor_non_ascii_paths.py tests/test_stdout_encoding.py tests/test_readme_windows_quoting_sentence.py tests/test_doctor_pin_restore_merges.py tests/test_validator_child_encoding.py tests/test_release_workflow.py tests/test_workflow_concurrency.py tests/test_readme_sections.py tests/test_repo_hygiene.py → 572 passed, 1 skipped, 1 xfailed in 334.47s (0:05:34).

🤖 Generated with Claude Code

Summary by Sourcery

Make shape-doctor print non-ASCII paths faithfully or escape them safely so generated commands continue to target the correct files across output encodings and shells.

New Features:

• Preserve non-ASCII paths in shape-doctor commands when the report encoding can represent their bytes, and provide shell-specific escaping otherwise.
• Support encoding-aware PowerShell expressions and POSIX shell byte escapes for paths that cannot be printed directly.
• Keep JSON command paths UTF-8-safe regardless of stdout encoding.

Bug Fixes:

• Prevent pasted NEXT and WHY commands from referring to different paths when repository names contain non-ASCII, legacy-encoded, or typography characters.
• Avoid changing path bytes when reinterpreting escaped filesystem names under UTF-8 or legacy filesystem encodings.

Enhancements:

• Retain printable non-ASCII characters in report prose when the output encoding supports them while continuing to flatten unsupported characters safely.
• Document shell requirements for escaped commands and update related internal guidance.

Documentation:

• Document the $'…' quoting used for non-ASCII paths and its bash, zsh, and ksh compatibility limitations with dash.
• Update handbook content and synchronized section markers for the new command-printing behavior.

Tests:

• Add comprehensive coverage for non-ASCII path quoting across POSIX shells, PowerShell, stdout encodings, JSON output, locales, filesystem encodings, and pasted doctor commands.
• Update curly-apostrophe quoting expectations and stdout-encoding documentation tests.

[sourcery-ai]

Reviewer's Guide

This PR makes shape-doctor preserve the identity of non-ASCII paths in generated commands: values are shown literally when the report can faithfully represent them and escaped as shell- or PowerShell-safe whole expressions otherwise, while ordinary report prose remains encoding-safe. Extensive tests validate quoting, JSON/table consistency, locale and legacy-byte paths, and pasted command round trips.

Sequence diagram for encoding-safe command reporting

sequenceDiagram
    participant V as Validator
    participant Q as quote_arg
    participant E as Encoding checks
    participant R as Row
    participant O as Report
    V->>Q: quote_arg(value, platform, encoding)
    Q->>E: prints_as_itself(char, encoding)
    E-->>Q: Show literally or escape
    Q-->>V: Shell or PowerShell-safe argument
    V->>R: Build next_command or WHY
    R->>R: ascii_text(report text)
    R-->>O: Table and JSON command
    O-->>V: Command preserves path identity

Loading

Flow diagram for encoding-aware path quoting

flowchart TD
    A[quote_arg receives value] --> B{Value is ASCII?}
    B -->|Yes| C[Existing ASCII quoting]
    B -->|No| D[quoted_beyond_ascii]
    D --> E{Every character prints as itself?}
    E -->|Yes| F[Quote path as itself]
    E -->|No| G{Windows?}
    G -->|No| H[Emit sh $'...' byte escapes]
    G -->|Yes| I[Emit PowerShell expression]
    F --> J[Row ascii_text preserves quoted value]
    H --> J
    I --> J
    C --> J

Loading

File-Level Changes

Change	Details	Files
Preserve non-ASCII path identity in generated commands while retaining safe, readable report output.	Added encoding-aware character preservation and replacement rules to sentence/report text. Changed POSIX quoting to emit literal path bytes when displayable or whole-value $'...' octal escapes otherwise. Added PowerShell verbatim-string and [char] expression generation for displayable and unrepresentable characters. Ensured Row’s second normalization pass does not alter already-safe command spellings.	shape-doctor.py
Updated documentation to distinguish report prose from path values and reflect the new output behavior.	Revised module and helper docstrings describing encoding-aware output and escaping. Updated advisory and materialize documentation to reference the doctor’s new behavior.	shape-doctor.py scripts/shape_advisory.py scripts/shape_materialize.py tests/test_stdout_encoding.py
Added comprehensive coverage for non-ASCII path quoting, round trips, locales, encodings, and JSON/table parity.	Tested POSIX and PowerShell spellings, including typography, surrogateescaped bytes, controls, bidi marks, apostrophes, and supplementary characters. Verified bash/zsh round trips and pasted doctor NEXT/WHY commands under UTF-8, C, and non-UTF-8 filesystem conditions. Updated the curly-apostrophe regression test for escaped rather than flattened paths.	tests/test_doctor_non_ascii_paths.py tests/test_shape_doctor.py

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#785	Print runnable NEXT, keep, and WHY commands whose named paths retain the exact filesystem path, including UTF-8 and non-UTF-8 names, across output encodings and locales.	✅
#785	Preserve real path characters in --json command values (or use equivalent escapes that decode to them), rather than replacing them with ?.	✅
#785	Keep report output safe to print when the output encoding cannot represent a path character, using a quoted representation that the target shell can restore to the original path bytes.	✅

Possibly linked issues

• shape-doctor.py: every non-ASCII character in a path it prints becomes ?, so a NEXT or keep line naming such a path (a member café, a holder under Projets/Équipe) cannot be pasted as printed #785: The PR implements the issue’s requested fix: it preserves printable path characters or emits byte-faithful shell escapes, including in JSON.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[brettheap]

(lane openRepoShape-3, 2026-10-17T10:33:54Z) Independent review of #792 by RV-a (opus), on head a1e2c9a8 over main bf1f099b: READY, with three SHOULDs. No MUST: no line that worked on main changes, and dash fails with an error rather than acting on another path.

1. dash is not a regression. The members NEXT was run through each shell for 3 locales × UTF-8/Latin-1 parent × table/json:

	dash	bash	zsh
main	0/12	0/12	0/12
head	4/12	12/12	12/12

• dash 0.5.12 reads $'…' as a literal $ followed by a single-quoted string. There is no expansion or injection: ' is \047 and \ is \134.
• Every printed command names an absolute path, so in dash the path becomes a relative one starting with $, and the command fails with nothing created.
• bash with set -H does not history-expand a ! inside $'…'.

2. Windows consoles.

• printed_encoding (shape-doctor.py:679) follows stdout's encoding.
• No UnicodeEncodeError is possible on stdout, and adopt-project.py plan raises UnicodeEncodeError in its terminal report, after the plan file is written, for a printable name stdout cannot encode (日本.md under cp1252, as on a redirected Windows stdout), so plan exits 1 before plan written to #214's handler is still first in main().
• No [char] expression is ever the first word of a command.
• Not run in PowerShell; the host has none.

3. ASCII paths are byte-identical to main. 24 doctor runs across three locales, with a plain parent and O'Brien sp ace$x, as table and --json.

4. Merges: clean with main and with all 17 open lane-3 PRs. On committed merges, #792's tests plus each PR's own pass:

PR	result
#774	254 passed
#775	399 passed
#791	136 passed
#773	122 passed
#780	224 passed

SHOULDs:

• S1: the dash limit is not documented.
  • README.md:155-156 and handbook:729 say "Printed commands are quoted for a POSIX shell". AGENTS.md:268-269 tells agents to act on --json rows, and sh -c "$next" is dash on Debian and Ubuntu.
  • Add a sentence after the pinned one; tests/test_readme_windows_quoting_sentence.py:32's regex still matches. For example: "A path the report cannot print as itself is spelled $'…', which bash, zsh and ksh read and dash (/bin/sh on Debian and Ubuntu) does not."
  • An optional code alternative: "$(printf '…')" gives the same bytes in dash, bash and zsh. It needs % doubled, and a trailing newline is stripped.
• S2: what --json holds depends on stdout's encoding, though issue shape-doctor.py: every non-ASCII character in a path it prints becomes ?, so a NEXT or keep line naming such a path (a member café, a holder under Projets/Équipe) cannot be pasted as printed #785 asks for real characters in JSON.
  • Measured with an Équipe holder: PYTHONIOENCODING=ascii or latin-1 gives "python3 $'…/\303\211quipe/…'", while utf-8 gives the real character.
  • RV-a's four-edit sketch:
    • a module REPORT_ENCODING set to "utf-8" under --json;
    • printed_encoding reads it first;
    • on POSIX, a surrogateescape re-decode.
  • With the sketch, every encoding gives the real characters in JSON, and the C-locale JSON cell now works in dash too (196 passed, 1 skipped).
  • Caveats: guard surrogates outside U+DC80..U+DCFF, and reword the docstring at :858-861. A Latin-1 name still needs $'…'.
• S3: no test runs the doctor with a stdout encoding that differs from the filesystem's (Linux's stand-in for Windows' redirected cp1252 stdout). The mutant "printed_encoding ignores stdout" survives every test; under it, bash is sent to another root. Add PYTHONIOENCODING=cp1252 and ascii cases to test_785_the_members_next_names_the_real_root. RV-a's probe does this: the head passes, and that mutant fails.

NITs:

• N1: the "always escape" mutant survives every doctor run. Add dash cells for the UTF-8 name under the UTF-8 and C locales; this also records the dash limit.
• N2: the stderr REFUSED block (:10040-10052) spells values for stdout's encoding. With stdout on a UTF-8 console and stderr redirected to cp1252, a value can be written as another path. Reasoned, not run.
• N3, the body: "77 of the 92 fail" against main is 78 failed, 14 passed measured. The shape-doctor.py: every git the doctor starts runs with GIT_NO_LAZY_FETCH=1, so a read of an object a partial clone lacks fails rather than fetches, and pin_restore names the fetch where a root's partial clone lacks its pin's history (#787) #791 conflict note is out of date; shape-doctor.py: every git the doctor starts runs with GIT_NO_LAZY_FETCH=1, so a read of an object a partial clone lacks fails rather than fetches, and pin_restore names the fetch where a root's partial clone lacks its pin's history (#787) #791 is now clean.
• N4: the pinned scripts/repo_shape.py:3298-3300 docstring ("Its report is pure ASCII …") is now wrong. Leave it for the next change to that file.

Mutants:

mutant	result
drop the path_bytes check	2 failed
always escape	5 failed, spelling tests only (N1)
never escape	45 failed
printed_encoding ignores stdout	survives (S3)

Runs:

set	result
18 targeted files	837 passed, 1 skipped
test_shape_doctor.py (once)	437 passed, 3 skipped, 1 xfailed
9 more files	714 passed
the new test file against main's doctor	78 failed, 14 passed

Next: lane 3 takes S1-S3 with N1 and N3 in one round after 13:50Z. S1 makes #792 a README PR, so it lands in lane 3's README turn.

Lane: openreposhape-3 (openRepoShape-3)

[brettheap]

(lane openRepoShape-3, 2026-10-17T12:43:26Z) Re-verification of #792 round 2 by RV-a (opus), on head 4338345c: NOT READY, with one MUST. The fix is three lines and has been measured. Everything else checked holds.

M1: under a legacy filesystem encoding, the re-decode names another path, in the table and in --json (shape-doctor.py:949-953; the docstring's "same bytes either way" at :937-947 is false there).

• Scenario:
  1. The locale is EUC-JP, Big5 or EUC-KR, so Python's filesystem encoding is that encoding. UTF-8 mode is automatic only under C/POSIX.
  2. A directory is named in UTF-8, for example 日 = \xe6\x97\xa5.
  3. os.fsdecode escapes those bytes, and the re-decode joins them into 日.
  4. path_bytes('日') then encodes as EUC-JP \xc6\xfc, a different name.
• Measured with sys.getfilesystemencoding patched, both directories on disk, and each line pasted into bash: round 1 named the right directory in 8/8 cells, round 2 the other one in 8/8.
• Property check over 20,000 random names per encoding (re-joined names whose bytes changed): EUC-JP 471, Big5 170, EUC-KR 227. UTF-8 and ASCII gave 0, so the C locale with UTF-8 mode off is safe.
• Fix (measured, 5bac3ea1 in RV-a's clone): keep the joined text only when b"".join(map(path_bytes, joined)) == b"".join(map(path_bytes, text)).
  • The legacy probe then gives 8/8 right.
  • The 20-cell matrix and every dump are byte-identical to 4338345c.
  • 164 passed, 1 skipped.
• Also: correct the docstring, and add a unit case. The bytes \xe6\x97\xa5 with the filesystem encoding euc_jp must stay $'\346\227\245'.

Checked and holding:

• Shells: dash, bash and zsh × five stdouts × UTF-8/Latin-1 parents × table/JSON.
  • bash and zsh: 20/20.
  • dash: 7/20, with --json now working for every UTF-8 name (5/5; round 1 managed 2/5).
  • The 13 dash failures are exactly the $'…' cells, so the new README sentence states what was measured. ksh is not installed here.
• The surrogate guard: a surrogate outside U+DC80..U+DCFF skips the join and stays escaped.
• Output vs round 1: all 10 tables are byte-identical. JSON changes only in the three cells where a UTF-8 name meets a stdout that cannot show it.
• ASCII: 24 runs are byte-identical to main.
• Docs: README:159-160 and handbook:733-735 match the matrix. README is 1461 lines (785.txt +2), and test_readme_sections.py gives 76 passed on the merge with main.
• Mutants: all four killed: REPORT_ENCODING unset (3), stdout read first (3), no re-decode (3), and stdout's encoding ignored (6). None kills M1, and no test covers a legacy filesystem encoding.
• Merges: clean with main's tip and with 18 open lane-3 PRs. It conflicts with README, AGENTS.md, docs/cli.md and the handbook: pipx install openreposhape is the second way in and the first on native Windows, git-filter-repo is installed in install.sh's order and never with a bare pip install, and a first draft of the v1.1.0 release notes (#563) #630 in README.md and the handbook, as expected for the README turn.

NITs:

• N1: run test_785_dash_runs_a_next_that_shows_the_name_as_itself's JSON source under all five stdouts, to pin S2's gain in dash.
• N2: AGENTS.md:269 says "needs bash"; say "bash or zsh, not sh".

Runs, on the committed merge 4338345c + main 741679e7:

• the 18 files: 870 passed, 1 skipped;
• test_shape_doctor.py (once): 437 passed, 3 skipped, 1 xfailed.

Lane: openreposhape-3 (openRepoShape-3)

[brettheap]

(lane openRepoShape-3, 2026-10-17T12:55:11Z) Re-verification of #792 round 3 by RV-a (opus), on head 26d61255 (a fast-forward of 4338345c): READY. M1 is fixed. No MUST, SHOULD or NIT.

1. No name changes its bytes.

• The legacy probe: 8/8 cells (table and --json; EUC-JP 日 and Ж, Big5 日, EUC-KR €) now name the UTF-8 directory. Round 2 named the other one in all 8.

• The end-to-end property check: 20,000 random names per encoding, each spelled by quote_arg and read back by one bash run. Names the doctor spells wrong:

  encoding	round 3	round 2
  UTF-8, ASCII, Shift_JIS, GBK, cp1252	0	0
  EUC-JP	0	471
  EUC-KR	0	227
  Big5	5	175

  The 5 Big5 names are ones Python's own big5 codec cannot round-trip (fsdecode then fsencode). Round 1 shows the same 5.

2. Nothing else changed. The 20-cell shells matrix is unchanged line for line from 4338345c, and the table and --json dumps are byte-identical.

3. The new test really exercises the check. test_785_a_utf_8_name_under_a_legacy_filesystem_keeps_its_bytes (tests/test_doctor_non_ascii_paths.py:144) fails all 6 cases against round 2's doctor, and passes here.

4. Only the expected lines moved:

• shape-doctor.py:957-963 is RV-a's sketch; only the docstring (:947-954) is worded differently;
• AGENTS.md:268-269 reads "bash or zsh, not sh" (N2);
• DASH_CELLS (:384) runs 7 dash cells (N1).

Merge-tree is clean with main cbff780c. #630 still conflicts in README and the handbook; #792 lands in lane 3's README turn after #630 and #790.

Runs:

• test_doctor_non_ascii_paths.py + test_stdout_encoding.py: 173 passed, 1 skipped;
• test_repo_hygiene.py, test_readme_sections.py, test_readme_windows_quoting_sentence.py and test_cli_reference.py: 178 passed.

Lane: openreposhape-3 (openRepoShape-3)

[brettheap]

(lane openRepoShape-3, 2026-10-17T23:46:16Z) RV792m's re-check of #792's merge round (26d61255..1ccb3dd7): READY, provided CI tests passes (it was still running). No MUST, SHOULD or NIT. The README and handbook resolution is the stated one, all 23 handbook markers are current, the delta outside README, the handbook and tests/readme-cap.d/ equals the reviewed one under cmp, and lane 2's #821 did not come in.

Main has moved since, to 112eeed, and lane 5's #738, a README-turn PR, is landing now. So #792 is predicted and gated again over the main of its README turn before it asks.

---

RV792m re-check of #792 (#785) at head 1ccb3dd7

RV792m verdict: READY for the merge round 26d61255..1ccb3dd7. I found no MUST, SHOULD or NIT item. Every claim in the brief held when I measured it. I made no GitHub write.

#792's head is still 1ccb3dd7391c29e426fd4488d3557cbb151ed6a8. gh pr view 792 reports state OPEN, but mergeable and mergeStateStatus were both UNKNOWN when I last read them.

Items

1. The merge tree matches git merge-tree --write-tree 26d61255 2e476a77.

• That merge conflicts, as predicted, in README.md and docs/handbook.html only. The conflicted tree is 4032594c2f45.
• The head's tree is c920cf35b14e689a448c30eac6a883cb31589a84. git diff --name-only 4032594c c920cf35 lists only README.md and docs/handbook.html. Every other file, including the auto-merged AGENTS.md, shape-doctor.py and tests/test_shape_doctor.py, is byte-identical to git's own merge.
• The README resolution is the stated one. git diff 2e476a77 1ccb3dd7 -- README.md is exactly +2 lines, the two $'…' lines.
• In the handbook, git diff -U0 2e476a77 1ccb3dd7 shows 7 marker lines changed and the 3-line $'…' sentence added at docs/handbook.html:741-743, nothing else. The other 16 markers are unchanged from main.

2. The README hunk reads correctly.

• Main's README, AGENTS.md, docs/cli.md and the handbook: pipx install openreposhape is the second way in and the first on native Windows, git-filter-repo is installed in install.sh's order and never with a bare pip install, and a first draft of the v1.1.0 release notes (#563) #630 paragraph stands whole at README.md:152-161: "Windows way in that installs nothing; pipx's openRepoShape runs the same flow ... quoted for PowerShell, on Windows."
• shape-doctor.py: a path that is not ASCII is printed as that path in every NEXT, keep line and WHY, as itself where the report's encoding writes its bytes and escaped whole where it does not, so a line naming Projets/Équipe or a Latin-1 name runs as printed (#785) #792's two lines follow in the same paragraph at README.md:162-163: "Off Windows, a path shape-doctor.py cannot print as itself is written $'…': bash, zsh and ksh read that; dash (/bin/sh on Debian and Ubuntu) does not."
• Nothing is lost or duplicated. The superseded old wording ("That two-liner IS the Windows way in from PowerShell, which has no twin...") is gone, and main's text replaces it.
• README.md is 1476 lines. Main's README is 1474, and tests/readme-cap.d/785.txt is +2, so it sits at its cap.
• AGENTS.md is 494 lines, at the cap asserted at tests/test_repo_hygiene.py:667.
• tests/readme-cap.d/785.txt is byte-identical to the one at 26d61255 (cmp).

3. All 23 handbook markers are current. I ran handbook_marker_problems, _handbook_markers, readme_segments and readme_fingerprint from the head's tests/test_repo_hygiene.py:

• problems: 0, markers: 23, current: 23 of 23.
• The 7 § Starting markers are all 3d4a01ffadc322f88dfffd9cef50b1c48c58b9b3, at docs/handbook.html lines 475, 615, 1092, 1129, 1191, 1436 and 1735, which matches the writer's list.
• Cross-check without the helpers: an awk cut of that README segment, passed through git hash-object --no-filters, gives 3d4a01ff…. Main's own segment hashes to 45a3f886…, which is what main's 7 markers carried.

4. The delta equals the reviewed one.

• Outside README.md, docs/handbook.html and tests/readme-cap.d/, git diff -U0 2e476a77 1ccb3dd7 and git diff -U0 bf1f099b 26d61255 are identical under cmp.
• The 772 figure counts the diff text including file headers (grep -v '^@@\|^index'). It is 750 +/- lines without them. Both measures are identical, and git patch-id --stable gives 2d43f4c1… for both.
• The only differences are two hunk-header offsets, from lines main added above them: AGENTS.md 268 vs 270, and shape-doctor.py +9972 vs +9964.

5. Lane 2's #821 did not come in.

• repo_shape.py, setup-project.py, shape-doctor.py, README and the handbook: under Git Bash (MSYSTEM set) the lines that follow the host are quoted for sh, through one question, host_shell_dialect, so the doctor's lines, setup's family lines and family.py's NEXT, switch -c, add and cd lines paste whole in the shell that printed them, and README's Windows sentence says so #821's head is 7581ffb9, state OPEN. It is not an ancestor of 1ccb3dd7 or of main, and none of its 7 own commits is in the head.
• host_shell_dialect has no match on the head. tests/test_git_bash_dialect.py does not exist there.
• MSYSTEM appears only in tests/conftest.py, with the same count (2) at 1ccb3dd7, 2e476a77 and 26d61255.
• Every repo_shape.py, setup-project.py, shape-doctor.py, README and the handbook: under Git Bash (MSYSTEM set) the lines that follow the host are quoted for sh, through one question, host_shell_dialect, so the doctor's lines, setup's family lines and family.py's NEXT, switch -c, add and cd lines paste whole in the shell that printed them, and README's Windows sentence says so #821 file that shape-doctor.py: a path that is not ASCII is printed as that path in every NEXT, keep line and WHY, as itself where the report's encoding writes its bytes and escaped whole where it does not, so a line naming Projets/Équipe or a Latin-1 name runs as printed (#785) #792 does not also touch is identical to main at 2e476a77. The two files both PRs touch hold none of repo_shape.py, setup-project.py, shape-doctor.py, README and the handbook: under Git Bash (MSYSTEM set) the lines that follow the host are quoted for sh, through one question, host_shell_dialect, so the doctor's lines, setup's family lines and family.py's NEXT, switch -c, add and cd lines paste whole in the shell that printed them, and README's Windows sentence says so #821's symbols.
• The + lines the two diffs share are only SPDX, from __future__ and @pytest.fixture boilerplate.

6. Tests on 1ccb3dd7, under a stub gh on PATH exiting 97, with GH_TOKEN=bogus and an empty GH_CONFIG_DIR:

• The 8 docs files, via /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=…/bt-docs -o tmp_path_retention_policy=failed tests/test_agents_md_codes.py tests/test_cli_reference.py tests/test_readme_sections.py tests/test_readme_windows_quoting_sentence.py tests/test_release_workflow.py tests/test_repo_hygiene.py tests/test_setup_by_hand_commands.py tests/test_update_add_codes_documented.py:
  300 passed in 59.23s
• tests/test_doctor_non_ascii_paths.py with the same flags, --basetemp=…/bt-nonascii: 126 passed in 108.87s (0:01:48).
• git diff --check 2e476a77 1ccb3dd7 exits 0, and so does git diff --check origin/main 1ccb3dd7.

Extra runs on the predicted squash (scratch commit 1bb5d656, tree b7048467…, on main c100843b). I added these because the merge auto-merged shape-doctor.py against #730, #752 and #759, and the extra files below are the ones those commits and the workflow changes touched:

• The 8 docs files, tests/test_doctor_non_ascii_paths.py, tests/test_stdout_encoding.py, tests/test_doctor_pin_restore_merges.py, tests/test_validator_child_encoding.py and tests/test_workflow_concurrency.py: 653 passed, 1 skipped, 1 xfailed in 313.43s (0:05:13).
• tests/test_shape_doctor.py, run once: 437 passed, 3 skipped, 1 xfailed in 812.11s (0:13:32).
• The marker check on that tree also reports 23 of 23 current, with 0 problems.

7. Merge with current origin/main.

• Fresh fetch, git merge-tree --write-tree origin/l3-785-non-ascii-paths origin/main is clean, exit 0. Predicted squash tree: 2d171f1e2ef046d618ef4104ab31e668629b9eb4, with origin/main at d8b8eb339ad90e2c229134ee7d320dfd713117e.
• Main moved twice while I worked. The last move is d8b8eb33, #family.py's reftable refusal, which touches only scripts/family.py and two tests. Against that tree, README.md, docs/handbook.html, AGENTS.md, tests/readme-cap.d/ and shape-doctor.py are identical to the c100843b prediction I tested.
• Against c100843b the predicted squash tree is b7048467e05f90e53fdcedf54d0880e4e25587b9. It is clean and differs from main in exactly the 10 PR files, with the same patch-id as the PR's net diff.
• I did not re-run tests on 2d171f1e itself.

Checks, gh pr checks 792:

• SonarCloud Code Analysis: pass (58s).
• tests-git-bash: pass (1m57s).
• package: pass (30s).
• tests: pending.

I did not wait for CI.

Scratch paths

• Clone, no checkout: /workspace/projects/openRepoShape-worktrees/_lane3-scratch/RV792m
• Scratch root: /workspace/projects/openRepoShape-worktrees/_lane3-scratch/RV792m-scratch/
  • head worktree at 1ccb3dd7: RV792m-head
  • predicted-squash worktree at local commit 1bb5d656, never pushed: RV792m-pred
  • marker script: RV792m-markers.py
  • test logs: RV792m-pred-tests.log, RV792m-sd-tests.log
  • basetemps: bt-docs, bt-nonascii, bt-pred, bt-sd
  • stub gh: RV792m-stub/gh
  • delta files: RV792m-delta-*.txt, RV792m-full-*.diff

Lane: openreposhape-3 (openRepoShape-3)

[brettheap]

(lane openRepoShape-3, 2026-10-18T00:24:44Z) RV792p's re-check of #792's fix round (1ccb3dd7..4948c7a4): READY, provided CI tests passes (run 39082233093 was pending). The merge of bee84d55 is exact, and the fix is the 3 lines that move tests/test_doctor_non_ascii_paths.py from PASTE_ENV to #818's paste_env(). Each git the tests run now gets the session's no-maintenance settings and the file protocol. On the predicted squash over 7b21059 the file runs 126 passed, test_no_auto_maintenance.py passes, and the handbook markers are current with README at 1476 of 1477.

#792 asks for its README turn once tests is green, gated again over the main of that time.

---

RV792p re-check of #792 (#785) at head 4948c7a4

VERDICT: READY. No MUST items.

Items

• SHOULD: none.
• NIT 1. tests/test_doctor_non_ascii_paths.py:50 lists paste_env before commit_in. Every other import of this module sorts its names. A sorted line fits in 79 columns: commit_in, detached_member, emptied_with_its_repository_kept, paste_env,. No linter in the repo checks it, and no test failed on it.

Checks 1 to 4

1. The merge is exact.

• git merge-tree --write-tree 1ccb3dd7 bee84d55 exits 0 with tree c5694c966170d10c85379350d638a7c36a6476b0.
• 86d4c708^{tree} is the same tree.
• The parents of 86d4c708 are 1ccb3dd7 and bee84d55.
• bee84d55 is an ancestor of origin/main (exit 0).

2. The fix is only that.

• git diff --stat 86d4c708 4948c7a4 shows one file, 3 insertions and 3 deletions: the import line (:50), the docstring line (:266) and the comprehension line (:268).
• PASTE_ENV appears at 4948c7a4 only in tests/test_no_auto_maintenance.py:533. That is a docstring on main; none of shape-doctor.py: a path that is not ASCII is printed as that path in every NEXT, keep line and WHY, as itself where the report's encoding writes its bytes and escaped whole where it does not, so a line naming Projets/Équipe or a Latin-1 name runs as printed (#785) #792's files have it.
• The fix was needed. A detached worktree of 86d4c708 fails collection:
  • ImportError: cannot import name 'PASTE_ENV' from 'test_doctor_legs_members_next'
  • no tests collected, 1 error
  • exit 2

3. Semantics.

• I read paste_env() (test_doctor_legs_members_next.py:62) and appended_to_command_scope (conftest.py:285) on origin/main. paste_env() takes os.environ at the call without PYTHONPATH, adds GIT_IDENTITY, appends the file protocol to the held GIT_CONFIG_* entries, and sets PYTHONDONTWRITEBYTECODE.
• environ()'s filter drops only LC_*, LANG*, PYTHONIO* and PYTHONUTF8. No GIT_* key matches.
• I proved this with a throwaway probe test on the predicted-squash tree, with the session fixture live, for all 3 LOCALES cells. The probe is in an untracked file in RV792p-wt-probe, never committed.
  • os.environ held GIT_CONFIG_COUNT=2: gc.auto=0, maintenance.auto=false, plus the session's GIT_CONFIG_GLOBAL.
  • environ() returned GIT_CONFIG_COUNT=3: gc.auto=0, maintenance.auto=false, protocol.file.allow=always, in that order.
  • The GIT_CONFIG_GLOBAL it returned equals the session's file.
  • A git config --show-scope --get-all run under that environment reported gc.auto and maintenance.auto at both global and command scope, and protocol.file.allow at command scope.
  • A second environ() call returned an identical dict, so there is no growth across calls.
• Calling paste_env() per environ() call changes nothing that shape-doctor.py: a path that is not ASCII is printed as that path in every NEXT, keep line and WHY, as itself where the report's encoding writes its bytes and escaped whole where it does not, so a line naming Projets/Équipe or a Latin-1 name runs as printed (#785) #792's tests assert. The tests assert on the doctor's bytes, --json values, the exit codes of pasted lines, rev-parse of the kept branch, and the --root word equalling the root's bytes. None reads the environment itself.
• It does change what each git subprocess sees. The old import-time dict was built at collection, before the session fixture ran. It carried no maintenance settings and a GIT_CONFIG_COUNT of its own. Now each git gets the session's settings, so the detached-maintenance race from tests: the holder fixtures copy the family fixture with shutil.copytree while git's background gc or maintenance can still be packing it, so a test errors at setup when a loose object vanishes mid-copy (#440's race, through a file copy) #645 cannot reach shape-doctor.py: a path that is not ASCII is printed as that path in every NEXT, keep line and WHY, as itself where the report's encoding writes its bytes and escaped whole where it does not, so a line naming Projets/Équipe or a Latin-1 name runs as printed (#785) #792's tests.
• The other callers in test_doctor_legs_members_next.py (lines 115, 456, 528, 623, 799, 1033) call paste_env() at the call too.
• tests/test_doctor_non_ascii_paths.py does not write os.environ. test_shape_doctor.py only touches LANES_LANE and PYTHONPATH, in fixtures these tests do not use.

4. #818's rule.

• test_no_test_sets_a_git_config_count_of_its_own passes with shape-doctor.py: a path that is not ASCII is printed as that path in every NEXT, keep line and WHY, as itself where the report's encoding writes its bytes and escaped whole where it does not, so a line naming Projets/Équipe or a Latin-1 name runs as printed (#785) #792's files in the tree.
• I called files_off_their_count(tests/) directly and it returned {}.
• Own counts are [] for test_doctor_non_ascii_paths.py, test_shape_doctor.py and test_stdout_encoding.py.
• shape-doctor.py: a path that is not ASCII is printed as that path in every NEXT, keep line and WHY, as itself where the report's encoding writes its bytes and escaped whole where it does not, so a line naming Projets/Équipe or a Latin-1 name runs as printed (#785) #792's files write no GIT_CONFIG_COUNT of their own.

Check 5: tests on the predicted squash

I fetched fresh. origin/main is 7b210593 and the PR head is 4948c7a4. The prediction is git merge-tree --write-tree origin/main 4948c7a4, exit 0, then commit-tree. That gives scratch commit 9274238a2f3aeb210b5ee3ba135959f5573bc0b4 with tree 4a4f1bf00e2e19108053c1c242b9bd1ca924e6c4. I never pushed it, and the worktree stayed clean.

I ran each command under the stub gh (exit 97) with GH_TOKEN=bogus and an empty GH_CONFIG_DIR. GIT_ALLOW_PROTOCOL was unset. I used /usr/bin/python3 -m pytest -q -p no:cacheprovider -o tmp_path_retention_policy=failed --basetemp=<scratch>/bt-RV792p-<name>, with timeout 1500.

Files	Result line
tests/test_doctor_non_ascii_paths.py	126 passed in 76.02s (0:01:16)
tests/test_no_auto_maintenance.py	36 passed in 5.90s
tests/test_stdout_encoding.py	47 passed, 1 skipped in 11.85s
the 8 docs files	331 passed in 42.22s

• The one skip in tests/test_stdout_encoding.py is tests/test_stdout_encoding.py:286: the default this is about is Windows' own.
• The 8 docs files were test_agents_md_codes.py, test_cli_reference.py, test_readme_sections.py, test_readme_windows_quoting_sentence.py, test_release_workflow.py, test_repo_hygiene.py, test_setup_by_hand_commands.py and test_update_add_codes_documented.py.
• I did not run tests/test_shape_doctor.py, as you said.

Marker check, using the predicted tree's tests/test_repo_hygiene.py helpers:

• handbook_marker_problems(readme, page, README_HEADING_TO_HANDBOOK_ID, HANDBOOK_READS) returned [].
• The ledger problems from readme_cap_raises were [].
• README.md is 1476 lines; the cap is 1422 + 2 (460.txt) + 5 (550.txt) + 12 (563.txt) + 22 (564.txt) + 10 (606.txt) + 1 (683.txt) + 2 (785.txt) + 1 (786.txt) = 1477. Headroom is 1 line.
• Main alone is 1474 lines against a cap of 1475, also 1 line of headroom. shape-doctor.py: a path that is not ASCII is printed as that path in every NEXT, keep line and WHY, as itself where the report's encoding writes its bytes and escaped whole where it does not, so a line naming Projets/Équipe or a Latin-1 name runs as printed (#785) #792 adds 2 lines and its own 785.txt raise of 2.
• The 786.txt entry is the README-turn PR already on main.

Check 6: CI and the tree

• The predicted squash tree 4a4f1bf00e2e19108053c1c242b9bd1ca924e6c4 is over main 7b2105937567da30231db6a42749f258af48d9f9.
• gh pr checks 792 -R opensoft/openRepoShape showed this, read at the time:

Check	State
SonarCloud Code Analysis	pass (1m11s)
package	pass (29s)
tests-git-bash	pass (2m16s)
tests	pending

• The pending tests job is the Linux job, run 39082233093, which I did not wait for. gh pr view 792 showed headRefOid 4948c7a4, state OPEN, mergeable and mergeStateStatus UNKNOWN.
• origin/main is 2 commits past bee84d55: 5fc049b7 and 7b210593. They merge into the branch cleanly in the prediction.

Scratch paths

• Clone: /workspace/projects/openRepoShape-worktrees/_lane3-scratch/RV792p
• Scratch root: /workspace/projects/openRepoShape-worktrees/_lane3-scratch/RV792p-scratch
• Predicted squash worktree: .../RV792p-scratch/RV792p-wt
• Worktree of 86d4c708: .../RV792p-scratch/RV792p-wt-merge
• Probe worktree, holding the untracked probe test: .../RV792p-scratch/RV792p-wt-probe
• Probe scripts: .../RV792p-scratch/RV792p-probe.py and .../RV792p-scratch/RV792p-probe2.py
• Stub gh: .../RV792p-scratch/RV792p-stub/gh
• Run scripts: .../RV792p-scratch/RV792p-runA.sh and .../RV792p-scratch/RV792p-runB.sh
• Logs, all in .../RV792p-scratch/: RV792p-nonascii.log, RV792p-nam.log, RV792p-stdout.log, RV792p-docs.log
• Basetemps: .../RV792p-scratch/bt-RV792p-*
• I made no GitHub write and ran no rm -rf. No test processes of mine are still running.

Lane: openreposhape-3 (openRepoShape-3)

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sourcery-ai]

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 3 days and 17 hours by commenting @sourcery-ai review. Upgrade to get a review now.