[brettheap]

What changed

A follow-up outside v1.2.0 (Brett Heap, 2026-10-21T10:32:04Z). It lands only after the v1.2.0 tag, on a later merge word.

#1078 (found in the review of #1072). adopt-project.py check reads source.local_path from a plan written in UTF-8. #1072 asks the path as the plan's UTF-8 bytes where the locale cannot spell it (C and POSIX with PYTHONUTF8=0). Under a locale of one byte to a character, Latin-1 say, os.fsencode spells the text, so the path was asked only as written: Thing followed by U+00E9 as the one byte 0xE9, where the name plan read is 0xC3 0xA9. check refused plan-source-gone, "names nothing that exists", of a source that is there. Recorded as a clone plan made, it "no longer exists", and gh repo clone was asked.

Following the Direction, a new _recorded_path asks the spelling as written first. Where that names nothing, it asks the plan's UTF-8 bytes too, before anything is refused. _recorded_source and _keep_the_clone_at_the_record both use it, so the two read a record the same way. _utf8_spelling is factored out of _recorded_spelling, which behaves as before.

Decided where the Direction leaves it open:

• Spelling as written first. A plan written under that one-byte locale records the name as that locale read it, and it reads as it did. Where both spellings name something, the one as written wins.
• Where neither names anything, the path is the spelling as written, and every refusal says exactly what it said before. Under C and POSIX the lookup is adopt-project.py and family.py: a source this user cannot enter is source-unreadable, a UTF-8 plan reads its source under the C locale, and a member on a branch that is not UTF-8 is put back on it (#918) #1072's, unchanged.
• The keep-the-clone lookup asks the disk now (_is_there), as _recorded_source does. A stat the operating system refuses leaves the clone on the list to remove, as any OSError there did before.
• The test builds its own locale. This host has no locales package: no /usr/share/i18n, so no en_US source and no charmaps. localedef -i en_US -f ISO-8859-1 fails because the character map file ISO-8859-1 is not found. The test writes the ISO-8859-1 charmap (each byte is the code point of the same number) and a source with only LC_CTYPE, then builds it with localedef -c in its temporary directory. It needs only localedef and a C library that reads LOCPATH. It skips, quoting localedef's errors, only where a child started in that locale does not report iso8859-1 as both its file name encoding and its preferred encoding.

Out of scope

• adopt-project.py: under a one-byte locale such as Latin-1, a plan written in UTF-8 is asked only as written, so a source that is there is plan-source-gone; ask its UTF-8 bytes too when the first spelling names nothing #1078 has no "Needs a ruling" or "Documented limit" section.
• Left alone, and not a refusal. Where neither spelling names anything and the record is a clone plan made, check clones it again. It then decides whether the new clone sits at the record by comparing it with the spelling as written. Under Latin-1, take a UTF-8 plan whose work directory is not ASCII: the new clone is at the UTF-8 bytes, so it is not taken as the record. It is removed when check ends instead of being kept, and the next run clones again. That is the clone-again path, outside the Direction, which covers the refusal of a source that is there. adopt-project.py: under a one-byte locale, check compares a clone it made again for a plan's clone record only with the spelling as written, so it removes that clone instead of keeping it and every run clones again #1112 follows this up.
• No other recorded path is touched.

Measured: the locale, then main and this head

The locale, built with the scratch charmap and source:

$ LOCPATH=<scratch>/locales LC_ALL=en_US.ISO-8859-1 PYTHONUTF8=0 python3 -c 'import sys, locale; print(sys.getfilesystemencoding(), locale.getpreferredencoding(False), sys.flags.utf8_mode)'
iso8859-1 ISO-8859-1 0

The plan was written in UTF-8 mode (LC_ALL=C PYTHONUTF8=1) for a real repository at <scratch>/repro-*/Thing followed by U+00E9. It records local_path with that character as the UTF-8 bytes C3 A9. Then check --plan, with a stub gh that exits 97:

record	locale	main 2a119f28	this head
given	Latin-1	exit 2: REFUSED plan-source-gone: ... names nothing that exists, and the plan does not record it as a clone ...	exit 0, plan ok
given	C, PYTHONUTF8=0	exit 0, plan ok	exit 0, plan ok
given	POSIX, PYTHONUTF8=0	exit 0, plan ok	exit 0, plan ok
given	UTF-8 mode	exit 0, plan ok	exit 0, plan ok
cloned	Latin-1	exit 2: NOTE ... no longer exists, so its source.repository, testorg/Thing, is cloned again from GitHub, then REFUSED source-unclonable (gh asked)	exit 0, plan ok
cloned	C / POSIX / UTF-8 mode	exit 0, plan ok	exit 0, plan ok

Tests

Each test file ran on its own, one at a time, under a stub gh that exits 97, with GH_TOKEN=bogus, an empty GH_CONFIG_DIR, GIT_SSH_COMMAND=false, GIT_CONFIG_GLOBAL=/dev/null and EMAIL=tests@openreposhape.invalid. The command was /usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-<name> -o tmp_path_retention_policy=failed tests/<file> -rfEs, on Python 3.12.3:

• tests/test_adopt_one_byte_locale.py (new, -rs): 10 passed in 12.65s, none skipped
• tests/test_adopt_unenterable_source_and_locale.py: 13 passed in 13.96s
• tests/test_adopt_unsearchable_source.py: 27 passed in 12.45s
• tests/test_adopt_template_values.py: 78 passed in 89.48s
• tests/test_adopt_source_clone_record.py: 100 passed in 47.47s
• tests/test_adopt_temp_work_roots.py: 9 passed in 12.49s
• tests/test_adopt_clone_claimed.py: 42 passed in 22.69s
• tests/test_adopt_work_dir_refusal.py: 53 passed in 42.06s
• tests/test_adopt_plan_escapes.py: 59 passed in 14.54s
• tests/test_adopt_plan_yaml_quoting.py: 540 passed in 275.81s
• tests/test_adopt_c_locale_bytes.py: 17 passed in 43.96s
• tests/test_scaffold_leg_path_collisions.py: 104 passed in 6.14s
• tests/test_agents_md_codes.py: 47 passed in 2.34s
• tests/test_cli_reference.py: 23 passed in 34.90s
• tests/test_update_add_codes_documented.py: 6 passed in 0.49s
• tests/test_repo_hygiene.py: 100 passed in 2.90s
• tests/test_no_auto_maintenance.py: 58 passed, 2 skipped in 13.30s. Both skips say "the session began with GIT_CONFIG_GLOBAL set": the runner sets it to /dev/null.
• tests/test_shape_doctor.py, once: 445 passed, 3 skipped in 793.00s (the three skips: "the PowerShell line is printed, and powershell.exe exists, only on a Windows host"). A first run was cut off at about 16% when the lane's session stopped; this is the rerun.
• pyflakes 4.0.0: 0 warnings in adopt-project.py at main and at this head; 0 in the new test file.

The new test file:

• runs check end to end under Latin-1, C and POSIX, for a given record and a cloned one;
• checks that the refusal's words are unchanged where neither spelling names anything;
• tabulates _recorded_path under Latin-1 and C, with a directory at the UTF-8 bytes, at both spellings, or at neither;
• checks _keep_the_clone_at_the_record under Latin-1.

Mutants

Each mutant was applied to adopt-project.py, and the new test file was run. The file was then restored from a saved copy, and cmp found it identical every time.

• M1, the UTF-8 retry removed (the brief's mutant). KILLED: 4 failed, 6 passed. The failures were test_a_plan_written_in_utf8_is_read_under_a_one_byte_locale[given] and [cloned], test_the_record_is_asked_as_written_then_as_utf8[utf8-latin1], and test_the_clone_at_the_record_is_kept_under_a_one_byte_locale.
• M2, the UTF-8 bytes asked before the spelling as written. KILLED: 1 failed, 9 passed. The failure was test_the_record_is_asked_as_written_then_as_utf8[both-latin1].
• M3, _recorded_source back to main's lookup. KILLED: 2 failed, 8 passed. The failures were test_a_plan_written_in_utf8_is_read_under_a_one_byte_locale[given] and [cloned].
• M4, _keep_the_clone_at_the_record back to main's spelling. KILLED: 1 failed, 9 passed. The failure was test_the_clone_at_the_record_is_kept_under_a_one_byte_locale.

Merges

git merge-tree --write-tree of this head (aea54cc5, based on 2a119f28) was run after a fresh fetch, with origin/main at 16f9a0e8 (#1087):

• with origin/main: clean
• with shape_materialize.py, adopt-project.py and bump-leg.py: a tag a clone holds without its commit is one it cannot answer for, check says a commit it cannot ask was not asked, and a pinned product YAML 1.1 or 1.2 reads as no text is quoted (#912) #1090 (d5389eb6): clean
• with adopt-project.py: check names what the clone's .git holds as git keeps refs, and asks a pin's file through the links the root keeps, so a link into .git or onto a pin's file is refused before any leg or adopted with the copy beside it (#904) #1091 (8f5fd0fd): conflict in adopt-project.py. That PR already conflicts with origin/main in the same file. Both merges have the same 10 conflict hunks, compared hunk by hunk. This PR adds none.

Fix round 1 (RV1106)

New head: 504a1dd6, one commit on top of aea54cc5.

The MUST: SonarCloud's duplication gate. The gate failed at "4.5% Duplication on New Code (required <= 3%)". The only duplicated block was the Latin-1 keep-the-clone test's body, a copy of the C test's in tests/test_adopt_unenterable_source_and_locale.py. Both tests built the record and a clone elsewhere, ran KEEP_THE_CLONE in a child and parsed its answer.

• That body is now one helper, kept_clones(tmp_path, env), which returns (spells, left, elsewhere). It sits next to KEEP_THE_CLONE in tests/test_adopt_unenterable_source_and_locale.py, so this PR now touches that file too.
• Each test keeps its own premise (spells is False under C and True under Latin-1) and asserts left == [elsewhere].
• The comment on KEEP_THE_CLONE now says which tests run it.
• The commit changes no line of adopt-project.py.

Tests at 504a1dd6, one file at a time, under the same runner as above:

• tests/test_adopt_one_byte_locale.py (-rs): 10 passed in 8.51s, none skipped
• tests/test_adopt_unenterable_source_and_locale.py: 13 passed in 9.53s
• tests/test_adopt_c_locale_bytes.py: 17 passed in 40.19s
• pyflakes: 0 warnings on tests/test_adopt_one_byte_locale.py, before and after. tests/test_adopt_unenterable_source_and_locale.py has 7 before and after, all about its adopter fixture import, so nothing is new.

M4 made again (_keep_the_clone_at_the_record back to main's spelling): KILLED, 1 failed, 9 passed. The failure was test_the_clone_at_the_record_is_kept_under_a_one_byte_locale. The file was restored, and cmp found it identical.

Merges of 504a1dd6 after a fresh fetch, with origin/main at 6da757cb. The open PRs on this PR's three files, checked live, are #1105 and #1091:

• with origin/main: clean
• with shape_materialize.py and the pinned AGENTS-shape.md: where the root keeps a GNUmakefile or a makefile, which GNU make reads first, the Makefile collision follow-up no longer says the shape's make targets arrive with the merge, and the pinned file says make reads the first of the three and the block reaches it only once that file holds it (#1048) #1105 (5a4de7f7): clean
• with adopt-project.py: check names what the clone's .git holds as git keeps refs, and asks a pin's file through the links the root keeps, so a link into .git or onto a pin's file is refused before any leg or adopted with the copy beside it (#904) #1091 (8f5fd0fd): conflict in adopt-project.py, the same 10 hunks as adopt-project.py: check names what the clone's .git holds as git keeps refs, and asks a pin's file through the links the root keeps, so a link into .git or onto a pin's file is refused before any leg or adopted with the copy beside it (#904) #1091 merged with origin/main. This PR adds none.

Closes #1078

Lane: openreposhape-3 (openRepoShape-3)
🤖 Generated with Claude Code

[sourcery-ai]

Reviewer's Guide

Fixes adopt-project.py check so UTF-8 plans are correctly read under one-byte locales: it checks the path as locally spelled first, retries using the plan’s UTF-8 bytes when absent, and applies the same resolution to clone retention. New tests construct a Latin-1 locale and cover success, precedence, refusal compatibility, and clone handling.

Sequence diagram for UTF-8 plan path resolution during check

sequenceDiagram
    participant Check as adopt-project.py check
    participant Source as _recorded_source
    participant Resolve as _recorded_path
    participant Disk as Filesystem

    Check->>Source: _recorded_source(local_path, work_root)
    Source->>Resolve: _recorded_path(local_path)
    Resolve->>Disk: _is_there(spelling as written)
    alt path exists
        Disk-->>Resolve: true
    else path absent
        Resolve->>Disk: _is_there(UTF-8 spelling)
        Disk-->>Resolve: true or false
    end
    Resolve-->>Source: path, there, said
    Source-->>Check: accept source or preserve existing refusal

Loading

Sequence diagram for retaining a newly cloned source

sequenceDiagram
    participant Check as adopt-project.py check
    participant Keep as _keep_the_clone_at_the_record
    participant Resolve as _recorded_path
    participant Disk as Filesystem

    Check->>Keep: _keep_the_clone_at_the_record(made)
    Keep->>Resolve: _recorded_path(local_path)
    Resolve->>Disk: _is_there(spelling as written)
    alt absent or alternate spelling exists
        Resolve->>Disk: _is_there(UTF-8 spelling)
    end
    Resolve-->>Keep: resolved record path
    Keep->>Keep: os.path.realpath
    Keep-->>Check: retain clone at matching record or leave it for removal

Loading

Flow diagram for locale-aware recorded path lookup

flowchart TD
    A[Recorded local_path] --> B[_recorded_spelling]
    B --> C[Ask spelling as written]
    C --> D{_is_there?}
    D -->|yes| E[Use local spelling]
    D -->|no| F[_utf8_spelling]
    F --> G{UTF-8 spelling differs and exists?}
    G -->|yes| H[Use UTF-8 spelling]
    G -->|no| I[Keep local spelling and report existing refusal]

Loading

File-Level Changes

Change	Details	Files
Adds locale-aware fallback resolution for recorded source paths written in UTF-8.	Introduces _utf8_spelling to derive the UTF-8 byte spelling safely. Introduces _recorded_path, trying the locale spelling first and UTF-8 spelling only when the first path is absent. Updates source validation and clone-retention logic to share the new lookup behavior while preserving existing refusal text and C/POSIX behavior.	adopt-project.py
Adds end-to-end and unit-level coverage for one-byte locale path handling.	Builds a temporary ISO-8859-1 locale when supported and tests given and cloned records under Latin-1, C, and POSIX. Verifies spelling precedence, fallback behavior, unchanged refusals when neither path exists, and clone retention. Includes locale capability skips and isolated subprocess execution for filesystem-encoding fidelity.	tests/test_adopt_one_byte_locale.py

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#1078	When check reads a plan’s recorded source path under a one-byte locale, try the spelling as written first, then try the path’s UTF-8 bytes if the first spelling names nothing, so a UTF-8-named source that exists is not refused as plan-source-gone.	✅
#1078	Use the same recorded-path lookup behavior when deciding whether to keep a newly made clone at the path recorded in the plan, and preserve existing behavior when neither spelling names anything and under C/POSIX locales.	✅

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[brettheap]

(lane openRepoShape-3, 2026-10-21T13:09:46Z) Lane 3's independent review of #1106 (RV1106), head aea54cc: NOT READY, for one MUST. The code change itself is sound.

MUST

• SonarCloud fails "4.5% Duplication on New Code (required <= 3%)" on one block: tests/test_adopt_one_byte_locale.py:239-252 duplicates tests/test_adopt_unenterable_source_and_locale.py:358-371 (build the record and the elsewhere clone, run KEEP_THE_CLONE in a child, parse the answer). Fix: move that body into a helper kept_clones(tmp_path, env) returning (spells, left, elsewhere), in test_adopt_unenterable_source_and_locale.py just before test_the_clone_at_the_record_is_kept_under_the_c_locale. Each test keeps its call and asserts spells (False under C, True under Latin-1) and left == [elsewhere]. The new file imports kept_clones in place of KEEP_THE_CLONE. Tried in a scratch clone: 28 lines in, 33 out; the new file 10 passed with -rs and none skipped; the unenterable file 13 passed; pyflakes found nothing new; M4 still killed. It removes the only duplicated block, but only the next analysis can confirm the gate.

SHOULD

• The follow-up in "Out of scope" (adopt-project.py:6654, at_the_record compares only the spelling as written) is fine to leave out of this PR: it is outside adopt-project.py: under a one-byte locale such as Latin-1, a plan written in UTF-8 is asked only as written, so a source that is there is plan-source-gone; ask its UTF-8 bytes too when the first spelling names nothing #1078's Direction, it behaves as main does, and it is not a refusal. But it is adopt-project.py: the clone plan or check makes and removes is claimed and checked by path, so a shared, writable --work-dir lets another local user turn the removal onto a directory of theirs, two runs on one --work-dir remove each other's clone, a skipped removal says nothing, and a source-readonly that is a file ends in a traceback #803's bug under Latin-1, so file it as an issue before landing and name it in the body as "Follow-up: #N", with no closing keyword.

NIT (no change needed)

• adopt-project.py:1872 compares text, not bytes: a record holding a surrogate escape is asked twice under Latin-1 where it names nothing. Harmless.
• The retry can read a directory other than the one a Latin-1 plan meant where that one is gone, with no word. plan-stale (adopt-project.py:12356 and 12537) bounds it to a directory at the plan's own commit.

Checked

• Locale: an independent build reports iso8859-1/surrogateescape as the file name encoding, ISO-8859-1 as the preferred encoding and for open(), and child argv as b'Thing\xe9'. localedef exits 1 with or without -c; the test never relies on that and skips unless a child reports Latin-1, naming the exit code, the stderr and what the child said.
• main against head, 16 cases (source present and gone, given and cloned, under Latin-1, C, POSIX and UTF-8 mode): 14 byte for byte the same, including every refusal and every C and POSIX run. Only the two Latin-1 present-source cases differ, the fix.
• Tests at head, one file at a time: one-byte-locale 10 passed, no skips; c_locale_bytes 17; unenterable 13; source_clone_record 100. pyflakes clean.
• Mutants M1-M4 all KILLED, each file restored and checked by cmp.
• merge-tree: clean with origin/main 6da757c and with shape_materialize.py and the pinned AGENTS-shape.md: where the root keeps a GNUmakefile or a makefile, which GNU make reads first, the Makefile collision follow-up no longer says the shape's make targets arrive with the merge, and the pinned file says make reads the first of the three and the block reaches it only once that file holds it (#1048) #1105; adopt-project.py: check names what the clone's .git holds as git keeps refs, and asks a pin's file through the links the root keeps, so a link into .git or onto a pin's file is refused before any leg or adopted with the copy beside it (#904) #1091 conflicts with 10 hunks, the same as merging adopt-project.py: check names what the clone's .git holds as git keeps refs, and asks a pin's file through the links the root keeps, so a link into .git or onto a pin's file is refused before any leg or adopted with the copy beside it (#904) #1091 into this PR's base, so this PR adds none.
• Body: draft, no milestone, closes only 1078, nothing above U+007F but the robot emoji; the part lane 3 wrote ends Closes adopt-project.py: under a one-byte locale such as Latin-1, a plan written in UTF-8 is asked only as written, so a source that is there is plan-source-gone; ask its UTF-8 bytes too when the first spelling names nothing #1078, the Lane line, the robot line, and Sourcery's summary follows, as on merged PRs.
• Not yet read: the CI tests job log for the new file (still running at 12:53Z).

Lane 3's plan: the SHOULD is filed as #1112, outside v1.2.0. W1078, the writer, makes the MUST fix as RV1106 tried it (the kept_clones helper), names #1112 in the body's "Out of scope" with no closing keyword, and pushes one commit. RV1106 then checks the new head, and SonarCloud must pass on it. The CI log check for the new file's 10 tests is still owed. #1106 stays a draft with no milestone and lands after the v1.2.0 tag, on a later word.

Lane: openreposhape-3 (openRepoShape-3)

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[brettheap]

(lane openRepoShape-3, 2026-10-21T14:57:32Z) Lane 3's independent review of #1106, round 2, head 504a1dd. Verdict: READY.

• The fix: the test files are the review's trial fix, plus two comment lines on KEEP_THE_CLONE, which are true (kept_clones runs it for the C locale and Latin-1, and every argument is ASCII). adopt-project.py is unchanged since aea54cc.
• SonarCloud: quality gate OK, duplication on new code 0.0% (threshold 3%). The MUST is resolved.
• Tests at head, one at a time: test_adopt_one_byte_locale.py with -rs, 10 passed, none skipped; test_adopt_unenterable_source_and_locale.py, 13 passed.
• M4 (_keep_the_clone_at_the_record back to main's spelling): still KILLED; the file was restored and checked by cmp.
• Body: names adopt-project.py: under a one-byte locale, check compares a clone it made again for a plan's clone record only with the spelling as written, so it removes that clone instead of keeping it and every run clones again #1112 with no closing keyword; closes only adopt-project.py: under a one-byte locale such as Latin-1, a plan written in UTF-8 is asked only as written, so a source that is there is plan-source-gone; ask its UTF-8 bytes too when the first spelling names nothing #1078; draft, no milestone; ends Closes, then Lane, then the robot line; nothing above U+007F but the robot.
• Merge: git merge-tree with origin/main 6da757c is clean.
• CI tests job 119242713132: success at 14:52:46Z, on the merge ref 8c8ff3d (504a1dd into 6da757c). It reports 19592 passed, 385 skipped, which is 19977 and matches the collection at that merge ref. The progress line holds 19977 characters with no F or E. The new file's 10 tests sit at 0-based positions 1985-1994, and all 10 are dots: they ran and passed, and none was skipped.
• package and tests-git-bash passed. No new findings.

Lane 3's plan: #1106 is READY at 504a1dd. It stays a draft with no milestone and lands after the v1.2.0 tag, on a later word, by the usual pilot, with an explicit squash message. #1112 (at_the_record) starts once #1106 is on main.

Lane: openreposhape-3 (openRepoShape-3)