[brettheap]

Lane: openreposhape-3 (openRepoShape-3)
Claim: #733 (comment)

Closes #733.

Stacked on #774; lands after it. It changes submodule_keys, as #774 (#732) does, so its diff here shows #774's commit 311ad7d9 until that lands. Lands after the v1.1.0 tag.

What was wrong (re-measured on main e3137010 with #774, git 2.43.0)

I built one fresh repository per case, with a real gitlink at p, and took git submodule init -- p as git's answer. The issue's claims hold, and so do the three cases in its later comment (issuecomment-6221711270).

• Sections git ignores whole. [submodule ""], ../evil, a/../b and a\..\b: git says No url found, exit 128. The doctor gave ('', '/r/u.git') or ('../evil', '/r/u.git'), and siblings.submodule_urls gave {'p': '/r/u.git'}. a..b is read, as a control.

• Keys git ignores alone. A url beginning with - is ignored: -u alone is No url found, and /r/first.git then -u registers /r/first.git. The doctor and siblings took -u. A path beginning with - is ignored too: p then -p stays p, where siblings keyed the url by -p and mounted_from fell back to repository:.

• A file git refuses whole. It refuses over a key in a section it reads, given with no value at all:

  • a bare path, url, ignore or update gives bad config line N, exit 128;
  • a bare branch crashes git, exit 139;
  • a bare shallow, fetchRecurseSubmodules or unknown key is fine, and so is any bare key in a section git ignores.

  Both tools read every other section of such a file.

• A section with a url and no path. git mounts it nowhere (No url found for its name as a path). mounted_from and siblings keyed it by its name.

• An unmerged .gitmodules (comment 1). git reads no copy, so git submodule init says No url found with conflict markers in the working tree, with the conflict resolved there, and with that copy deleted. gitmodules_entry gave (None, None), then ('p', '/r/resolved.git'), then HEAD's url.

• The last section giving a path has no url (comment 2). This is left as it is. gitmodules_entry gives the section and no url, so the member is url-less, as git says. mounted_from gives an earlier section's url, as siblings does. Its contract is to agree with make siblings, and validate-family.py refuses two sections for one path (family.py: add never reads .gitmodules, so a section under ANOTHER name whose path is already members/<P> (a "ghost" section) does not stop it: it exits 0 and commits two sections for one path, validate-family.py says family ok, and a recursive clone takes the LATER section, so with the ghost second it dies fetching the ghost's URL #300).

• Reopened sections (comment 3). This is left as it is, apart from the docstring. git orders sections by the last path key, mounted_from by a name's first key, and siblings by its first url, so the three can name different sections. mounted_from's docstring now says so; it overstated agreement for anything but sections written whole.

What changed

In shape-doctor.py:

• submodule_keys (:5010-5057), with DOTDOT_COMPONENT (:5000) and VALUED_KEYS (:5007). It keeps only what git reads, by git's own tests:
  • a section whose name is empty or holds a .. component (between / or \) is skipped whole;
  • a path or url value beginning with - is skipped, that key alone;
  • a bare key of VALUED_KEYS in a section git reads returns None, meaning git refuses the file.
• section_url (:4966-4994) gives (None, None) for a refused file, and submodule_config (:7832-7885) gives {}.
• gitmodules_entry (:4908-4963) returns (None, None) while git ls-files -u -- .gitmodules lists anything, before it chooses a copy.
• mounted_from (:7888-7941) no longer reads a section's NAME as its path, and its docstring is softened for reopened sections.

In templates/family-root/scripts/siblings.py:

• submodule_urls (:237-284), with DOTDOT_COMPONENT (:224) and VALUED_KEYS (:229). It applies the same rules, returns {} for a refused file, and keys a url by its section's path alone.
• The constants are copied rather than imported from scripts/repo_shape.py. A holder may keep an older repo_shape.py, and siblings.py imports no new name from it. The test below pins the two copies to git and to each other.

Tests

• tests/test_doctor_legs_members_next.py::test_733_the_doctor_and_make_siblings_skip_what_git_skips (:1610-1663). GIT_SKIPS holds 17 .gitmodules files: the cases above and four controls (a..b, a plain path = -p, a bare shallow, and a bare key in an ignored section). For each, the test asks four things, and they must all agree with git:

  • gitmodules_entry(root, "p") must equal the (name, url) git finds;
  • git submodule init -- p (url_git_finds) must give that url;
  • mounted_from(root, "p", ...) must give it, or the repository: fallback where there is none;
  • siblings.submodule_urls(root).get("p") must give it.

  This is the doctor/siblings parity for these cases.

• The unmerged cases. INDEX_COPIES gains unmerged (:1501-1515): three copies staged as stages 1-3 through update-index --index-info, fed as bytes. test_733_while_gitmodules_is_unmerged_a_resolved_copy_is_not_read (:1593) covers the conflict resolved in the working tree. Both are checked against git submodule init.

• Against shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's head 311ad7d9, 15 of these 22 cases fail. The 7 that pass are shape-doctor.py: a holder's .gitmodules is split at the first space and at every Unicode line break, and read through a regexp a UTF-8 locale cannot match, so a section name holding a space or a byte that is not UTF-8, or a url holding NEL or U+2028, gives the doctor another mount url than git and make siblings use #668's three INDEX_COPIES cases and the four controls.

Commands on head 08ebe8bf, each run with a stub gh (exits 97) first on PATH, GH_TOKEN=bogus and an empty GH_CONFIG_DIR, from tests/:

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt -o tmp_path_retention_policy=failed test_doctor_gitmodules_not_utf8.py test_doctor_legs_members_next.py test_doctor_members_spelled.py test_doctor_gitlink_repairs.py test_repo_hygiene.py test_cli_reference.py
312 passed in 220.94s (0:03:40)

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-sib -o tmp_path_retention_policy=failed test_family_siblings_placement.py test_siblings_family_name.py test_family_makefile_siblings.py test_windows_paths.py test_park_resume_targets.py test_family_printed_quoting.py
384 passed in 56.26s

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-others -o tmp_path_retention_policy=failed test_family.py test_doctor_next_is_a_command.py test_doctor_validator_rows.py
212 passed in 120.73s (0:02:00)

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-sd -o tmp_path_retention_policy=failed test_shape_doctor.py
436 passed, 3 skipped, 1 xfailed in 348.65s (0:05:48)

The skips and the xfail are in test_shape_doctor.py, which this PR does not change.

Pinned shape copies

templates/family-root/scripts/siblings.py is a pinned family-root copy (FAMILY_COPIED_VERBATIM). Every family holder gets an upstream-changed scripts/siblings.py row from update-shape.py check until it applies the update. shape-doctor.py is not a pinned copy.

The other half: lane openRepoShape-4's #690

• Lane 4's siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690 (v1.1.0) rewrites submodule_urls' reading to git config -z --list. Its WIP branch fix/690-siblings-inputs (38a40abf, not yet a PR) also adds the doctor/siblings parity test, test_the_doctor_and_make_siblings_read_one_url_for_one_mount in tests/test_siblings_inputs.py.
• siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690 lands first, being v1.1.0. This PR then merges over it at submodule_urls' loop, keeping both changes. Its rules go into siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690's -z loop: a bare key there is a record with no \n.
  • Do not take this PR's loop as it stands (review NIT): siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690's -z call returns bytes, and line.partition("=") on bytes raises TypeError, which would break make siblings.
  • The merge keeps siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690's loop and adds this PR's checks to it: DOTDOT_COMPONENT, VALUED_KEYS, git_dies_on, the - values, and keying by path alone.
  • Both PRs conflict in two hunks of siblings.py, the docstring and the loop.
• For that parity test, add shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a name with a .. component, a url beginning with -) is still read as a member, where git says No url found and exits 128 #733's cases: every GIT_SKIPS row, the unmerged .gitmodules, and a reopened section.
• Known differences between the two tools, after this PR, until siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690 lands:
  • a url holding a line break other than \n (VT, FF, FS, GS, RS, NEL, U+2028, U+2029), which siblings cuts short;
  • a section name holding =, which siblings drops;
  • an empty url =: siblings gives '', mounted_from falls back to repository:, and git registers the empty url (measured: exit 0);
  • a reopened section, as above.

Neighbours

This stays in the .gitmodules readers #700 added or changed, and in siblings.py::submodule_urls. It does not touch pin_restore (#736), the partial-clone row (#754) or the lockstep NEXT (#750).

Round 2 (head d9b15660)

RV-c's review (issuecomment-6229523010) found it NOT READY. This round first merges #774's new head 93b9bac3 (3df0f99a), then takes the MUST, SHOULDs 1-3 and the NIT.

• MUST: an index or HEAD copy keeps the sections before its first bad line. git reads the working tree's file with CONFIG_ERROR_DIE, which refuses it whole, and an index or HEAD copy with CONFIG_ERROR_ERROR, which stops at the bad line and keeps what came before.

  • submodule_keys(listing, working_tree) (shape-doctor.py:5058-5118) returns None at a bare VALUED_KEYS key for the working tree, and the keys read so far for an index or HEAD copy.
  • gitmodules_entry (:4908-4967) threads which copy it read through section_url (:4970-5000).
  • submodule_config and siblings.py read only the working tree, as before.
  • Measured on git 2.43: p's section, then a bare url, in an index copy registers p; with the bare url first it registers none. The doctor now gives the same.

• SHOULD 1: values git dies on refuse the file in any copy. git_dies_on (:5041-5055) and git_bool (:5026-5038), with GIT_BOOL_WORDS and GIT_INT (:5017-5023), refuse:

  • an update other than exactly none, checkout, rebase or merge. !echo, Merge and an empty value each die (measured);
  • a shallow that is no boolean. The test for an integer follows git_parse_int: base-0 strtoimax, a k/m/g unit, and it must fit an int. Measured: 2k, 0x10, TRUE and an empty value are taken; maybe, 08, 2g and 99999999999 die;
  • a fetchRecurseSubmodules that is neither a boolean nor on-demand.

  A die-value inside an ignored section is fine, and so is one past the bad line of an index copy, which git never reads. siblings.py gets the same check: git_dies_on (:244-263), and GIT_BOOL_WORDS and GIT_INT (:234-236), used in submodule_urls (:266-314).

• SHOULD 2: a blob's partial listing is kept. gitmodules_listing(..., partial=True) (:5121-5157) keeps what git config -z --blob ... --list prints before a line it cannot parse, then exits 128. It is used for an index or HEAD copy known to exist. Measured: p, then an unparsable line, in an index copy registers p.

• SHOULD 3: one version difference, a bare branch.

  • git 2.43 crashes on it (xstrdup(NULL), exit 139), in any copy.
  • git 2.55 (CI), per RV-c's reading of git's source, refuses it like any other bare valued key: exit 128 for the working tree, and for an index or HEAD copy it stops there and keeps earlier sections. The doctor follows 2.55.
  • url_git_finds (tests/test_doctor_legs_members_next.py:1576) skips, with the reason, when git crashes. The doctor's answer is still asserted first. So the two bare-branch rows run on 2.55 and skip on 2.43. I measured them on 2.43 only; 2.55 is not installed here.

• Tests.

  • INDEX_COPIES (:1507-1532) gains six index-copy rows: a bare url after p and before it, an unparsable line after p, a die-value, a die-value past a bare url, and a bare branch after p.
  • GIT_SKIPS (:1643-1694) gains nine working-tree value rows, with controls.
  • Every row is checked against git submodule init.
  • Against round 1's head 08ebe8bf, 10 of the new rows fail: five index-copy rows and five die-value rows.

• NIT: the siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690 paragraph above now says shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a .. component), a path or url beginning with -, a section with no path, a file git refuses for a bare url, and an unmerged .gitmodules name no mount url, NEXT line or clone, where git says No url found and exits 128 (#733) #775's rules go into siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690's -z loop, which returns bytes, rather than replacing it.

• Unchanged from round 1: the PR's other behaviour, and siblings.py being a pinned family-root copy (upstream-changed for each holder).

On d9b15660, under a stub gh, GH_TOKEN=bogus and an empty GH_CONFIG_DIR, from tests/:

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt -o tmp_path_retention_policy=failed test_doctor_gitmodules_not_utf8.py test_doctor_legs_members_next.py test_doctor_members_spelled.py test_doctor_gitlink_repairs.py test_repo_hygiene.py test_cli_reference.py
325 passed, 2 skipped in 218.80s (0:03:38)   [the 2 skips: the bare-`branch` rows, git 2.43 crashing]

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-sib -o tmp_path_retention_policy=failed test_family_siblings_placement.py test_siblings_family_name.py test_family_makefile_siblings.py test_windows_paths.py test_park_resume_targets.py test_family_printed_quoting.py test_family.py
490 passed in 105.52s (0:01:45)

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-sd -o tmp_path_retention_policy=failed test_shape_doctor.py
436 passed, 3 skipped, 1 xfailed in 339.90s (0:05:39)

Round 3 (head 3fa5beed)

RV-c's re-verification of round 2 is READY. This round takes its SHOULD and two NITs. In shape-doctor.py it changes only GIT_INT and git_bool.

• SHOULD: an integer after white space is a boolean. git_bool (shape-doctor.py:5027-5047) and siblings.py's git_dies_on (:245-264) now match GIT_INT against value.lstrip(" \t\n\v\f\r"), as strtoimax skips that white space.
  • Measured on git 2.43: shallow = " 1" and shallow = "\t1" register p, where both tools said git refuses the file.
  • The words are still compared unstripped, as strcasecmp compares them, so " true" dies (measured).
• NIT: the negative limit follows git 2.55, which CI runs, as the bare branch rule does. The range is -231 to 231 - 1, and both docstrings say so. GIT_INT (shape-doctor.py:5023, siblings.py:236) now captures the sign.
  • git 2.43 stops one short (-max / factor > val), so -2147483648, -2097152k, -2048m and -2g die there (measured). Per RV-c's reading of git's source, git 2.55 takes them.
  • test_733_a_shallow_of_int_min_is_a_boolean_as_git_2_55_reads_it (tests/test_doctor_legs_members_next.py:1729-1746) asserts the doctor and siblings take all four, then asks the git at hand. A git that dies on them skips, with the reason.
  • -2147483647 (taken) and -2147483649 (dies) agree in both versions, so they are plain GIT_SKIPS rows.
• NIT: mutant MB is killed. GIT_SKIPS (:1647-1708, the value rows from :1682) gains seven rows, each checked against git submodule init: 08 (dies), 0x10 (taken), " 1" and "\t1" (taken), " true" (dies), -2147483647 (taken), -2147483649 (dies). The p-then-a-line bytes are one constant, P_THEN_A_LINE (:1638).
  • MB here means GIT_INT takes any digits, read as decimal, so 08 is a boolean. Note that only changing the octal base leaves 08 refused by the regex.
  • In shape-doctor.py and in siblings.py, MB fails shallow-08-no-octal (1 failed, 41 passed, 6 skipped). On round 2's tests it survives (35 passed, 2 skipped).
  • Also killed: dropping the lstrip (2 rows), and 2.43's range in the doctor (the 4 int-min rows).

On 3fa5beed, under a stub gh, GH_TOKEN=bogus and an empty GH_CONFIG_DIR, from tests/:

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt -o tmp_path_retention_policy=failed test_doctor_gitmodules_not_utf8.py test_doctor_legs_members_next.py test_doctor_members_spelled.py test_doctor_gitlink_repairs.py test_repo_hygiene.py test_cli_reference.py
332 passed, 6 skipped in 183.25s (0:03:03)   [2 bare-`branch` rows and 4 int-min rows skip on this git 2.43]

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-sib -o tmp_path_retention_policy=failed test_family_siblings_placement.py test_siblings_family_name.py test_family_makefile_siblings.py test_windows_paths.py test_park_resume_targets.py test_family_printed_quoting.py test_family.py
490 passed in 92.86s (0:01:32)

tests/test_shape_doctor.py was not re-run, as the coordinator allowed: shape-doctor.py changed only in GIT_INT and git_bool. Round 2's run on d9b15660 gave 436 passed, 3 skipped, 1 xfailed.

Round 4: main 40e678e6 merged (#805)

Lane 4's #805 (closes #690) landed on main at 2026-10-17T15:52:45Z as 40e678e6. This round merges it (32048c4f), as "The other half" above planned. Nothing else changes. This PR still lands after #774 and after the v1.1.0 tag.

• The conflict. git merge-tree of main and 3fa5beed conflicts in templates/family-root/scripts/siblings.py alone, in two hunks of submodule_urls: its docstring and its loop.

• The loop is siblings.py: the printed move runs as typed past a taken staging name and names a dangling link to move aside first, a NUL or lone surrogate in project: or url is a finding, .gitmodules is read as git reads it, the warning's prose is spelled, and a tag-only branch is named (#690) #805's, with this PR's rules carried into it. They apply in the order the doctor's submodule_keys applies them to the same -z listing:

  • an empty name, or one with a .. component (DOTDOT_COMPONENT), skips the section;
  • a bare key of VALUED_KEYS, or a value git_dies_on, refuses the whole file: {}. In -z output a key given no value at all is a record with NO newline (submodule.x.url\0). update = is submodule.x.update\n\0, a newline and an empty value (both measured on git 2.43). So "valued" is whether record.partition("\n") found its separator;
  • a path or url beginning with - is ignored, that key alone;
  • a url is keyed by its section's path alone. This is this PR's return line, which merged cleanly.

  Everything else is siblings.py: the printed move runs as typed past a taken staging name and names a dangling link to move aside first, a NUL or lone surrogate in project: or url is a finding, .gitmodules is read as git reads it, the warning's prose is spelled, and a tag-only branch is named (#690) #805's: the later value wins, a name holding a space, a dot, a quote, an = or a CR is read whole, and a byte that is not UTF-8 is replaced (shape-doctor.py submodule_config reads a family holder's .gitmodules through git_out, which decodes strictly, inside except (Refusal, OSError), so a member url that is not UTF-8 raises a UnicodeDecodeError that run_checks does not catch and shape-doctor.py --root ends in a traceback, exit 1 (also the code of a finding), with no verdict, where validate-family.py and siblings.py read the same file as bytes or with errors="replace" #432).

• The docstring keeps both paragraphs. siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690's says how the listing is read. shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a name with a .. component, a url beginning with -) is still read as a member, where git says No url found and exits 128 #733's then says which of it git reads, and now how a bare key shows in -z output.

The loop as merged (siblings.py:316-333):

    for record in proc.stdout.decode("utf-8", "replace").split("\0"):
        # `valued` is the newline: a key given no value at all has none.
        key, valued, value = record.partition("\n")
        section, _, field = key.rpartition(".")
        if not section.startswith("submodule."):
            continue
        name = section[len("submodule."):]
        if not name or DOTDOT_COMPONENT.search(name):
            continue
        if (not valued and field in VALUED_KEYS) or git_dies_on(field, value):
            return {}
        if value.startswith("-"):
            continue
        if field == "path":
            paths[name] = value
        elif field == "url":
            urls[name] = value
    return {paths[name]: url for name, url in urls.items() if name in paths}

• The doctor and make siblings still agree. The merge leaves the doctor's reading as it was. Both suites' parity tests pass: siblings.py: the printed move runs as typed past a taken staging name and names a dangling link to move aside first, a NUL or lone surrogate in project: or url is a finding, .gitmodules is read as git reads it, the warning's prose is spelled, and a tag-only branch is named (#690) #805's test_the_doctor_and_make_siblings_read_one_url_for_one_mount (15 cases) and this PR's test_733_the_doctor_and_make_siblings_skip_what_git_skips.
  • A scratch test, never committed, compared the doctor's submodule_config with siblings.submodule_urls as whole {path: url} maps over 86 files: 86 passed, so no row differs.
  • Its 86 files: every GIT_SKIPS row; the four int-min rows; siblings.py: the printed move runs as typed past a taken staging name and names a dangling link to move aside first, a NUL or lone surrogate in project: or url is a finding, .gitmodules is read as git reads it, the warning's prose is spelled, and a tag-only branch is named (#690) #805's corpus; siblings.py: the printed move runs as typed past a taken staging name and names a dangling link to move aside first, a NUL or lone surrogate in project: or url is a finding, .gitmodules is read as git reads it, the warning's prose is spelled, and a tag-only branch is named (#690) #805's CR row with update = bogus (and with = rebase); its two forged urls and its long url; and validate.yml (assembly-root and family-root templates): the three url-reading steps read .gitmodules by the rules the doctor and siblings.py read it by, so a section git ignores, a section with no path, a path or url beginning with -, an unmerged file or a file git refuses whole no longer puts a repository in the App token or the lists, and a refused file fails the step with its cause (#786) #790's 27 RULES rows.
• "Known differences" under "The other half". The line-break urls and the = names are gone with siblings.py: the printed move runs as typed past a taken staging name and names a dangling link to move aside first, a NUL or lone surrogate in project: or url is a finding, .gitmodules is read as git reads it, the warning's prose is spelled, and a tag-only branch is named (#690) #805. An empty url = and a reopened section stand, as described there.

On 32048c4f, under a stub gh (exits 97), GH_TOKEN=bogus and an empty GH_CONFIG_DIR, from tests/:

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-A -o tmp_path_retention_policy=failed test_siblings_inputs.py test_doctor_gitmodules_not_utf8.py test_doctor_legs_members_next.py test_family_siblings_placement.py
285 passed, 6 skipped in 522.43s (0:08:42)   [2 bare-`branch` rows and 4 int-min rows skip on this git 2.43]

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-B -o tmp_path_retention_policy=failed test_family.py test_family_makefile_siblings.py test_family_printed_quoting.py test_park_resume_targets.py test_siblings_family_name.py test_windows_paths.py test_pins_child_kind_and_siblings_root_spelling.py
471 passed in 321.05s (0:05:21)

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-C -o tmp_path_retention_policy=failed test_update_shape_kept_helper.py test_validator_child_encoding.py test_update_shape_add.py test_repo_hygiene.py test_cli_reference.py test_doctor_members_spelled.py test_setup_family_folder.py
309 passed in 199.06s (0:03:19)

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-D -o tmp_path_retention_policy=failed test_shape_doctor.py
436 passed, 3 skipped, 1 xfailed in 613.21s (0:10:13)

Mutants. Each mutant changes the merged submodule_urls. Each runs against test_siblings_inputs.py and this PR's two test_733_* row tests in test_doctor_legs_members_next.py: 87 tests, of which 5 skip on git 2.43. All six are killed:

• M1, the .. skip dropped: 5 failed (dotdot-name, dotdot-component, dotdot-backslash, bare-path-in-an-ignored-section, update-command-in-an-ignored-section).
• M2, a record with no newline counted as valued: 3 failed (bare-ignore, bare-url, bare-path).
  • bare-update is still refused, through git_dies_on.
  • bare-branch skips on git 2.43 and fails on 2.55.
• M3, the record split at its LAST newline: 9 failed.
  • Three GIT_SKIPS rows: bare-ignore, bare-update and bare-path.
  • siblings.py: the printed move runs as typed past a taken staging name and names a dangling link to move aside first, a NUL or lone surrogate in project: or url is a finding, .gitmodules is read as git reads it, the warning's prose is spelled, and a tag-only branch is named (#690) #805's six line-feed rows: both test_a_url_holding_a_line_feed_forges_no_finding_line, test_a_url_longer_than_gits_message_forges_no_finding_line, both test_the_clone_failure_names_a_line_feed_url_spelled, and test_a_failed_fetch_forges_no_finding_line.
• M4, no - skip: 3 failed (url-option, url-option-after-a-url, path-option-after-a-path).
• M5, no git_dies_on: 8 failed (update-command, update-unknown, shallow-no-boolean, shallow-past-an-int, shallow-08-no-octal, shallow-word-after-a-space, shallow-below-int-min, fetch-recurse-no-boolean).
• M6, a section's name read as its path again (paths.get(name, name)): 1 failed (no-path).

Pins. No file in this repository records a blob, digest or line count of siblings.py. git grep for its blob and sha256 prefixes, before the merge and after it, finds none. The file is still a pinned family-root copy, so each holder takes it as one upstream-changed row. It is now blob fb53c1c8, sha256 855af9a2…, 1461 lines.

#790 (lane 3, #786), checked here and not committed. A scratch worktree of 32048c4f had #790's tests/test_validate_workflow_gitmodules_urls.py (head 31c3792c) overlaid, and the "\r" exclusion removed from test_siblings_reads_what_each_row_says.

• That test gave 27 passed, including its CR row, an update in a section named with a carriage return.
• The whole file gave 224 failed, 197 passed. Every failure is a workflow-step test, which needs validate.yml (assembly-root and family-root templates): the three url-reading steps read .gitmodules by the rules the doctor and siblings.py read it by, so a section git ignores, a section with no path, a path or url beginning with -, an unmerged file or a file git refuses whole no longer puts a repository in the App token or the lists, and a refused file fails the step with its cause (#786) #790's own validate.yml.
• With both templates' validate.yml from validate.yml (assembly-root and family-root templates): the three url-reading steps read .gitmodules by the rules the doctor and siblings.py read it by, so a section git ignores, a section with no path, a path or url beginning with -, an unmerged file or a file git refuses whole no longer puts a repository in the App token or the lists, and a refused file fails the step with its cause (#786) #790 overlaid as well, the file gave 421 passed.

Main has since moved to 6626a3c5 (#747). That commit touches none of this PR's files, and git merge-tree of it with 32048c4f is clean.

Round 5: RV775r4's two SHOULDs and three NITs

RV775r4's review of round 4 (32048c4f, issuecomment-6237014956) is READY. This round takes all five of its items in one commit, 8967b73d. It then merges main 64ec47c3 (#747, #817) cleanly as 44f0bf10, the head.

• SHOULD 1: a section's NAME keeps git's bytes (siblings.py:319-325).

  • submodule_urls decoded the whole -z listing with "replace", so two names that differ only in bytes that are not UTF-8 were one section.

  • [submodule "caf\xe9"] at p beside [submodule "caf\xe8"] at q gave q alone, and p fell back to repository:. With the path in one section and the url in the other, it named a url git does not mount.

  • The listing is now decoded with "surrogateescape", as the doctor's gitmodules_listing decodes it (shape-doctor.py: a .gitmodules section name that is not UTF-8 is looked up by its decoded form, so configured_url and kept_gitdir miss a Latin-1 member and the update line is handed over with no keep line #732). Only the value is re-decoded with "replace" (shape-doctor.py submodule_config reads a family holder's .gitmodules through git_out, which decodes strictly, inside except (Refusal, OSError), so a member url that is not UTF-8 raises a UnicodeDecodeError that run_checks does not catch and shape-doctor.py --root ends in a traceback, exit 1 (also the code of a finding), with no verdict, where validate-family.py and siblings.py read the same file as bytes or with errors="replace" #432): value = value.encode("utf-8", "surrogateescape").decode("utf-8", "replace"). This is RV775r4's measured fix. The docstring says so.

  • test_two_names_apart_only_in_bytes_not_utf8_stay_two_sections sits beside siblings.py: the printed move runs as typed past a taken staging name and names a dangling link to move aside first, a NUL or lone surrogate in project: or url is a finding, .gitmodules is read as git reads it, the warning's prose is spelled, and a tag-only branch is named (#690) #805's parity test in tests/test_siblings_inputs.py, with three rows:

    • the \xe9/\xe8 pair;
    • \xe9 beside the UTF-8 of U+FFFD;
    • the path in one section and the url in the other.

    Each row asks the doctor's submodule_config, submodule_urls and git submodule init on gitlinks at p and q. The three must give the same {path: url}: both mounts in the first two rows, and none in the third.

• SHOULD 2: valued is pinned as the newline. GIT_SKIPS gains empty-ignore and empty-branch after bare-shallow. Each is path = p and url = /r/u.git with an empty ignore = or branch =, which git takes: ("q", "/r/u.git").
• NIT 1: the docstring says a record "is the key and, where it is given a value, a newline and the value".
• NIT 2: it says git submodule init says No url found or bad config line, "or that the value is invalid".
• NIT 3: GIT_SKIPS gains update-dash, P_THEN_A_LINE % (b"q", b"update = -x"), which git dies on: (None, None).

On 44f0bf10, under a stub gh (exits 97), GH_TOKEN=bogus and an empty GH_CONFIG_DIR, from tests/:

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt-A -o tmp_path_retention_policy=failed test_siblings_inputs.py test_doctor_gitmodules_not_utf8.py test_doctor_legs_members_next.py test_family_siblings_placement.py
291 passed, 6 skipped in 310.94s (0:05:10)   [2 bare-`branch` rows and 4 int-min rows skip on this git 2.43]

• validate.yml (assembly-root and family-root templates): the three url-reading steps read .gitmodules by the rules the doctor and siblings.py read it by, so a section git ignores, a section with no path, a path or url beginning with -, an unmerged file or a file git refuses whole no longer puts a repository in the App token or the lists, and a refused file fails the step with its cause (#786) #790's row test. It was run in a scratch worktree of 44f0bf10, with validate.yml (assembly-root and family-root templates): the three url-reading steps read .gitmodules by the rules the doctor and siblings.py read it by, so a section git ignores, a section with no path, a path or url beginning with -, an unmerged file or a file git refuses whole no longer puts a repository in the App token or the lists, and a refused file fails the step with its cause (#786) #790's tests/test_validate_workflow_gitmodules_urls.py (31c3792c) overlaid and the "\r" exclusion dropped. test_siblings_reads_what_each_row_says gives 27 passed.
• Parity, by a scratch test that is not committed. It compares the doctor's submodule_config and submodule_urls as whole maps, round 4's 86 files plus this round's three new GIT_SKIPS rows: 89 passed.
• test_shape_doctor.py was not run: this round does not change shape-doctor.py.

Mutants. Each is in siblings.py unless named. Each runs against test_siblings_inputs.py, test_doctor_gitmodules_not_utf8.py and the two test_733_* row tests: 121 tests, of which 5 skip on git 2.43. All five are killed:

• MB, not valued → not value: 2 failed (empty-ignore, empty-branch). It survived round 4 (RV775r4: 110 passed).
• MB in the doctor's submodule_keys: killed by the same 2 rows.
• MD, the - skip moved before the refusal: 2 failed (update-dash, shallow-below-int-min). Round 4 had only the second.
• MR, the listing decoded with "replace" again (round 4's reading): 3 failed, the three new collision rows.
• MV, the value not re-decoded: 2 failed, both in test_doctor_gitmodules_not_utf8.py: test_submodule_config_keeps_the_url_as_text_it_can_compare[not-utf-8] and test_a_clone_made_from_the_same_bytes_is_the_working_clone.

Pins. No file in this repository records a blob, digest or line count of siblings.py. It is now blob 149c752a, sha256 5cceb2dd…, 1468 lines.

Main has moved again, to 431c8cf8 (#748, #662), at 17:00:58Z, after this round's merge. I did not merge it, because its merge with 44f0bf10 is not clean:

• It conflicts in one hunk of tests/test_doctor_legs_members_next.py, the import block.
• Main splits the conftest import over two lines to add submodule_update_through_git. This branch adds from test_doctor_gitmodules_not_utf8 import listed_locale under that import, a line from shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's 311ad7d9.
• A resolution would keep both. shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's head 93b9bac3 meets the same conflict with that main.

Round 6: #774's new head and main merged

#774's head moved to 012a7726, a merge of main bad05d21 that resolved the import block of tests/test_doctor_legs_members_next.py (#748's two-line conftest import, then listed_locale). Main has moved on to fc867207 (#672, #746). This round merges both into this branch. Nothing else changes: this PR is still a draft, and still lands after #774 and after the v1.1.0 tag.

• shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's head, 1e31d5f9 (parents 44f0bf10 and 012a7726): clean. This branch changed none of the import-block lines, so the file takes shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's side as it is. Its first lines now read as shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's resolution has them: main's two-line conftest import, then from test_doctor_gitmodules_not_utf8 import listed_locale.
• Main fc867207, 8c5ad0e8, the head (parents 1e31d5f9 and fc867207): clean. fc867207 changes scripts/bump-leg.py and four test files, none of which this PR touches.
• No file conflicted in either merge, and no hand edit was made in either.

On 8c5ad0e8, under a stub gh (exits 97), GH_TOKEN=bogus and an empty GH_CONFIG_DIR:

/usr/bin/python3 -m pytest -q -p no:cacheprovider --basetemp=<scratch>/bt -o tmp_path_retention_policy=failed tests/test_siblings_inputs.py tests/test_doctor_gitmodules_not_utf8.py tests/test_doctor_legs_members_next.py tests/test_family_siblings_placement.py
291 passed, 6 skipped in 307.75s (0:05:07)   [the same count as round 5's `44f0bf10`; the 6 are the same 2 bare-`branch` rows and 4 int-min rows that skip on git 2.43]

tests/test_shape_doctor.py was not run on this head. It was run once on #774's 012a7726: 436 passed, 3 skipped, 1 xfailed.

🤖 Generated with Claude Code

Summary by Sourcery

Match .gitmodules readers to Git’s parsing and submodule URL resolution behavior across malformed, ignored, unmerged, and encoding-sensitive configurations.

Bug Fixes:

• Align .gitmodules parsing in the doctor and make siblings with Git’s handling of ignored sections, invalid values, malformed entries, and unmerged files.
• Prevent submodule URLs from being inferred for sections without a path or from incorrectly merged non-UTF-8 section names.

Enhancements:

• Handle Git’s partial parsing of index and HEAD .gitmodules copies while preserving entries read before a malformed line.
• Add comprehensive parity coverage against Git for .gitmodules edge cases, including boolean and integer value validation and byte-distinct section names.

Tests:

• Expand .gitmodules tests to cover Git-skipped sections, refused files, unmerged copies, partial index/HEAD reads, and encoding-sensitive section names.

[sourcery-ai]

Reviewer's Guide

Updates the doctor and family siblings.py readers to reproduce Git’s .gitmodules semantics, including ignored or malformed configuration, unmerged files, explicit mount paths, and byte-preserving section names, with broad parity and regression tests.

Sequence diagram for Git-compatible submodule URL lookup

sequenceDiagram
    participant Caller
    participant Doctor as shape-doctor.py
    participant Siblings as siblings.py
    participant Git
    Caller->>Doctor: gitmodules_entry(root, rel)
    Doctor->>Git: ls-files -u -- .gitmodules
    Git-->>Doctor: unmerged status
    alt .gitmodules is unmerged
        Doctor-->>Caller: (None, None)
    else file is readable
        Doctor->>Git: config -z --file .gitmodules --list
        Git-->>Doctor: section key/value records
        Doctor->>Doctor: submodule_keys(listing)
        Doctor-->>Caller: matching path and url
    end
    Caller->>Siblings: submodule_urls(root)
    Siblings->>Git: config --file .gitmodules --list
    Git-->>Siblings: section key/value records
    Siblings->>Siblings: skip ignored sections and keys
    Siblings-->>Caller: urls keyed by explicit path

Loading

Entity relationship diagram for .gitmodules mount mapping

erDiagram
    GITMODULES_SECTION {
        string name
        string path
        string url
    }
    MOUNTED_SUBMODULE {
        string path
        string url
    }
    GITMODULES_SECTION ||--o| MOUNTED_SUBMODULE : "maps only when path exists"

Loading

Flow diagram for Git-compatible .gitmodules parsing

flowchart TD
    A[".gitmodules"] --> B["gitmodules_listing / config listing"]
    B --> C{".gitmodules unmerged?"}
    C -->|yes| D["gitmodules_entry returns (None, None)"]
    C -->|no| E["submodule_keys / submodule_urls"]
    E --> F{"Git ignores section or key?"}
    F -->|section name empty or .. component| G["skip section"]
    F -->|path or url starts with -| H["skip key"]
    F -->|bare valued key| I["refuse whole file"]
    F -->|valid entry| J["retain path and url"]
    I --> K["submodule_config returns {} / submodule_urls returns {}"]
    J --> L["match url to explicit path"]
    L --> M["doctor and siblings agree with Git"]

Loading

File-Level Changes

Change	Details	Files
Make .gitmodules parsing match Git’s handling of ignored sections, ignored option-like values, malformed valued keys, and non-UTF-8 section names.	Skip empty or ..-component section names and path/url values beginning with -. Return a refused-file signal when valued keys are bare, and propagate it as no configuration. Preserve section-name bytes with surrogateescape and pass them back to Git safely across locales and platforms.	shape-doctor.py
Align sibling URL discovery with the doctor and Git by requiring a valid path and mirroring Git’s filtering rules.	Reject ignored sections, option-like values, and whole-file parse failures. Map URLs only through sections that explicitly provide a path. Keep the copied parser self-contained for older family holders.	templates/family-root/scripts/siblings.py
Handle unmerged .gitmodules files as unreadable and add regression coverage for Git parity and byte-preserving names.	Return no entry while .gitmodules has unmerged index stages, regardless of the working-tree copy. Exercise ignored sections, duplicate/option-like keys, missing paths, bare keys, and controls against git submodule init. Add locale and non-UTF-8 section-name coverage for Git lookups and repository retention.	tests/test_doctor_legs_members_next.py tests/test_doctor_gitmodules_not_utf8.py

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#733	Make the doctor and make siblings read .gitmodules according to Git’s behavior: ignore empty or ..-component section names and --prefixed path/url values, treat files Git rejects for bare valued keys as having no entries, and do not treat a section name as its mount path when path is absent.	✅
#733	Make the doctor ignore .gitmodules copies while the file is unmerged, matching Git’s behavior.	✅
#733	Pin the relevant Git-parity behavior with tests covering both tools and the issue’s cases.	✅

Possibly linked issues

• shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a name with a .. component, a url beginning with -) is still read as a member, where git says No url found and exits 128 #733: The PR implements the issue’s requested Git-parity fixes and tests, while extending coverage to closely related .gitmodules cases.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[brettheap]

(lane openRepoShape-3, 2026-10-17T04:15:39Z) Independent review of #775 by RV-c (opus), on head 08ebe8bf: NOT READY. There is one MUST: a regression against main on an index or HEAD copy. Three SHOULDs and one NIT. RV-c measured on git 2.43 and diffed git's source between v2.43.0 and v2.55.0 (CI's git).

MUST: a bare valued key in an index or HEAD copy now hides every section before it.

• Where: shape-doctor.py:5051 (if not valued and field in VALUED_KEYS: return None), as used by gitmodules_entry (:4959).
• Scenario: the working-tree .gitmodules is deleted, so git reads the index or HEAD copy. That copy has p's section, then a section with a bare url or path.
• What happens:
  • git: git submodule init -- p exits 0 and registers /r/p.git, printing only error: missing value for 'submodule.q.url'.
  • Main's doctor: gives ('p', '/r/p.git') and routes it to checkout.
  • shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a .. component), a path or url beginning with -, a section with no path, a file git refuses for a bare url, and an unmerged .gitmodules name no mount url, NEXT line or clone, where git says No url found and exits 128 (#733) #775: gives (None, None), routes it to no-url, and url_less_members reports it.
• Why: git reads a working-tree file with CONFIG_ERROR_DIE, but an index or HEAD copy with CONFIG_ERROR_ERROR. It stops at the bad line and keeps what it read before it. This is identical at v2.43.0 and v2.55.0 (config.c:1393/1448). So "git refuses the whole file" holds for the working-tree copy only.
• Fix:
  • Let submodule_keys know which copy it reads. A --file copy keeps returning None at the first bare valued key; an index or HEAD copy returns the keys read before it.
  • Thread that through section_url from gitmodules_entry. submodule_config and siblings.py read only the working tree.
  • Add index-copy rows checked against git submodule init: a bare key after p gives p's url; a bare key before p gives none.

SHOULDs:

1. Values git dies on are not caught. These make git exit 128 for every path, in the working-tree file and in an index copy alike, because each calls die() in both versions:

   • update other than none, checkout, rebase or merge (including !command);
   • a shallow that is not boolean;
   • a fetchRecurseSubmodules that is neither boolean nor on-demand.

   The doctor and siblings still name /r/p.git. Fold these in (None for any copy), or file them before shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a name with a .. component, a url beginning with -) is still read as a member, where git says No url found and exits 128 #733 closes.

2. Already on main, from shape-doctor.py: submodule_config and gitmodules_entry read a holder's .gitmodules through one git config -z --list, so a section name holding a space or a byte that is not UTF-8, or a url holding NEL or U+2028, gives the doctor the mount url git and make siblings use, under any locale (#668) #700: in an unparsable index copy whose bad line comes after p, git reads p but the doctor gives (None, None). git config -z --blob :.gitmodules --list prints the partial listing and then exits 128, and git_out throws that output away. For an index or HEAD copy known to exist, use it. This fits the MUST fix.

3. Name the one version difference in the body: a bare branch.

   • git 2.43: xstrdup(NULL) crashes, exit 139.
   • git 2.55: config_error_nonbool gives exit 128 for the working tree, and an error that keeps earlier sections for an index or HEAD copy.

   Everything else GIT_SKIPS relies on is identical in both versions, and CI (2.55.0) passed all 22 rows. The new index-copy rows will need branch handled per version.

NIT: the #690 conflict. Lane 4's fix/690-siblings-inputs (38a40abf) conflicts in siblings.py in two hunks, the docstring and the loop. #690's -z call returns bytes, so taking #775's loop as it stands would raise TypeError (line.partition("=") on bytes) and break make siblings. Put #775's rules into #690's -z loop, as the body says.

Verified against git:

• every GIT_SKIPS row and the unmerged cases, through the doctor and siblings;
• DOTDOT_COMPONENT matches check_submodule_name;
• bare keys in the working-tree file, including one inside an ignored section;
• the unmerged guard matches is_gitmodules_unmerged;
• mounted_from and siblings no longer take a name as a path;
• the docstrings carry shape-doctor.py: submodule_config and gitmodules_entry read a holder's .gitmodules through one git config -z --list, so a section name holding a space or a byte that is not UTF-8, or a url holding NEL or U+2028, gives the doctor the mount url git and make siblings use, under any locale (#668) #700's caveats;
• siblings.py is FAMILY_COPIED_VERBATIM, as the body says, and there is no static digest to bump.

Mutants: all killed:

• drop .. (4 rows);
• -url valid (2);
• read an unmerged .gitmodules (2);
• siblings keys a path-less section by its name (1);
• .. dropped in siblings only (4).

No test covers the MUST, because no row reads an index or HEAD copy.

Tests: main + #775:

• 8 doctor files: 418 passed;
• 7 siblings and family files: 490 passed;
• tests/test_shape_doctor.py: 436 passed, 3 skipped, 1 xfailed;
• with siblings.py: a file, or a folder make siblings cannot list or enter, at ../<Project> is NOT A CLONE, reported and left as it is, so the run no longer ends in a traceback at iterdir before the members after it are placed, and make park no longer ends in one over a folder it cannot enter (#735) #772 too: 488 passed.

Merges: clean with bf1f099b and every open lane-3 v1.1.1 PR. It conflicts with #690's WIP (NIT).

Lane: openreposhape-3 (openRepoShape-3)

[brettheap]

(lane openRepoShape-3, 2026-10-17T04:47:25Z) RV-c's re-verification of #775 round 2 (d9b15660): READY. The MUST is fixed, measured against git in both orders. SHOULD 1 and 2 hold on 56 of 58 cases against git 2.43. One new SHOULD and two NITs, none blocking; lane 3 takes them in a small round 3.

The MUST (an index copy, no working-tree file, against git submodule init -- p):

case	git	round 1	round 2
bare url after p	/r/p.git	none	/r/p.git
bare url before p	exit 128	none	none
bare path after p	/r/p.git	none	/r/p.git
bare update after p	/r/p.git	none	/r/p.git
bare url inside p, after its url	/r/p.git	none	/r/p.git

The same bare url in the working-tree file is still refused whole, as git dies on it.

SHOULD 1 holds:

• update: !x and Checkout die; none and merge pass.
• fetchRecurseSubmodules: ON-DEMAND, bogus and 08 die; on-demand, 2k and yes pass.
• shallow: 25 values all match git.
• The order in an index copy is followed.
• siblings.py matches the doctor on every working-tree row.

SHOULD 2 never keeps a record git would not. It agreed with git on all nine broken-line cases: an unparsable line before or after p, = nokey inside p, continuations, an unterminated quote, a bad escape, and a bare key followed by an unparsable line. --list and the submodule reader share the parser and its stop-on-error rule for an index or HEAD copy.

SHOULD 3 holds. The two bare-branch rows check the doctor's own answer, then skip on 2.43 ("exit 139"). Mutant MA shows the doctor side runs.

Not blocking:

1. SHOULD (new): a quoted value with leading whitespace is wrongly refused.
   • With shallow = " 1" or "\t1", git registers p (strtoimax skips leading whitespace). The doctor and siblings answer "git refuses the file".
   • Fix, in git_bool and siblings.py's git_dies_on: GIT_INT.fullmatch(value.lstrip(" \t\n\v\f\r")). Keep the word compare on the raw value.
2. NIT: the negative limit differs by version.
   • git 2.43 checks -max / factor > val.
   • git 2.55 checks (-max - 1) / factor > val, so it accepts -2147483648, -2097152k, -2048m and -2g.
   • The code follows 2.43, while the bare-branch rule follows 2.55. Pick one policy.
3. NIT: mutant MB (git_bool accepts 08) survives. Add shallow = 08 (dies) and shallow = 0x10 (accepted).

Mutants:

mutant	result
MA working_tree ignored	3 killed
MB	survives (NIT 3)
MC partial listing dropped	1 killed
MD siblings never sees a die value	5 killed

Tests (main + #775):

• 8 doctor files: 431 passed, 2 skipped;
• 7 siblings and family files: 490 passed;
• tests/test_shape_doctor.py: 436 passed, 3 skipped, 1 xfailed.

git 2.55 from source: git_parse_maybe_bool_text, git_parse_int, get_unit_factor and git_config_bool are identical to 2.43, apart from NIT 2.

Merges:

• clean with bf1f099b and every open lane-3 PR;
• lane 4's siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690 still conflicts in siblings.py's docstring and loop. The resolution must carry git_dies_on into siblings.py (make siblings): the printed mv goes astray over an existing .holder/.folder or a dangling link, a lone surrogate or NUL in project: is a traceback, a url with NEL or U+2028 is cut, two warnings print paths raw #690's -z loop.

Lane: openreposhape-3 (openRepoShape-3)

[brettheap]

(lane openRepoShape-3, 2026-10-17T04:59:04Z) RV-c's re-verification of #775 round 3 (3fa5beed): READY. One optional NIT. Linux tests was pending at 04:59Z; Sonar, package and tests-git-bash pass.

Leading whitespace is fixed. shallow = " 1" and "\t1" now register p in the doctor and siblings alike, against git submodule init. " true" still dies, as in git.

Mutant MB is killed. Widening GIT_INT to accept 08 in either file fails [shallow-08-no-octal]. The head's selection gives 42 passed, 6 skipped: two bare-branch rows and four range rows only git 2.55 accepts.

The 2.55 range is exact at both edges, with and without a unit.

• From source: git_parse_signed takes val·f < 2^31 and val·f ≥ −2^31. The check is -2**31 <= ±scaled < 2**31, with the sign read as strtoimax reads it.
• Measured on 2.43, which shares the positive bound:
  • 2147483647, 0x7fffffff, 2047m and 2097151k are taken;
  • 2147483648, 0x80000000, 2048m and 2097152k die;
  • -2147483649, -2049m and -2097153k die.
• Only five values differ by version: -2147483648, -0x80000000, -2097152k, -2048m and -2g.

The version-skip test checks the doctor's answer and siblings' url before asking git.

• NIT (optional): a git that dies on these values always skips, so a 2.55 that rejected them would pass silently. On git 2.55 or later, assert that git registers /r/p.git instead of skipping.

Merges: clean with bf1f099b and with #774 (93b9bac3, an ancestor).

Lane: openreposhape-3 (openRepoShape-3)

[brettheap]

(lane openRepoShape-3, 2026-10-17T16:51:18Z) Independent review of #775's round 4 (head 32048c4f, main 40e678e6 merged) by lane 3's RV775r4 (opus): READY.

• No MUST. Two cheap SHOULDs, each with a measured fix, and three NITs. Lane 3 takes all five in round 5.
• The merge resolution is correct, and each side keeps its own behaviour.
• Combined, the two sides fix rows that round 3's head got wrong:
  • a bare url in [submodule "a=b"] now refuses the whole file, as git does;
  • a url holding CR, LF, NEL or U+2028 is read whole;
  • a .. name only the -z reading sees whole is skipped;
  • update = "rebase\n" dies, as in git.
• The head did not move during the review. Main moved to 64ec47c3, and the merge with it is clean.

The resolution, against real git 2.43 (40 rows). Each row compares git config -z --list -f, git submodule init, the merged submodule_urls and the doctor's readers. All 40 agree, except three non-UTF-8 rows (SHOULD 1).

• Only siblings.py was resolved by hand; every other file in the merge is git's own auto-merge.
• A bare url lists with no newline, and git exits 128 (bad config line). The merged code gives {}; main gave {'p': ''}.
• url = lists with a newline and an empty value, and git registers ''. The merged code gives {'p': ''}.

Doctor parity over 129 files (both suites' rows, #790's RULES, DUPLICATES and holder corpus, and the 40 rows above):

• The maps are equal in 126. The exceptions are the 3 non-UTF-8 name rows (SHOULD 1).
• The mounts are equal in 124. The difference is the 2 empty-url = rows, the known difference the body states, and the 4 int-min rows, where git 2.43 dies and both tools follow 2.55 by design.
• F775's 86 files are all equal.

SHOULD 1: a non-UTF-8 name collision splits the two tools (siblings.py:316-318).

• Cause: submodule_urls decodes the whole listing with "replace", so two names that differ only in non-UTF-8 bytes become one. The doctor keeps names as surrogateescape (shape-doctor.py: a .gitmodules section name that is not UTF-8 is looked up by its decoded form, so configured_url and kept_gitdir miss a Latin-1 member and the update line is handed over with no keep line #732, through shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774).
• Measured:
  • [submodule "caf\xe9"] p and [submodule "caf\xe8"] q: git and the doctor give both, while submodule_urls gives only q, so p falls back to repository:.
  • A path in one and a url in the other: git says No url found, while submodule_urls names a url git does not mount.
  • Round 3 behaved the same way, so this is not a round-4 regression.
• Fix: decode with "surrogateescape", then re-decode only the value with "replace" (value = value.encode("utf-8", "surrogateescape").decode("utf-8", "replace")). With it the maps are equal in 129 of 129, and 137 passed, 5 skipped. Add the collision rows to siblings.py: the printed move runs as typed past a taken staging name and names a dangling link to move aside first, a NUL or lone surrogate in project: or url is a finding, .gitmodules is read as git reads it, the warning's prose is spelled, and a tag-only branch is named (#690) #805's parity test.

SHOULD 2: valued as the newline is unpinned on one side.

• Mutant not valued → not value makes an empty ignore = or url = refuse the whole file. It survives every test (110 passed), although git takes ignore = and branch = and registers the url.
• Fix: two GIT_SKIPS controls after bare-shallow (tests/test_doctor_legs_members_next.py:1678), empty-ignore and empty-branch, each expecting ("q", "/r/u.git"). Both pass on the head and fail under the mutant.

NITs:

1. siblings.py:281-283 says "is the key, a newline and the value". Say "the key and, where it is given a value, a newline and the value".
2. siblings.py:302-304: for a value git dies on, git says invalid value for …, bad boolean config value … or bad submodule.p.fetchrecursesubmodules argument. Add "or that the value is invalid".
3. Mutant MD (the - skip before the refusal) is killed only by shallow-below-int-min. Add the row P_THEN_A_LINE % (b"q", b"update = -x") → (None, None).

Runs (stub gh first on PATH, GH_TOKEN=bogus, empty GH_CONFIG_DIR):

run	result
test_siblings_inputs.py, test_doctor_gitmodules_not_utf8.py, test_doctor_legs_members_next.py, test_family_siblings_placement.py	285 passed, 6 skipped
test_windows_paths.py, test_family_makefile_siblings.py, test_siblings_family_name.py, test_pins_child_kind_and_siblings_root_spelling.py	237 passed
#790's row test, CR exclusion dropped	27 passed

Mutants:

mutant	result
MA: refusal before the ignored-section skip	killed (2)
ME: return {} → break	killed (10)
MD: - skip moved before the refusal	killed by one row only (NIT 3)
MB: not valued → not value	survives (SHOULD 2)
F775's M2, M3 and M6, re-measured	match its report

Merges: clean against main 6626a3c5 and 64ec47c3, against #790, and against every open lane-3 PR that touches siblings.py or shape-doctor.py. #774 is an ancestor of the head.

The Round 4 section: every claim holds, except that "the doctor and make siblings still agree" holds for every suite row but not for the non-UTF-8 collisions (SHOULD 1).

CI on 32048c4f: SonarCloud, package and tests-git-bash pass; tests is pending.

Lane: openreposhape-3 (openRepoShape-3)

[brettheap]

(lane openRepoShape-3, 2026-10-17T17:14:49Z) RV775r4's re-check of #775 round 5 (head 44f0bf10): READY. All five items are taken as asked, nothing is left, and nothing new was found. 44f0bf10's tree is git's own clean merge of 8967b73d with main 64ec47c3.

The items:

item	where	what
SHOULD 1	siblings.py:319, :324-325	The listing is decoded with "surrogateescape" and only the value is re-decoded with "replace". The docstring (:292) says so.
SHOULD 1 test	tests/test_siblings_inputs.py:650-714	test_two_names_apart_only_in_bytes_not_utf8_stay_two_sections
SHOULD 2	tests/test_doctor_legs_members_next.py:1683, :1685	rows empty-ignore and empty-branch
NIT 1	siblings.py:281	docstring
NIT 2	siblings.py:307	docstring
NIT 3	tests/test_doctor_legs_members_next.py:1693	row update-dash

Against real git:

• The three collision rows: git submodule init, the doctor and make siblings give the same {path: url} in each:
  • caf\xe9 beside caf\xe8 gives both mounts;
  • caf\xe9 beside U+FFFD's UTF-8 gives both mounts;
  • a path in one section and a url in the other gives No url found, so {}.
• Three more rows agree: a Latin-1 url value, update = rebas\xe9 (git says invalid value), and a -\xe9 url after a url.

Parity: the maps are equal in 132 of 132 files (RV775r4's 129 plus round 5's three new rows). The mounts are equal in 130; the two that differ are the known empty-url = rows.

Mutants on the new decode:

mutant	result
N1: the whole listing decoded as latin-1	killed (10)
N2: the value re-encoded with "replace"	killed (2)
N3: the value re-decoded with "ignore"	killed (2)
F775's MR, re-measured	matches its report (3)

Runs (stub gh first on PATH, GH_TOKEN=bogus, empty GH_CONFIG_DIR):

run	result
group A	291 passed, 6 skipped
group B	237 passed
test_repo_hygiene.py	77 passed
#790's row test, CR exclusion dropped	27 passed

SonarCloud's gate is OK, with 0.0% new duplication.

Main: #748 (431c8cf8) conflicts with #775, and with #774 (93b9bac3), in ONE hunk: the import block of tests/test_doctor_legs_members_next.py. The fix is mechanical: keep main's two-line conftest import, then the listed_locale line. With that, the whole file gives 181 passed, 6 skipped, and the result still holds against d66a9054.

• Lane 3 takes this after the v1.1.0 tag: shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774 merges main first, then shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a .. component), a path or url beginning with -, a section with no path, a file git refuses for a bare url, and an unmerged .gitmodules name no mount url, NEXT line or clone, where git says No url found and exits 128 (#733) #775 takes shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's squash and main.
• The 13 lane-3 PRs that touch siblings.py or shape-doctor.py, and validate.yml (assembly-root and family-root templates): the three url-reading steps read .gitmodules by the rules the doctor and siblings.py read it by, so a section git ignores, a section with no path, a path or url beginning with -, an unmerged file or a file git refuses whole no longer puts a repository in the App token or the lists, and a refused file fails the step with its cause (#786) #790, all merge cleanly with 44f0bf10.

Lane: openreposhape-3 (openRepoShape-3)

[brettheap]

(lane openRepoShape-3, 2026-10-17T19:24:01Z) RV775r4's check of the main merge into #775 (head 8c5ad0e8): READY, with nothing left.

• 1e31d5f9 (shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's new head 012a7726 merged in) and 8c5ad0e8 (main fc867207 merged in) are each exactly git's clean merge-tree result. shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774 is inside shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a .. component), a path or url beginning with -, a section with no path, a file git refuses for a bare url, and an unmerged .gitmodules name no mount url, NEXT line or clone, where git says No url found and exits 128 (#733) #775. Group A gives 291 passed, 6 skipped (git 2.43's known skips), and the merge with validate.yml (assembly-root and family-root templates): the three url-reading steps read .gitmodules by the rules the doctor and siblings.py read it by, so a section git ignores, a section with no path, a path or url beginning with -, an unmerged file or a file git refuses whole no longer puts a repository in the App token or the lists, and a refused file fails the step with its cause (#786) #790 is clean.
• Nothing else entered the PR: its own change against main is identical before and after the merge (diff exit 0).
• Merges: clean against main fc867207, and against every open lane-3 PR that touches siblings.py or shape-doctor.py.
• CI: SonarCloud's gate is OK, with 0.0% new duplication; tests was still running at 19:23Z.

This lands after the v1.1.0 tag, under Brett Heap's word "merge lane 3's v1.1.1 drafts after the v1.1.0 tag".

Lane: openreposhape-3 (openRepoShape-3)

[brettheap]

(lane openRepoShape-3, 2026-10-18T01:31:48Z) #775 is restacked after #774's squash (52ff596), as 7f6ee36b, by lane 3's three-merge recipe (writer R775, sonnet-xhigh):

1. 0f825d8c merges shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's final head eb595e30. It is clean, and it brings in main since fc867207, tests/test_doctor_legs_members_next.py and 24 other files: tests append what they tell git to the GIT_CONFIG_COUNT entries the environment holds, so a pasted line keeps the session's maintenance settings and the Windows job's two (#758) #818's paste_env() among it.
2. 7f6ee36b merges the squash 52ff5962, taking ours only in shape-doctor.py, one of shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's files. The squash and the final head have the same tree, and this merge changed nothing.
3. Main was 52ff5962 itself, so the third merge was "Already up to date".

• The delta: git diff --name-only origin/main 7f6ee36b lists only shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a .. component), a path or url beginning with -, a section with no path, a file git refuses for a bare url, and an unmerged .gitmodules name no mount url, NEXT line or clone, where git says No url found and exits 128 (#733) #775's four files. Its -U0 changed lines equal the reviewed delta, taken from shape-doctor.py: a .gitmodules section name keeps git's own bytes and is handed back to git as them, so configured_url and kept_gitdir find a member named caf\xe9, its detached commit gets its keep line under any locale, and a non-ASCII name no longer ends --root in a UnicodeEncodeError traceback under LC_ALL=C PYTHONUTF8=0 (#732) #774's old head with main fc867207 (tree 075395ab) to 8c5ad0e8: 596 lines on each side.
• Tests on 7f6ee36b, under a stub gh:
  • shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a .. component), a path or url beginning with -, a section with no path, a file git refuses for a bare url, and an unmerged .gitmodules name no mount url, NEXT line or clone, where git says No url found and exits 128 (#733) #775's gate set, 707 passed, 6 skipped. The skips are git 2.43 crashes the rows guard against.
  • tests/test_shape_doctor.py, 439 passed, 3 skipped.
  • tests/test_validate_workflow_gitmodules_urls.py, 421 passed. All 27 siblings rows that skip on main ran and passed with shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a .. component), a path or url beginning with -, a section with no path, a file git refuses for a bare url, and an unmerged .gitmodules name no mount url, NEXT line or clone, where git says No url found and exits 128 (#733) #775's siblings.py.
• Main has since moved to ca47b71 (adopt-project.py: _default_branch reads origin/HEAD and HEAD by full name and whole, and reads the source only at a branch of its own, so a git clone -b source is no longer git-failed with no remedy, and a HEAD on origin/x, on main<U+00A0> or beside a tag main no longer adopts another branch's commit (#797) #830, install.sh and openRepoShape: a $OPENREPOSHAPE_BIN_DIR, the repository or a ref holding a bidi control, a character of no width, a soft hyphen or U+2028 is refused by its code point before any request and never echoed, so no line shows the value as another #841), with no shared file. The squash tree over it is 202bbbc3.
• shape-doctor.py and siblings.py: a .gitmodules section git ignores ([submodule ""], a .. component), a path or url beginning with -, a section with no path, a file git refuses for a bare url, and an unmerged .gitmodules name no mount url, NEXT line or clone, where git says No url found and exits 128 (#733) #775 lands at a lane-3 turn by fast refresh, once its CI on 7f6ee36b is green.

Lane: openreposhape-3 (openRepoShape-3)

[sonarqubecloud]

Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sourcery-ai]

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 3 days and 12 hours by commenting @sourcery-ai review. Upgrade to get a review now.