Skip to content

repo_shape.py and bootstrap.py: COMMIT_RE, SHA256_RE and COMMIT_40 end in \Z, not $, so a pin's commit: or tree_sha256: that ends in a line break is refused as not 40 or 64 hex by every validator and adopt-project.py (#1017) - #1126

Merged
brettheap merged 4 commits into
mainfrom
fix/1017-repo-shape-match-end
Oct 23, 2026
Merged

repo_shape.py and bootstrap.py: COMMIT_RE, SHA256_RE and COMMIT_40 end in \Z, not $, so a pin's commit: or tree_sha256: that ends in a line break is refused as not 40 or 64 hex by every validator and adopt-project.py (#1017)#1126
brettheap merged 4 commits into
mainfrom
fix/1017-repo-shape-match-end

Conversation

@brettheap

@brettheap brettheap commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Lane: openreposhape-5 (openRepoShape-5)
Claim: #1017 (comment)

Goal

scripts/repo_shape.py ends COMMIT_RE and SHA256_RE in $, and every caller reads them with .match. In Python $ also matches just before a trailing line feed, so a pin's commit: or tree_sha256: that ends in a line break passed as well formed. This makes both patterns refuse it, wherever a value is decided to be valid, and changes nothing else about them.

What still held on main

Measured on main (2a119f28, then re-checked on 6da757cbe; the use sites of both patterns are the same on 823624d93, and no new copy of either pattern was added). Python 3.12.3:

pattern value .match on main
COMMIT_RE / SHA256_RE 40 / 64 hex a match
same hex and a line feed a match (the defect)
same hex and a carriage return and line feed None
same hex and a space None

A line feed reaches the patterns through YAML: the double-quoted escape "<hex>\n" and a block scalar both read as the hex and a line feed. On main:

  • validate-manifest.py: shape.commit and shape.digests.tree_sha256 ending in a line feed -> manifest ok, exit 0.
  • validate-family.py: shape.digests.tree_sha256 in family.yaml ending in a line feed passed (family ok, exit 0). shape.commit did not pass: it exited 1 with shape-pin-manifest-disagree, because contracts/shape-pin.yaml and family.yaml's shape.commit differ only by the line feed, so the same 40 hex is printed on both sides; the finding that names the commit as not 40 hex, family-shape-commit, was not raised. A member pin's tree_sha256 ended in a line feed gave member-digest-mismatch, not the malformed finding, and a member pin's commit gave member-commit-unresolvable, not member-pin-commit.
  • validate-pins.py: a leg pin's commit gave exit 2 pin-commit-unresolvable, not pin-tag-only; a leg digest gave pin-digest-mismatch, not pin-digest-malformed; a shape pin's commit gave only shape-pin-manifest-disagree, not shape-pin-commit.
  • adopt-project.py check (_pin_row_problem): the row read as well formed, with no plan-pin-malformed.
  • templates/assembly-root/scripts/bootstrap.py holds a copy of the commit pattern, COMMIT_40, with the same defect: shape_upstream_notice printed its notice for a pin ending in a line feed.
  • shape-doctor.py pin_commit returned "<hex>\n" for such a leg pin.

Checked and not affected: bump-leg.py --to and family.py --at/--to go through checked_value (a fullmatch that already refuses a line feed); bump-leg.py line 2212 strips first; update-shape.py line 558 reads git_out, which is stripped; shape-doctor's recorded_gitlink splits fields; src/openreposhape/standard.py uses fullmatch. The @(?P<sha>[0-9a-fA-F]{40}) patterns in bump-leg.py and validate-pins.py are unanchored workflow-ref patterns.

What changed

  • scripts/repo_shape.py: COMMIT_RE and SHA256_RE end in \Z, not $ (comment added). \Z matches only at the very end of the value, so each caller's .match refuses a line feed as it already refused a carriage return and line feed, and a space. 40 and 64 digits, either case, are unchanged.
  • templates/assembly-root/scripts/bootstrap.py: COMMIT_40 likewise (comment added).
  • Nothing else in production changes. No README, doc, handbook or AGENTS.md text pins the old pattern bytes (grepped), and tests/test_pinned_copy_text.py and every tests/test_bootstrap*.py pass unchanged (shard tB and shard tA).
  • Rows:
    • new tests/test_pin_pattern_line_break.py (46 rows: 42 at first, and 4 added in fix round 1, below): the three patterns x case, length, after-suffix (line feed, carriage return and line feed, space, two line feeds, carriage return, tab) and before-prefix; the holder's gate (four rows and a baseline); the bootstrap upstream notice (two rows); the doctor's pin_commit (two rows);
    • tests/test_pin_validator.py: 3 rows (leg commit, leg digest, shape-pin commit);
    • tests/test_manifest_validator.py: 2 rows (shape.commit, shape digest) and a helper shape_of;
    • tests/test_adopt_pins.py: 2 malformed() cases (commit-line-break, digest-line-break);
    • tests/test_shape_doctor.py: 1 ROOT_PIN_FAULTS param (a-commit-ending-in-a-line-break).

Rows that fail on main

Run with the two production files of main checked out, on a head with the same tree as this one (git 2.43.0, EMAIL=service@opensoft.dev):

  • the new file, test_pin_validator.py, test_manifest_validator.py, test_adopt_pins.py: 16 failed, 307 passed in 294.83s. The 16:
    • new file (9): line-feed-COMMIT_RE, line-feed-SHA256_RE, line-feed-bootstrap-COMMIT_40, test_a_member_pin_commit_that_ends_in_a_line_break_is_refused, test_a_member_pin_digest_that_ends_in_a_line_break_is_malformed, test_a_family_shape_commit_that_ends_in_a_line_break_is_a_finding, test_a_family_shape_digest_that_ends_in_a_line_break_is_a_finding, test_the_upstream_notice_is_silent_over_a_pin_that_ends_in_a_line_break, test_the_doctor_reads_a_pins_commit_as_the_validators_do[40-hex-and-a-line-feed];
    • test_pin_validator.py (3): a commit, a digest and a shape-pin commit that end in a line break;
    • test_manifest_validator.py (2): a shape commit and a shape digest;
    • test_adopt_pins.py (2): plan-pin-malformed-commit-line-break, plan-pin-malformed-digest-line-break;
  • test_shape_doctor.py -k a_root_pin_that_does_not_read: 1 failed, 12 passed, 436 deselected in 57.84s (a-commit-ending-in-a-line-break).

With the change the same files pass (see the runs).

Mutants (the new file only, -rf)

mutant failed passed
M1: both repo_shape patterns back to $ 11 35
M2: only COMMIT_RE back to $ 7 39
M3: only SHA256_RE back to $ 4 42
M4: only the bootstrap COMMIT_40 back to $ 2 44

Re-measured in fix round 2, at 4c1f80cf, on the 46-row file. The first round measured its 42 rows: 7/35, 4/38, 3/39 and 2/40. M1 is M2 and M3 together.

Each is killed by at least one row, and each pattern is killed by a row of its own. Every mutant was undone: the production files are byte for byte (cmp) their backups of this change.

Runs with the change (head ae30dbd0e0eecfb7092c5ef257da2367d114fc62, worktree clean at each START)

git 2.43.0 (/usr/bin/git):

  • shard tA, 35 files: 2782 passed, 8 skipped in 4138.49s, rc=0;
  • shard tB, 39 files: 2630 passed, 6 skipped in 4532.76s, rc=0;
  • trio (tests/test_repo_hygiene.py tests/test_agents_md_codes.py tests/test_cli_reference.py): 170 passed in 37.48s, rc=0.

The two shards are 74 targeted files: every file that names a pin code (pin-tag-only, pin-digest-*, shape-pin-*, manifest-shape-*, family-shape-*, member-pin-*, update-pin-*, neutral-pin-*, plan-pin-*, not 40/64 hex) or one of COMMIT_RE, SHA256_RE, COMMIT_40, every tests/test_bootstrap*.py (13 of 13), test_shape_doctor.py, 6 of the 19 test_update_shape*.py files, test_family.py, test_scaffold_e2e.py, and the pin tests. They replaced a wide run (226 importer files, about 27k tests) that was killed by PID under load (load average 72 on 32 cores, 2% in 11 minutes); that run is not evidence. The new file and the four edited files are all inside the shards. The other 13 test_update_shape*.py files, which name no pin code, ran as a third run (480 passed, 3 skipped in 441.06s, rc=0), and tests/test_scaffold_leg_path_collisions.py as a fourth (104 passed in 4.35s, rc=0). No full suite was run.

git 2.47.1 (git --version printed first in each log):

  • the new file, test_pin_validator.py, test_manifest_validator.py, test_adopt_pins.py: 323 passed in 208.37s, rc=0.
  • test_shape_doctor.py -k a_root_pin_that_does_not_read: 13 passed, 436 deselected in 52.85s, rc=0.

Merge-trees

Refreshed after the fix round, against the re-fetched origin/main:

Fix round 1

One new commit on top, 03b2b97b9d3cccc42b4703cb2f7c978693488266, after the independent review of ae30dbd0e0eecfb7092c5ef257da2367d114fc62 (not ready on one must). It changes tests/test_pin_pattern_line_break.py and nothing else; no production file changes.

  • Composite asserts (SonarCloud python:S9073, the two issues at lines 164 and 173). member_pin and holder_shape ended in assert commit and digest, text. Each value now has its own assert, with a message that names it (fixture drift: no member pin commit, no member pin tree_sha256, no shape commit, no shape tree_sha256).

  • Two callers had no row of their own, so a .rstrip("\n") before the match in either survived every row. The new rows (the file now has 46):

    • test_a_neutral_pin_commit_that_ends_in_a_line_break_is_refused: the neutral product's pin in validate-pins.py, exit 2, REFUSED neutral-pin-tag-only. On main the commit passed COMMIT_RE and the run exited 1 with neutral-pin-source-unreadable, git refusing the name;
    • test_a_neutral_pin_digest_that_ends_in_a_line_break_is_malformed: the same pin's tree_sha256, exit 1, neutral-pin-digest-malformed. On main it was recomputed and compared (neutral-pin-digest-mismatch);
    • test_a_recorded_source_commit_that_ends_in_a_line_break_is_a_finding, two rows (submodules, answered-paths): a record row's source_commit: in validate-manifest.py, exit 1, manifest-submodule-commit and manifest-answered-commit. On main: manifest ok.

  • The four rows fail on main. The new file with main's two production files checked out: 13 failed, 33 passed (the nine of the first round, and these four).

  • Mutants, each applied alone to the committed head, the file restored afterwards (sha256 94f125c5f5e4b8a0 for validate-pins.py and 260c4742fe422597 for validate-manifest.py, the same as before the mutant, and clean against HEAD; repo_shape.py and bootstrap.py unchanged at 623af12131447f95 and 4102d375f4c9bc38). The new file only:

    • S1, .rstrip("\n") on the neutral pin's commit before the match (validate-pins.py line 1790): 1 failed, 45 passed, killed by the neutral-commit row;
    • S1b, the same on the neutral pin's recorded digest before the match (line 1800): 1 failed, 45 passed, killed by the neutral-digest row;
    • S2, .rstrip("\n") on source_commit before the match (validate-manifest.py line 1264): 2 failed, 44 passed, killed by both source_commit rows;
    • before this round S1 and S2 survived.

  • Runs, at head 03b2b97b9d3cccc42b4703cb2f7c978693488266, worktree clean at each START:

    • git 2.43.0: the new file, tests/test_pin_validator.py and tests/test_manifest_validator.py: 175 passed in 58.03s, rc=0;
    • git 2.43.0: the trio: 170 passed in 39.29s, rc=0;
    • git 2.43.0, extra: tests/test_neutral_pin_validator.py (the new rows use its helpers): 39 passed in 99.62s, rc=0;
    • git 2.47.1 (git --version printed first): the new file: 46 passed in 19.32s, rc=0.
    • The shards, the other git 2.47.1 runs and the extras above ran at ae30dbd0e; the two production files and the four other test files are the same at this head.

Fix round 2

This round is one merge of main and one new commit, after rc1126's re-check: READY at 03b2b97b, with two prose NITs (#1126 issuecomment-6340973564).

Not changed

Other ^...$ patterns are used with .match on values read from files or arguments: PROJECT_ID_RE, DATE_RE and QUALIFIED_RE (validate-manifest.py, validate-family.py), REPOSITORY_RE (update-shape.py), PIN_ARG_RE (shape_materialize.py), and NAME_RE, REF_RE and PATH_RE (setup-project.py). They have the same $ property and are outside the two patterns this issue names, so they are left for #1139, filed from this PR's review.

Closes #1017

🤖 Generated with Claude Code

Summary by Sourcery

Require pin identifiers and tree digests to end exactly at the expected hexadecimal value, preventing trailing line breaks from being accepted as valid pins.

Bug Fixes:

  • Reject commit and tree digest values with trailing line breaks across repository validation, adoption, bootstrap, and shape-doctor flows.

Enhancements:

  • Expand regression coverage for malformed pin values and affected validation outcomes.

Tests:

  • Add focused coverage for regex anchoring, validator behavior, bootstrap notices, adoption checks, and shape-doctor pin parsing.

Brett Heap @claude
repo_shape.py and bootstrap.py: COMMIT_RE, SHA256_RE and `COMMIT_…
ae30dbd 
…40` end in `\Z`, not `$`, so a pin's `commit:` or `tree_sha256:` that ends in a line break is refused as not 40 or 64 hex by every validator and `adopt-project.py` (#1017)

`$` also matches just before a trailing line feed, and every caller reads these
patterns with `.match`, so a `commit:` or `tree_sha256:` read as the hex and a
line feed (the double-quoted escape for a line feed, or a block scalar) passed as
well formed. On main, validate-manifest.py and validate-family.py passed one
(`manifest ok`, `family ok`), validate-pins.py refused one for another reason
(`pin-commit-unresolvable`, `pin-digest-mismatch`), and the assembly root's
bootstrap.py, which holds a copy of the commit pattern as `COMMIT_40`, printed
its upstream notice over one.

`\Z` matches only at the very end of the value, so each caller's `.match`
refuses a trailing line feed as it already refused a carriage return and line
feed, and a space. Nothing else about the patterns changes: 40 and 64 digits,
either case.

Rows: tests/test_pin_pattern_line_break.py (the three patterns, the holder's
gate, the upstream notice, the doctor's `pin_commit`), and a row each in
test_pin_validator.py, test_manifest_validator.py, test_adopt_pins.py and
test_shape_doctor.py. Each fails on main and passes with the change.

Closes #1017

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 21, 2026

Copy link
Copy Markdown

Reviewer's Guide

The PR fixes an anchoring defect where Python’s $ accepted a single trailing line feed in 40-character commits and 64-character digests, changing the canonical and bootstrap validators to use \Z and adding targeted regression tests across every affected consumer.

Flow diagram for strict pin validation

flowchart LR
    Input["commit or tree_sha256 value"] --> Pattern{Strict \Z pattern}
    Pattern -->|"exact hex length"| Valid["Accepted"]
    Pattern -->|"trailing line feed or other suffix"| Invalid["Rejected as malformed"]
Loading

File-Level Changes

Change Details Files
Harden hexadecimal pin validation against trailing line feeds by replacing $ with \Z in the canonical validators and the bootstrap copy.
  • Require commit and SHA-256 values to terminate at the true end of the string while preserving exact lengths and case handling.
  • Keep the template bootstrap validator synchronized with repo_shape.COMMIT_RE.
  • Leave unrelated regexes and production behavior unchanged.
 scripts/repo_shape.py
templates/assembly-root/scripts/bootstrap.py
Add regression coverage for pattern behavior and all affected validation and pin-reading paths.
  • Cover trailing line feeds, CRLF, whitespace, extra digits, prefixes, and case variants for all three patterns.
  • Verify malformed/refused findings for manifest, family, leg-pin, shape-pin, and adopted-plan values.
  • Verify bootstrap upstream notices and shape-doctor pin reads reject newline-terminated commits.
  • Document baseline behavior and test the scaffolded-family gate.
 tests/test_pin_pattern_line_break.py
tests/test_pin_validator.py
tests/test_manifest_validator.py
tests/test_adopt_pins.py
tests/test_shape_doctor.py

Assessment against linked issues

Issue Objective Addressed Explanation
#1017 Make COMMIT_RE and SHA256_RE reject trailing line breaks and other trailing characters while continuing to accept exactly 40 or 64 hexadecimal digits. ✅
#1017 Update the copied commit pattern in bootstrap.py so it applies the same strict end-of-value validation. ✅
#1017 Add regression tests demonstrating that newline-terminated commit and digest values are rejected by the patterns and relevant validators/readers. ✅

Possibly linked issues

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@brettheap brettheap mentioned this pull request Oct 21, 2026
Draft
@brettheap

brettheap commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Independent review of #1126 (#1017) at ae30dbd0, for lane openRepoShape-5: NOT READY

The fix is right. Nothing that worked before is refused now, and I found no other copy that needs the change. The only blocker is two composite asserts in the new test file, which SonarCloud has flagged on the PR. Splitting them is a two-line fix round.

Findings

MUST: two composite asserts (fix round). tests/test_pin_pattern_line_break.py:164 and :173

  • Both lines are assert commit and digest, text.
  • SonarCloud has two open MAJOR python:S9073 issues on PR 1126, at exactly these lines (keys AaFfCc0yxzEwGRCiMIPw and AaFfCc0yxzEwGRCiMIPx).
  • The quality gate still reads OK: the ratings are 1 and new duplication is 0.0%. I call this a MUST only because the lane's rule is that S9073 always gets a fix round.
  • The scenario: when the fixture drifts and one of the two re.search calls finds nothing, the message cannot say which one failed.
  • The fix: one assert for each of the two values.

NIT: the body misstates what main does with family.yaml's shape.commit. Body line 19 (in $Q/v118/w1017/body.md) says both shape fields "passed". Measured on 4ed7c7bf:

  • shape.digests.tree_sha256 with a line feed does pass: family ok, exit 0.
  • shape.commit with a line feed exits 1 with FINDING shape-pin-manifest-disagree: contracts/shape-pin.yaml commit 4ed7c7bf... != family.yaml shape.commit 4ed7c7bf.... The same 40 hex is printed on both sides.
  • STATE.md has the same error.

NIT: the merge-tree section is stale. Body lines 80-82 name 823624d93 and "the two PRs that landed since (#1101, #1102)".

NIT: two callers have no row of their own, so a caller-side mutant survives (details under Rows and mutants). These callers are protected only through the shared pattern; that is what the PR changes, so they work. The gap is coverage, not a bug:

  • the neutral-product pin at templates/assembly-root/scripts/validate-pins.py:1791 (and its digest at :1800);
  • the chain record's source_commit at templates/assembly-root/scripts/validate-manifest.py:1264.

Checked and true

1. The defect on main (4ed7c7bf; git diff --quiet 823624d93 4ed7c7bf5 over repo_shape.py, templates/, shape-doctor.py, update-shape.py, family.py and bump-leg.py is empty). Python 3.12.3:

  • .match takes 40 or 64 hex followed by a line feed, for COMMIT_RE, SHA256_RE and COMMIT_40 alike.
  • It refuses the hex followed by \r\n, a space or two line feeds. fullmatch refuses the line feed.
  • repo_shape.parse_yaml reads both commit: "<hex>\n" and commit: | as the hex plus a line feed. A single-quoted '<hex>\n' is a literal backslash and n, already refused.

Per-tool outcomes on main, through the PR's own rows run against main's production files (17 failed, 319 passed in 330.72s):

Tool and field (value ends in a line feed) On main
validate-manifest.py shape.commit, shape digest manifest ok, exit 0
validate-pins.py leg commit REFUSED pin-commit-unresolvable (not pin-tag-only)
validate-pins.py leg digest pin-digest-mismatch
validate-pins.py shape-pin commit only shape-pin-manifest-disagree
validate-family.py member pin commit REFUSED member-commit-unresolvable
validate-family.py member pin digest member-digest-mismatch
adopt-project.py check exit 0
shape-doctor.py pin_commit returned '...\n'
shape-doctor.py gate exit 3 where 1 is expected
bootstrap.py upstream notice printed

A direct probe of shape_upstream_notice agrees: main prints the notice for a "<hex>\n" pin, and the head is silent. The 17 failing ids are exactly the writer's 16 plus the doctor row.

2. No caller relied on $. I grepped every use outside tests/. Each value is one of:

  • A YAML read, str()-ed:
    • adopt-project.py (the plan's commit: and tree_sha256:, and source.commit in _not_a_whole_clone);
    • validate-manifest.py 993, 999 and 1264;
    • validate-pins.py 755, 1265, 1367, 1791 and 1800;
    • validate-family.py 664, 669, 756, 1435, 1646 and 2417;
    • update-shape.py 822;
    • shape-doctor.py 3694, 5483, 6424 and 10368;
    • bootstrap.py 1797.
  • Stripped git output: git_out strips in text mode (repo_shape.py:1406), which covers update-shape.py:558. bootstrap.py:1802 reads _try_git, which strips, or _cached_tip, which uses .strip().
  • Split fields: recorded_gitlink, for shape-doctor.py:2933.
  • checked_value first: bump-leg.py:1669 and family.py 6521 and 9334. SAFE_ARG_RE fullmatch already refuses the line feed.
  • Stripped first: bump-leg.py:2250.
  • Not these patterns: src/openreposhape/standard.py has its own and uses fullmatch.

No site where valid input is now refused. The only values that change are YAML values that end in a line feed. No tool writes one; it takes a double-quoted escape, or a block scalar written by hand.

3. Copies.

  • No other copy shares the $ property. templates/family-root/scripts/bootstrap.py and siblings.py hold no hex pattern. In shape_materialize.py, README_SHAPE_LINE_RE is a MULTILINE per-line pattern and OBJECT_ID_RE uses fullmatch. release.yml uses fullmatch. The @(?P<sha>...) patterns are unanchored.
  • The pinned-copy rule holds: the PR adds no import, and test_pinned_copy_text.py passes at the head.
  • No test or doc pins the old bytes or hashes. I grepped for the old blob ids 6309c115/49eb0412, the old sha256s ebbb9061/92d74d78 and {40}$/{64}$: the only hits are the three code lines.

4. Rows and mutants. Run in a worktree at the predicted merge, with the new file (and target rows) only:

Mutant Change Result
M1 (writer's) both repo_shape patterns back to $ 7 failed / 35 passed, matches the writer
M2 (writer's) COMMIT_RE only 4/38, matches
M4 (writer's) COMMIT_40 only 2/40, matches
W1 COMMIT_RE ends \n?\Z killed, 4/38
W2 SHA256_RE ends \n?\Z killed, 3/39
W3 validate-pins.py leg commit .rstrip("\n") before the match killed (exit 0 where 2 expected)
W4 validate-family.py:1435 digest .rstrip killed
W5 adopt-project.py _pin_row_problem .rstrip killed
W6 shape-doctor.py pin_commit .rstrip killed
W7 bootstrap.py:1797 .rstrip killed
W8 validate-manifest.py:993 .rstrip killed
S1 .rstrip on the neutral-pin commit, validate-pins.py:1791 survived: 67 passed over the new file and test_pin_validator.py
S2 .rstrip on source_commit, validate-manifest.py:1264 survived: 146 passed over the new file and test_manifest_validator.py

A caller's .match changed to .search is equivalent under ^...\Z, so I did not count it. After the runs the production files were restored, with sha256 623af12131447f95 and 4102d375f4c9bc38, the same as the writer's.

5. Runs (all through run9.sh, EMAIL=service@opensoft.dev):

  • Head, git 2.43: 593 passed in 384.49s. That is the five files (with only the edited doctor test, 13 rows) plus test_pinned_copy_text.py and the trio.
  • Head, git 2.47.1: the new file, 42 passed in 11.17s. git --version printed 2.47.1, both outside and under run9's PATH.
  • Predicted merge with 4ed7c7bf (commit 81bffaa3, tree 0db1b9ca), git 2.43: the five files, 336 passed in 511.31s.

6. Title and body.

  • The title is 240 characters and true.
  • The only closing keyword in the live body and in the commit message is Closes #1017, and closingIssuesReferences is [1017] only. body-guard.py gives rc=0.
  • Body numbers reproduced by me: 42 rows; the failures on main (16+1, matching my 17 out of 336); M1, M2, M4; 13 of 13 test_bootstrap*.py files; 19 test_update_shape*.py files with 6 in the shards; shards of 35 and 39 files, 74 in all, no duplicates; 17 open PRs in open-prs2.json.
  • The rest are numbers I only read in the writer's logs (see below).

Sonar-style checks:

  • No production function changed (only module constants and comments), and the test helpers are flat, so cognitive complexity is not an issue.
  • Sonar reports duplication at 0.0%.
  • The added production lines are at most 79 columns. Three lines of the new test file are 80-81 columns, within the widths of sibling test files (82-90).
  • The diff adds no non-ASCII character.
  • The commit is authored by Brett Heap service@opensoft.dev, with the Lane and Co-Authored-By trailers.

Could not verify

  • The writer's 74-file shards (tA 2782 passed and 8 skipped; tB 2630 passed and 6 skipped), the extra update-shape run (480 passed, 3 skipped), the collisions run (104 passed), g247a (323) and g247b (13). I read the logs (all rc=0) but did not re-run them.
  • The full test_shape_doctor.py: I ran only the edited parametrized test.
  • The body's measurements on 2a119f28 and 6da757cbe. I measured on 4ed7c7bf.
  • test_pinned_copy_text.py's own count. 87 is inferred from 593 - 336 - 170, not measured alone.
  • No run on git 2.55 (CI) or on Windows.

Follow-up assessment (item 7; input for the lane's issue, not findings against this PR)

  • PROJECT_ID_RE: reachable from files (the id: of project.yaml, family.yaml and an adoption plan, and a member's project.yaml read by family.py) and from --id.
    • Measured on main: validate-manifest.py with id: "atlas\n" gives only FINDING manifest-topic (expected 'xf-project-atlas\n', exit 1).
    • validate-family.py with id: "inkrouter\n" gives family ok, exit 0.
    • The scaffold and family.py init run checked_value after it (refused). adopt-project.py --id and the plan id: were not measured end to end.
  • DATE_RE: reachable from files and from --elected-on.
    • Measured on main: validate-manifest.py elected_on: "2026-09-02\n" gives manifest ok, exit 0; validate-family.py created_on: "2026-09-04\n" gives ok, exit 0.
    • shape_materialize.py's --elected-on and shape_advisory.py then call date.fromisoformat, which refuses it (measured: election-date-malformed).
  • QUALIFIED_RE: reachable from files.
    • Measured on main: validate-manifest.py leg repository: "testorg/Atlas\n" gives manifest ok, exit 0.
    • validate-family.py member repository: "InkRouter/IRRS\n" gives FINDING member-repository-name ("whose name is 'IRRS\n'"), exit 1.
  • REPOSITORY_RE (update-shape.py:209/680): reachable from --upstream and from the shape pin's source_repository:. On main it takes the line feed, and the next line, checked_value("--upstream", ...), refuses it as unsafe-value (measured in-process). That message names --upstream even when the value came from the pin.
  • PIN_ARG_RE: --pin is .strip()-ed first. adopt-project.py builds repository@commit with the commit COMMIT_RE-checked first, which this PR makes \Z. A line feed in the repository falls mid-string and the character class refuses it. So nothing reaches its $ unguarded after this PR.
  • NAME_RE/REF_RE/PATH_RE (setup-project.py): argv and $OPENREPOSHAPE_REPO only. why_refused checks CONTROL_RE and then fullmatch (measured: "It holds a control character"). Not affected.

Scratch left

  • Notes and logs in /tmp/claude-1000/-workspace-projects-openRepoShape/50cede29-ab82-4ca3-9f67-d98e1a58c8d2/scratchpad/v118/rv1126/: r1-head-243.log, r2-head-2471.log, r3-merge-243.log, base-main-rows.log, probe-out.txt, probe-run.log, mutants-summary.log, mut/*.log, tools/, the probe roots nb1/ and nb2/, issue1017.txt, pr1126.json and pr-body.md.
  • My basetemps under /tmp/claude-1000/-workspace-projects-openRepoShape/50cede29-ab82-4ca3-9f67-d98e1a58c8d2/scratchpad/v110/bt/: rv1126-base, rv1126-mut, rv1126-probe, rv1126-r1, rv1126-r2 and rv1126-r3. The first reviewer's rv1126-g247, -h1, -h2, -m1, -m2, -onmain, -ph and -pm are still there too.
  • The first reviewer's four worktrees, and my own four (rv1126-head, -main, -merge, -mut), are removed with git worktree remove. refs/l5rv/rv1126-pr and refs/l5rv/rv1126-merge are deleted.
  • Nothing of mine is running (pgrep -af rv1126 finds nothing).
  • Nothing was pushed, posted or filed.

Lane: openreposhape-5 (openRepoShape-5)

@brettheap brettheap mentioned this pull request Oct 21, 2026
Open
Brett Heap @claude
tests/test_pin_pattern_line_break.py: a line feed ending a neutral pi…
03b2b97 
…n's commit or digest, or a record's `source_commit:`, has a row, so `.rstrip` before the match in validate-pins.py or validate-manifest.py fails; each read asserts one value (#1017)

Fix round 1 for #1126, from its independent review.

`member_pin` and `holder_shape` ended in `assert commit and digest, text`
(SonarCloud S9073): when one of the two reads found nothing, the message could
not say which. Each value now has its own assert and a message that names it.

Two callers of the patterns had no row of their own, so a `.rstrip("\n")` before
the match survived every row: the neutral product's pin in validate-pins.py
(`commit:` is refused as `neutral-pin-tag-only`, `tree_sha256:` is
`neutral-pin-digest-malformed`) and the `source_commit:` of a `submodules:` or
`answered_paths:` row in validate-manifest.py (`manifest-submodule-commit`,
`manifest-answered-commit`). On main the first was the product's checkout
refusing the name (`neutral-pin-source-unreadable`), the second a digest
recompute (`neutral-pin-digest-mismatch`), the third `manifest ok`.

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@brettheap

brettheap commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Re-check of #1126 (#1017) fix round 1 at 03b2b97b, for lane openRepoShape-5: READY

All five checks hold. There is no MUST or SHOULD, and two NITs, both about prose. The fix round does what rv1126 asked, and the SonarCloud S9073 issues are now closed.

Findings

NIT 1: the test file's module docstring still says what the body no longer says. tests/test_pin_pattern_line_break.py:25-27 says validate-family.py "passed" shape.commit and shape.digests.tree_sha256 in family.yaml on main. That is true of the digest and false of the commit.

  • With main's two production files, test_a_family_shape_commit_that_ends_in_a_line_break_is_a_finding fails with FINDING shape-pin-manifest-disagree: contracts/shape-pin.yaml commit 03b2b97b... != family.yaml shape.commit 03b2b97b..., and "1 finding(s)". The same hex is printed on both sides, and family-shape-commit was not raised.
  • The digest row on main printed family ok, exit 0.
  • The writer fixed this in the PR body, which matches that behaviour, but not in the docstring. One clause of the docstring fixes it.

NIT 2: the body's "Mutants" table is measured on the 42-row file. It reads 7/35, 4/38, 3/39 and 2/40, which sum to 42. The file now has 46 rows. I re-measured on the head:

Mutant Failed Passed
M2: COMMIT_RE back to $ 7 39
M3: SHA256_RE back to $ 4 42
M4: COMMIT_40 back to $ 2 44

Every pattern is still killed by a row of its own. M1 is the union of M2 and M3, so I did not run it. Nothing is wrong, the table is just stale.

Main moved to 038dafba after the body's merge-tree section (written against 4ed7c7bf). It is clean against the new main too (below), so this is only a refresh before landing.

Checked and true

1. The MUST is fixed.

  • The four asserts are at :171, :172, :181 and :182:
    • assert commit, f"fixture drift: no member pin commit in {text}"
    • assert digest, f"fixture drift: no member pin tree_sha256 in {text}"
    • assert commit, f"fixture drift: no shape commit in {text}"
    • assert digest, f"fixture drift: no shape tree_sha256 in {text}"
  • An ast scan found 27 asserts in the file and no BoolOp (and or or) in any of them, message included.
  • SonarCloud has closed both S9073 issues (AaFfCc0yxzEwGRCiMIPw and ...Px, status CLOSED). The PR's quality gate reads OK, with ratings 1 and 0.0% new duplication, and there are no other issues on the PR.

2. The four new rows.

  • Each asserts what its name says and reuses the helpers it imports:
    • the neutral commit row asserts exit 2 and REFUSED neutral-pin-tag-only;
    • the neutral digest row asserts exit 1 and FINDING neutral-pin-digest-malformed;
    • the two source_commit rows go through assert_refused with SUBMODULE_RECORD and ANSWER_RECORD, expecting manifest-submodule-commit and manifest-answered-commit.
  • With main's scripts/repo_shape.py and templates/assembly-root/scripts/bootstrap.py checked out (blob ids 49eb0412 and 6309c115, sha256 92d74d78 and ebbb9061), the new file gives 13 failed, 33 passed in 13.37s, as the writer counted. The four new rows each fail for the stated reason:
    • neutral commit: neutral-pin-source-unreadable, with git ls-tree exiting 128, "Not a valid object name 324db5a4..." and a line feed after the commit;
    • neutral digest: neutral-pin-digest-mismatch;
    • both source_commit rows: manifest ok.
  • Each mutant was applied alone to the head in a scratch worktree, with the file restored afterwards. The sha256 prefixes match the writer's, and git status was clean after each:
    • validate-pins.py is 94f125c5f5e4b8a0 and validate-manifest.py is 260c4742fe422597;
    • repo_shape.py is 623af12131447f95 and bootstrap.py is 4102d375f4c9bc38.
  • Mutants on the new file only:
Mutant Change Result
S1 .rstrip("\n") before the match, validate-pins.py:1791 killed, 1 failed / 45 passed, by the neutral-commit row
S1b the same on the digest, :1800 killed, 1/45, by the neutral-digest row
S2 the same on source_commit, validate-manifest.py:1264 killed, 2/44, by both rows
S1a (mine) .rstrip("\n") on the assignment, :1790 killed, 1/45
O1 (mine) .strip() in place of .rstrip("\n") at :1791 killed, 1/45
O2 (mine) .strip() in place of .rstrip("\n") at :1800 killed, 1/45
O3 (mine) .strip() at manifest :1264 killed, 2/44
O4 (mine) .replace("\n", "") at manifest :1264 killed, 2/44
O5a (mine) .rstrip("\n") for the submodules record only killed, 1/45 ([submodules])
O5b (mine) .rstrip("\n") for the answered-paths record only killed, 1/45 ([answered-paths])

No mutant survived. O5a and O5b show that both source_commit rows are needed.

3. Runs through run9.sh, EMAIL=service@opensoft.dev, at the head, worktree clean at each start.

  • Git 2.43.0:
    • the new file, test_pin_validator.py and test_manifest_validator.py: 175 passed in 44.31s;
    • the trio: 170 passed in 43.66s;
    • extra, test_neutral_pin_validator.py, test_pinned_copy_text.py and test_adopt_pins.py: 278 passed in 224.70s;
    • the four new rows run alone: 4 passed, so there is no import-order dependence.
  • Git 2.47.1 ($Q/v115/w764/git247/bin/git): the new file, 46 passed in 13.02s. Under run9's PATH composition, command -v git gives that binary and git --version gives 2.47.1.

4. Merges.

5. The body's NITs.

Could not verify

Scratch I leave

  • I removed my three worktrees (rc1126-head, -base, -merge) with git worktree remove, and deleted refs/l5rv/rc1126-pr and refs/l5rv/rc1126-merge.
  • Logs and scripts are under /tmp/claude-1000/-workspace-projects-openRepoShape/50cede29-ab82-4ca3-9f67-d98e1a58c8d2/scratchpad/v119/rc1126/:
    • the logs base-main-prod.log, head-243-a.log, head-243-trio.log, head-243-extra.log, head-243-iso.log, head-2471.log, merge-243.log and chain-mut.out;
    • mut/*.log, bak/*.orig, mt-038dafba.txt, mutate.py, mut.sh and chain-mut.sh.
  • The review body and PR JSON I read are v119/rc1126-review-body.md, rc1126-pr.json and rc1126-pr-body.md.
  • Basetemps under $Q/v110/bt/: rc1126-base, -ex, -g247, -h1, -iso, -mg, -mut and -trio.
  • Nothing of mine is running. I pushed, posted and filed nothing, and changed no branch.

Lane: openreposhape-5 (openRepoShape-5)

@brettheap brettheap mentioned this pull request Oct 22, 2026
Merged
Brett Heap and others added 2 commits October 22, 2026 13:18
Brett Heap @claude
Merge origin/main (038dafb) into fix/1017-repo-shape-match-end
c7da27d 
Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Brett Heap @claude
tests/test_pin_pattern_line_break.py: the module docstring says `vali…
4c1f80c 
…date-family.py` passed only the digest of `family.yaml`'s `shape:` on main, and gave `shape-pin-manifest-disagree` over the commit, not that it passed both (#1017)

NIT 1 of rc1126's re-check of #1126 (issuecomment-6340973564). The docstring said
validate-family.py passed `shape.commit` and `shape.digests.tree_sha256` in
family.yaml on main. Only the digest passed (`family ok`, exit 0). The commit
passed `COMMIT_RE` too, so `family-shape-commit` was not raised, but
`contracts/shape-pin.yaml` holds the commit without the line feed, and the gate
gave `shape-pin-manifest-disagree` with the same hex printed on both sides. The
clause now says that. A docstring only: no code and no row changes.

Lane: openreposhape-5 (openRepoShape-5)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@sonarqubecloud

sonarqubecloud Bot commented Oct 22, 2026

Copy link
Copy Markdown

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@brettheap brettheap mentioned this pull request Oct 22, 2026
Merged
@brettheap

brettheap commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Re-check of #1126 (#1017) for lane openRepoShape-5: READY at 4c1f80cf

I checked only the delta from 03b2b97b, the head rc1126 passed (issuecomment-6340973564). No MUST, no SHOULD, no NIT. One body note, below.

Checked and true

1. The merge is clean and adds nothing of its own.

  • c7da27dc merges main 038dafba into 03b2b97b. Its tree is 4729e601b8566546055685abce9e2d0b035b0589, and git merge-tree --write-tree 03b2b97b 038dafba gives the same tree.
  • The PR's seven files have the same blobs at 03b2b97b and 4c1f80cf, except tests/test_pin_pattern_line_break.py.
  • Main did not touch any of the other six between the old merge base 6da757cbe and 038dafba.

2. The NIT commit 4c1f80cf changes only the module docstring of tests/test_pin_pattern_line_break.py (6 insertions, 2 deletions).

  • The added lines are at most 77 wide, ASCII, with no CR and no trailing space.
  • The file still has 27 asserts, and none holds an and or an or (ast scan).

3. The new docstring is true of main.

  • I put main's scripts/repo_shape.py and templates/assembly-root/scripts/bootstrap.py (sha256 92d74d78 and ebbb9061) in my scratch tree. The file then gives 13 failed, 33 passed, as the body says.
  • test_a_family_shape_commit_that_ends_in_a_line_break_is_a_finding fails with FINDING shape-pin-manifest-disagree: contracts/shape-pin.yaml commit 4c1f80cf... != family.yaml shape.commit 4c1f80cf.... The same hex is on both sides, and family-shape-commit is not raised.
  • test_a_family_shape_digest_that_ends_in_a_line_break_is_a_finding fails on assert 0 == 1, and the gate printed family ok: InkRouter (inkrouter), 2 member(s).
  • That is what the docstring now says, so rc1126's NIT 1 is fixed.

3b. The mutants. NIT 2 of the same comment was the stale table. I applied each mutant alone to 4c1f80cf, ran the new file with -rf, and restored it.

Mutant Result In the body
M1: both repo_shape patterns back to $ 11 failed, 35 passed 11/35
M2: only COMMIT_RE 7 failed, 39 passed 7/39
M3: only SHA256_RE 4 failed, 42 passed 4/42
M4: only bootstrap.COMMIT_40 2 failed, 44 passed 2/44
  • Each mutant is killed by a row of its own.
  • After each one, the sha256 was back to 623af12131447f95 for repo_shape.py and 4102d375f4c9bc38 for bootstrap.py, and git status showed 0 modified files.
  • NIT 2 is fixed.

4. Runs at 4c1f80cf, with the worktree clean at each START.

  • git 2.43.0: tests/test_pin_pattern_line_break.py and tests/test_path_guard_hygiene.py, 95 passed (46 + 49) in 38.56s, rc=0.
  • git 2.47.1 (git --version printed first): the same two files, 95 passed in 42.17s, rc=0.

5. Merge-trees.

Body note (not a finding)

The body's merge-tree bullet for fix round 2 is out of date:

Could not verify

  • I did not re-run the writer's two 74-file shards or its extra runs. They ran at ae30dbd0, and no production file has changed since.
  • I did not measure Python 3.14 or Windows. The delta adds no text that depends on the version.

Lane: openreposhape-5 (openRepoShape-5)

This was referenced Oct 22, 2026
Draft
Draft
Merged
Draft
Open
@brettheap brettheap added this to the v1.2.1 milestone Oct 23, 2026
@brettheap
brettheap marked this pull request as ready for review October 23, 2026 19:00
@sourcery-ai

sourcery-ai Bot commented Oct 23, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 4 hours and 46 minutes by commenting @sourcery-ai review. Upgrade to get a review now.

@brettheap
brettheap merged commit 422a590 into main Oct 23, 2026
5 checks passed
@brettheap
brettheap deleted the fix/1017-repo-shape-match-end branch October 23, 2026 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

v1.2.1

1 participant

@brettheap