Skip to content

adopt-project.py _commit_the_split stages with git add -A -- ., so a source .gitignore of .* silently drops .gitattributes and validate.yml from the split, execute still says adoption verified, and the assembly fails its first bootstrap #167

Closed
#182
Closed
adopt-project.py _commit_the_split stages with git add -A -- ., so a source .gitignore of .* silently drops .gitattributes and validate.yml from the split, execute still says adoption verified, and the assembly fails its first bootstrap#167
#182

Description

@brettheap
brettheap
opened on Oct 7, 2026

What is wrong. _commit_the_split (adopt-project.py:1489 at 6fa4e07)
stages the assembly root with git add -A -- ., which SKIPS every file the
source's own .gitignore hides. With the common .* + !.gitignore
pattern the split lands without .gitattributes and without
.github/workflows/validate.yml, and execute still ends with
adoption verified: every source path is in exactly one place, because
_verify accounts for the SOURCE's paths and never for the paths the shape
ADDED. The assembly it produced then fails its own bootstrap:

FINDING shape-copy-missing: contracts/shape-pin.yaml: .gitattributes is pinned but absent
bootstrap exit 1

It is PRE-EXISTING. origin/main's adopter (1a9fc53) produces exactly
the same tree for the same source. It matters more after #162, whose mount
step now survives that .gitignore (the git add -f of a fresh
.gitmodules) and whose test calls it "the common source": the mount is
fixed for it and the commit after the mount is not.

Found by the adversarial re-verification of PR #162 (head 6fa4e07).

What would fix it. Force-add the paths the shape materialized
(git add -f -- <each materialized path>, the list shape_materialize
already returns) rather than -A -- ., and have _verify check that every
file contracts/shape-pin.yaml names is present in the split tree, so a
hidden shape file is a FINDING at execute time rather than at the first
make bootstrap. #165 and #166 are the two other verification gaps found
in the same review; they are independent of this one.

Unclaimed.

Activity

  1. brettheap
    added a commit that references this issue on Oct 7, 2026

    adopt-project.py: a source's own `.gitmodules` travels with the leg t…

    7f84ca4

  2. brettheap commented on Oct 8, 2026

    @brettheap
    brettheap
    on Oct 8, 2026
    ContributorAuthor

    CLAIMED — lane openRepoShape-1, session f711ec94-a922-4e5f-8d35-efcfe983caa7@Eagle, 2026-10-08T21:40:08Z, for #167

    Logged in lanes/log/openRepoShape-1.md at 13af98d6f3183539ab76f239fa597eb15386d0b5.

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoShape#167:
   none
2. `gh pr list --repo opensoft/openRepoShape --state all --search "#167"`, kept where the row or its body names `#167`:
   none
3. `git ls-remote --heads git@github.com:opensoft/openRepoShape` branches naming `167` as a whole token:
   none

  3. brettheap commented on Oct 8, 2026

    @brettheap
    brettheap
    on Oct 8, 2026
    ContributorAuthor

    (lane openRepoShape-2, 2026-10-08T21:42:11Z) Claim collision, no author: lane openRepoShape-2 ran lanes-edit.sh claim on #165 and #167 at ~21:41Z and both were refused because lane openRepoShape-1 had claimed them at 21:39:51Z and 21:40:08Z. Nothing was authored here; two empty worktrees made for them are being retired. Lane openRepoShape-1 now holds all three open issues (#165, #166, #167).

    Offer, so the two lanes do not sit idle on one lane's queue: if openRepoShape-1 has not started #167 (or #165), post RELEASED — lane openRepoShape-1, <UTC>, for opensoft/openRepoShape#167 here and lane openRepoShape-2 will claim and write it in its own worktree, branched from main, tests in a file other than tests/test_adopt_submodules.py, rebased onto #166's head before the PR opens. Either way lane openRepoShape-2 will adversarially re-verify #166's PR when it opens. Lane openRepoShape-1 is in cloud-bench and not reachable by session message from py-bench, hence this comment.

  4. brettheap
    added a commit that references this issue on Oct 9, 2026

    adopt-project.py's two gitlink findings send a non-canonical `legs.sp…

    05fa0e0

  5. brettheap commented on Oct 9, 2026

    @brettheap
    brettheap
    on Oct 9, 2026
    ContributorAuthor

    (lane openRepoShape-2, 2026-10-09T02:25:43Z) REVIEW of fix/adopt-split-force-add-shape-files, the #167 branch, at 119d666 (one commit over main 7f84ca4). Read-only, own detached worktrees; nothing authored, no claim held. Full report with drivers and every run log: opensoft/brett-wip handoffs/xFactory/attachments/lane-openRepoShape-2/review167-119d666/report.md. Lines at 119d666.

    Verdict: MERGEABLE WITH NITS. The fix is correct for #167 and needs no change on its own. #167 reproduced on main (.* + !.gitignore: "adoption verified", split lacks .gitattributes and validate.yml, bootstrap.py exit 1 shape-copy-missing, make bootstrap exit 2) and is fixed at 119d666 (all 11 pin rows present, bootstrap 0, make bootstrap 0). Every ignore source tried is beaten by the -f staging: a root .gitignore (.*, **/*.yml, contracts/, .github/, scripts/, shape/), tracked .gitignores inside .github/ and contracts/, info/exclude, a global core.excludesFile, export-ignore attributes, a spec-only adoption with --allow-empty-leg code, and three submodule plans; strays the source ignores stay out. _account_for is untouched, so AGENTS.md step 7's "every source path by blob sha" still holds; only "added" grows (17 to 19). The staging argv is exactly materialized.written (18 paths, collision copies at shape/<rel>, POSIX spelling), never -f .. Commit-message claims all hold except two edges below. docs/cli.md unchanged; git diff --check clean. Tests at 119d666 (adopt subset + cli_reference + the new test_adopt_ignored_shape_files.py): 121 passed / 1 skipped (SHAPE_MEDXEHR_CLONE unset), no filter-repo skips; with main's adopt-project.py substituted, 15 of the 15 new tests fail, 5 of them on the right assertion.

    Cross-PR sequencing (the one P2, already resolved upstream). git merge of 119d666 with #165's EARLIER head d780495 is textually clean and then fails all 10 of tests/test_adopt_verify.py with adopt-shape-file-missing: contracts/shape-pin.yaml, because #165's fixtures build split commits with no pin. #165's current head 05fa0e0 already adds the pin to ROOT_FILES and asserts added 3; 119d666 + 05fa0e0 passes 14/14, and 119d666 + 09d7889 + 05fa0e0 gives 182 passed / 1 skipped. So #165 must land at 05fa0e0 or later, never at d780495; nothing to change here. Lane 1's "#167 is independent" holds for #166 (119d666 + 09d7889: clean, 168 passed) and for #168 (119d666 + a25dcd7: clean, 45 passed in the two new test files).

    P3

    1. The check reads contracts/shape-pin.yaml, which is not always the pin the shape wrote. adopt-project.py:1803, :1844. When the source keeps its OWN contracts/shape-pin.yaml at the root (contracts/ moved from its default spec to root), the shape's pin lands at shape/contracts/shape-pin.yaml and _shape_file_findings reads the source file: a non-pin fails closed (adopt-shape-pin-unreadable, naming the collision), a source pin whose rows happen to be in the tree passes vacuously ("adoption verified", bootstrap 1), and a source pin naming src/app/main.py (sent to code) yields a false adopt-shape-file-missing whose remedy git add -f -- src/app/main.py cannot work. Rare (needs that collision AND the human moving contracts/). Fix: compare materialized.written with the split tree instead of parsing a pin; that needs no YAML and also closes 3.
    2. Narrow regression: a symlinked directory the shape writes into, kept at the root, now aborts execute after both legs are pushed. :1488. Source with contracts -> governance (or .github -> ci) symlink kept at root and no .gitignore: main exits 0, verified, bootstrap 0 (the files are written through the link and committed at its target); 119d666 exits 2 fatal: pathspec 'contracts/spec-pin.yaml' is beyond a symbolic link, legs already created, no split branch. Fix: in place() (scripts/shape_materialize.py:993-1011) treat a target whose parent resolves through a symlink or into a gitlink as a collision, or have check refuse such a root-kept path before any leg exists.
    3. Only PINNED files are verified, not "every file the shape wrote". The 6 templated files (README.md, project.yaml, contracts/spec-pin.yaml, contracts/code-pin.yaml, AGENTS.md, CLAUDE.md) are not checked: with the staging monkeypatched back to -A (as the PR's own last test does) and those paths ignored, execute says verified, bootstrap.py exits 0 ("no project.yaml: this project has not elected the schema") and make validate exits 2 yaml-unreadable: project.yaml. Reachable only if the staging regresses. Same fix as 1.
    4. "160000" literal at :1842; adopt-project.py _verify skips every root gitlink and compares a kept root .gitmodules byte for byte, so any plan that keeps a source submodule or its .gitmodules in the root ends with exit 1 and adopt-lost #165 and adopt-project.py: a plan that sends .gitmodules to one leg and a submodule it registers to another passes check and execute, says adoption verified, and the assembly's recursive clone then fails: plan should ask, check should find, execute should refuse #166 both add GITLINK_MODE for exactly this. Whichever lands second should use it (in the three-way merge the constant is at :153 and this literal at :2168).
    5. Contradictory remediation in one output: the per-file finding says "stage it with git add -f, commit, and push the split branch again" (:1816-1821) while _verify's tail says "fix the plan and re-run into a fresh --local-remote-dir"; in the collision case neither is the remedy. Make the tail conditional on which findings fired.
    6. Pre-existing, outside the diff: a source .gitignore that hides a mount path (* + !.gitignore, or spec/code) dies at git submodule add (:1291, no --force) with "The following paths are ignored" AFTER both legs are pushed, on main and here alike. Filed separately by this lane.
    7. Pre-existing sibling, outside the diff: scaffold-project.py:809 (and git_init_commit at scripts/shape_materialize.py:520, scripts/family.py:669) stages with git add -A -- . and honours a global core.excludesFile: a scaffold under excludes .*, contracts/, scripts/ exits 0 and the new repository lacks .gitattributes, .github/, .gitignore, contracts/*, scripts/*. Same class as adopt-project.py _commit_the_split stages with git add -A -- ., so a source .gitignore of .* silently drops .gitattributes and validate.yml from the split, execute still says adoption verified, and the assembly fails its first bootstrap #167, same fix. Filed separately by this lane.
    8. Docs: README.md:777-781 and AGENTS.md step 7 still describe the verification as source-path accounting only; adopt-shape-file-missing and adopt-shape-pin-unreadable are undocumented.
    9. Untested: _pinned_shape_paths' UnicodeDecodeError arm, a non-mapping or empty pin, files: [], a non-mapping row; the collision case (1); git add -f failing (2); spec-only, info/exclude and core.excludesFile under .* (all pass by hand, none pinned). Trivia: _tree_of(assembly, split_commit) now runs twice per _verify.

    Not verified: #166's unpushed local head (merge textually clean only), the Windows CI job, an LFS/filter-driver .gitattributes (applies equally to -A), and the GitHub-side execute path (unchanged code).

  6. brettheap
    added 2 commits that reference this issue on Oct 9, 2026

    adopt-project.py: a shape file written through a symlink the plan kee…

    b7083ca

    adopt-project.py: a pinned path counts as present only where it leads…

    b635473

  7. brettheap commented on Oct 9, 2026

    @brettheap
    brettheap
    on Oct 9, 2026
    ContributorAuthor

    (lane openRepoShape-2, 2026-10-09T03:31:00Z) DELTA REVIEW of fix/adopt-split-force-add-shape-files, the #167 branch, at b635473 (commits b7083ca and b635473 after the 119d666 this lane reviewed at issuecomment-6072995006). Read-only, own detached worktrees; nothing authored, no claim held. Full report with drivers and run logs: opensoft/brett-wip handoffs/xFactory/attachments/lane-openRepoShape-2/review167-b635473/report.md. Lines at b635473.

    Verdict: MERGEABLE WITH NITS. #167's reproduction is fixed (11/11 pin rows present, bootstrap 0); the symlink regression from the prior review is fixed (C14, C14b, C14c: verified, bootstrap 0, where 119d666 died "beyond a symbolic link" after the legs); every ignore source is still beaten (root patterns, subdirectory .gitignores, info/exclude, core.excludesFile, export-ignore); the new _SplitTree symlink resolver matches a real checkout on 12,325 differential fuzz probes with 0 mismatches; the new pin-reading arms all fail closed; the per-finding remediation now matches the footer. Not rebased onto main (merge-base 7f84ca4), but merges CLEAN with main b18cca9, with #166's 794828c, with #165's 05fa0e0, and with all three; adopt tests green on each (166 / 238 / 150 / 252 passed, 1 skip SHAPE_MEDXEHR_CLONE, no filter-repo skip); 136 passed / 1 skipped on the branch alone (+15 tests over 119d666); render-cli-reference.py --check ok everywhere; the b635473 claim about the last link is proved by mutation. Prior findings: FIXED 3 (the #165 sequencing P2, upstream; P3-2 symlinked root; P3-5 remediation), PARTIAL 2 (P3-1, P3-9), NOT TAKEN 3 (P3-3 only pinned files verified; P3-4 the "160000" literal, now at :1859 beside the new SYMLINK_MODE/FILE_MODES, while GITLINK_MODE lands with #165/#166; P3-8 docs), out of scope 2 (#174, #175).

    Still open from the prior review, P3-1 (PARTIAL): the check reads the ROOT contracts/shape-pin.yaml, which is the SOURCE's after a collision. :1986-2019. C9a (source pin kind: other kept at root) now exits 1 adopt-shape-pin-unreadable with the plan answer named; C9c exits 1 with one finding and no git add advice; but C9b (source pin files: [README.md]) still prints "adoption verified" while bootstrap fails, and so does the same source through a symlink (D13: contracts -> governance plus the source's governance/shape-pin.yaml). So b7083ca's claim "the check now agrees with bootstrap everywhere" does not hold for those two (N2 below). Fix as before: compare materialized.written against the split tree, which needs no pin parsing and also closes P3-3.

    NEW

    N1 (P3). A root-kept symlink that leads INTO a leg mount still aborts execute after both legs are pushed. :1524-1527 (_where_it_landed returns the in-mount path) and :1507 (the forced add). Source with a tracked .github -> spec answered root: check 0; execute exit 2 fatal: Pathspec 'spec/workflows/validate.yml' is in submodule 'spec', both leg remotes hold commits, no split branch, and the shape's validate.yml is left untracked inside the spec leg's working tree. Same on the three-way merge. (main: verified, bootstrap 1; 119d666: exit 2 "beyond a symbolic link".) Expected: what the docstring promises for a link that leads out, not staged and reported as one adopt-shape-file-missing, or refused by check first. Fix: return None from _where_it_landed when the landed path lies at or under a gitlink of the assembly index (the two mounts). (tools/s_delta.py wt D7b.)

    N2 (P3). b7083ca's message overclaims "agrees with bootstrap everywhere ... the same findings as before on the splits whose bootstrap fails": true on C14b/C14c (0 findings, bootstrap 0, which 119d666 misreported) and on every failing split except C9b and D13 (0 findings, bootstrap fails). Minor: C14 goes from 1 finding to 2, more precise. Either close P3-1 or narrow the sentence in the PR body.

    N3 (P3, pre-existing, adjacent to the delta). check refuses no root-kept symlink at or above a shape path that dangles or leaves the tree. scripts/shape_materialize.py:996 (path.exists() is False for a dangling link) and :1007 (mkdir). D4 (.gitattributes -> nowhere/attrs), D5 (.github -> tooling/gh with tooling/ sent to code) and D7 (.github -> spec/ci): a Python TRACEBACK (FileNotFoundError / FileExistsError) after both legs are pushed, exit 1, no split branch, identical on main. D6 (.github -> ../outside-ci) and D6b (absolute target): the materializer WRITES the shape's validate.yml outside the assembly; main says verified (D6b even bootstraps on this machine), b635473 now exits 1 with a correct finding but the file is still written outside. Filed separately by this lane; distinct from #169 (non-canonical leg paths) and #172 (mount paths colliding with shape directories).

    Also noted: .github -> .git/info (D9): git add -f of a path inside .git silently stages nothing, and the verification correctly reports it. Non-ASCII and space in a link target (D8b) adopt and bootstrap. The brief's exact pytest command exits 4 on the branch alone because tests/test_adopt_execute_refusals.py arrives with #168 on main; it passes once merged. Unverified: Windows and macOS CI, Python 3.9, the GitHub-side execute path, the full suite. Lane 1's local #166 worktree stood at ca5fb3d, not on origin; not reviewed.

  8. brettheap
    closed this as completedin 21f5c33on Oct 9, 2026
  9. brettheap
    added 4 commits that reference this issue on Oct 9, 2026

    Merge origin/main (21f5c33: #167, #168) into fix/adopt-canonical-leg-…

    5ab1a21

    Merge main (21f5c33, #167 via #182) into fix/adopt-split-submodule-plan

    5bef216

    adopt-project.py: the seeded leg of a source with no path for it is s…

    6c8f50f

    adopt-project.py: `plan` predicts the shape's collisions from `assemb…

    163173e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions