From 1c094bd53a6d6fe0777de05aa9dc7dc3124d59c1 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 9 Sep 2026 12:03:09 -0400 Subject: [PATCH 1/3] The family's workstation layout: init lands at //, and `make siblings` clones each member beside the holder without ever moving one (#76) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Brett Heap's RULING of 2026-09-09 on issue #76, walked through as multiple choice: (1a) the holder's target is `make siblings`, which clones each `family.yaml` member as a working copy beside the holder, on its tracking branch, idempotently, warns and prints the `mv` when the parent folder is not ``, and never moves anything; (1b) ONE implementation, TWO entry points, no second code path; (3) the doubled `/` layout stands. Unchanged from the body: `init` lands at `//`, and nothing ever moves a checkout. This is slice (A), the family side; the scaffold side (`--family` in setup-project.py) is a separate branch. WHY. The plain folder in the original family ruling — "like a holder FOLDER and some utilities for the family of services" (2026-09-04) — was never written down, and existed anyway, arranged by hand on every workstation. `init` materialized the holder in a TEMP directory unless `--work-dir` said otherwise, so the person moved it afterwards; nothing produced the members' working clones at all, and the first question anybody asked — "is InkRouter/InkRouter the family, and what is the parent folder?" — had no answer in any file. `init` now lands where you are standing: `--into ` is the PARENT directory (the invocation directory by default), the plain folder `/` is created in it, and the holder is cloned inside that. Standing in the family folder already, it does not nest a second one — and when that folder already holds its holder the run is refused by name rather than nesting a holder inside a holder. `--work-dir` keeps exactly its old meaning (`/`, no folder), because every rehearsal in this suite passes it; the two flags together are refused. The plan and the `--dry-run` print the landing path, and the closing block names the folder, the holder, both next commands and which of the two copies of a member you work in. `templates/family-root/scripts/siblings.py` is the implementation, copied into holders and digest-pinned like `bootstrap.py` and `validate-family.py` — which is what carries it to InkRouter/InkRouter, the holder that already exists, through `update-shape.py check` -> `upstream-added` -> `apply --add scripts/siblings.py`, per file, on a human's word. `make siblings` runs the copy; `family.py siblings --family-root ` runs THAT SAME FILE through `runpy`, the way the root `bootstrap` shim runs the assembly root's bootstrap, so it works from a checkout of the standard before the holder is anywhere else. It reads the member urls the way `make bootstrap` does — out of `.gitmodules`, `family.yaml`'s `repository:` as the fallback — and imports the credential resolution and the member rows from the `bootstrap.py` beside it rather than defining either twice. What it refuses to do is most of it. An existing clone is FETCHED and nothing else: no checkout, no reset, no pull, no branch switch, because somebody is working in there — and it is first verified to BE that member (origin, and its own `project.yaml` id), with a mismatch reported and skipped rather than written into. A directory that is not a clone at all is left alone. When the holder's parent folder is not named after the family it WARNS and prints the exact `mv` — including the staging name `mv A A/A` cannot be spelled without — and moves nothing: relocating a checkout breaks an open shell, an editor and any linked worktree, whose `.git` file carries an absolute path. The doctor pattern, not a mover, which is the half of the ruling that keeps this from being the thing the issue deliberately did not propose. ONE COMMIT, not two: the init landing and the siblings target are separable in code and not in prose. `init`'s own closing block names `make siblings`, and the README, AGENTS.md, the holder's README and its AGENTS-shape.md each explain the folder, the two copies of every member and the never-moves rule as one thing. A commit whose output named a target that did not exist yet would be a commit that has to be read with the next one open. Docs: the holder's README gains a "Workstation layout" section (the tree, the two copies and which one you work in, the doubled name, where handoff files live) and a `make siblings` row; its AGENTS-shape.md now has four commands and says what the utility never does; the README's Families section gains the layout, the landing rule and the extended InkRouter tree; AGENTS.md's family procedure gains the landing and the rule that the assistant does not run that `mv` either. Tests, all offline into bare repositories in a temporary directory: init's default landing, `--into`, standing in the family folder, the refusal when that folder already holds a holder, `--work-dir` unchanged, both flags together, a non-empty landing, the dry run; and for siblings — clones beside the holder on their tracking branch with the legs placed (through `make siblings`), a second run all `present`, a clone somebody is working in untouched (branch and uncommitted file), a clone of another repository reported and skipped non-zero, a directory that is not a clone, a remote spelled differently accepted, an id that is not the row's skipped, a relative submodule url resolved against this remote, the parent-folder warning with the exact `mv` and nothing moved, the dry run, and `family.py siblings` producing the same layout table as `make siblings`. `test_family.py` asserts the fresh holder's `shape-pin.yaml` carries the new row and that the copy is byte-identical and executable; `test_update_shape_add.py` proves an existing holder is offered it as `upstream-added` and a project is offered nothing, because only the family's list names it. Caps moved with their rationale: README 1014 -> 1059, AGENTS.md 277 -> 303. Lane: xfactory-2 (xFactory-2) Co-Authored-By: Claude Fable 5.1 --- AGENTS.md | 30 +- README.md | 61 ++- scripts/family.py | 171 ++++++- scripts/shape_materialize.py | 12 +- templates/family-root/AGENTS-shape.md | 22 +- templates/family-root/Makefile | 13 +- templates/family-root/README.md | 52 ++ templates/family-root/scripts/siblings.py | 551 ++++++++++++++++++++++ tests/test_family.py | 425 +++++++++++++++++ tests/test_repo_hygiene.py | 38 +- tests/test_update_shape_add.py | 137 ++++-- 11 files changed, 1444 insertions(+), 68 deletions(-) create mode 100755 templates/family-root/scripts/siblings.py diff --git a/AGENTS.md b/AGENTS.md index 087d6246..0aa8fb7a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -244,6 +244,7 @@ that ship together already are. python3 scripts/family.py init --org --family [--reuse-empty-repo] python3 scripts/family.py add --family-root --member / python3 scripts/family.py bump --family-root --member --to +python3 scripts/family.py siblings --family-root ``` 1. **`init` creates a repository, so the human must say so first.** There is @@ -251,6 +252,13 @@ python3 scripts/family.py bump --family-root --member --to /` already exists and is EMPTY, add `--reuse-empty-repo`; if it has commits, stop and ask. + **It lands at `//`** — `--into` is the PARENT + directory and defaults to where you are standing, the folder `/` + is created in it, and the holder goes inside that. Standing in the family + folder already, it does not nest a second one. `--work-dir ` is the + override (`/`, no folder) and the two are refused together. + Say the landing path back to the human: the doubled `/` is + deliberate and reads like a mistake until somebody explains it. 2. **A family pins ASSEMBLY ROOTS, never legs.** `--member /`, never `-spec`. The tool refuses a leg, and the refusal is correct: a leg has no `project.yaml` and belongs to its own root. @@ -260,8 +268,26 @@ python3 scripts/family.py bump --family-root --member --to `, the same file — places the WORKSTATION layout**: each member + cloned BESIDE the holder, in the family folder, on its tracking branch, + with its own bootstrap run. Two copies of every member is intended: + `members/` inside the holder is pinned and DETACHED, for + `bootstrap` and `validate`; the sibling is where the human works. + **What it never does, and neither do you:** it MOVES NOTHING — when the + parent folder is not named after the family it warns and prints an `mv`, + and running that `mv` yourself relocates a checkout out from under a + shell, an editor and every agent lane, and breaks any linked worktree + (its `.git` file carries an absolute path). It also never touches an + existing clone beyond `git fetch`: no checkout, no reset, no pull, + because somebody is working in there. A directory that is a clone of a + different repository is reported and skipped; the exit is the human's + hand, not an overwrite. +6. `update-shape.py` updates a family root exactly as it updates a project, + and the same four refusals apply. It mirrors into `family.yaml`. An + existing holder receives `scripts/siblings.py` that way: `check` reports + it as `upstream-added` and `apply --add scripts/siblings.py` takes it, on + the human's word, per file. ## What you must not tell them diff --git a/README.md b/README.md index c6d7a7dc..5f98aaa3 100644 --- a/README.md +++ b/README.md @@ -756,22 +756,67 @@ them one gate, one release and one pin, which is the opposite of what "deploy separately" means. **Use one project when the parts ship together** — that is what the spec and code legs already are. +### The workstation layout: a plain folder, and two copies of every member + +Ruled by **Brett Heap on 2026-09-09** (issue #76). The holder is a repository; +the thing AROUND it is a **plain folder named after the family**, and the +members' working clones sit beside the holder in it: + +``` +InkRouter/ a PLAIN FOLDER, not a git repository + InkRouter/ the holder: family.yaml, members/ (pinned, detached) + IRRS/ a WORKING clone of InkRouter/IRRS, on main + IRSS/ a WORKING clone of InkRouter/IRSS, on main + IRRS-worktrees/… whatever the member's own work needs + session-handoff-*.md the estate's own files live in the folder +``` + +`scripts/family.py init` lands the holder there: `--into ` is the PARENT +directory (the directory you are standing in, by default), the folder +`/` is created in it, and the holder is cloned INSIDE that at +`//` — the same "lands where you were standing" rule +projects have. Standing in the family folder already, it does not nest a +second one. `--work-dir` remains the override that lands `/` +and makes no folder. The doubled name is kept on purpose: the holder keeps its +own repository name and `family.yaml` is what tells the two apart. + +`make siblings` in the holder — or `python3 scripts/family.py siblings +--family-root `, the same file through the other entry point — clones +each member at `../` from the same url `make bootstrap` uses, then +runs the member's own bootstrap so its legs are on their tracking branches. +**TWO COPIES OF EVERY MEMBER is correct and intended:** `members/` +inside the holder is pinned and DETACHED, and is what `bootstrap` and +`validate` read; the sibling beside the holder is where a human works, and +`family.py bump` is how the family follows it. An existing clone is +**fetched and nothing else** — no checkout, no reset, no pull — and one that +is a different repository is reported and skipped. It is idempotent. + +**Nothing ever moves a checkout.** When the holder's parent folder is not +named after the family, `make siblings` WARNS and prints the exact `mv` for a +human to run. A tool that relocated the directory would do it out from under +the shell, the editor and every agent lane standing in it, and would break +linked worktrees, whose `.git` files carry absolute paths — an act by +surprise, which is the one thing every refusal here exists to avoid. + ### The InkRouter example Eight services, each its own project, one holder: ``` -InkRouter/ the family holder — family.yaml, members/, utilities - members/IRRS -> InkRouter/IRRS assembly root, mounting IRRS-spec and IRRS-code - members/IRSS -> InkRouter/IRSS assembly root, mounting IRSS-spec and IRSS-code - …six more, added as they arrive +InkRouter/ the family FOLDER (see the layout above) + InkRouter/ the family holder — family.yaml, members/, utilities + members/IRRS -> InkRouter/IRRS assembly root, mounting IRRS-spec and IRRS-code + members/IRSS -> InkRouter/IRSS assembly root, mounting IRSS-spec and IRSS-code + …six more, added as they arrive + IRRS/ IRSS/ … the same members as WORKING clones, on main ``` ```sh -python3 scripts/family.py init --org InkRouter --family InkRouter --reuse-empty-repo -python3 scripts/family.py add --family-root ../InkRouter --member InkRouter/IRRS -python3 scripts/family.py bump --family-root ../InkRouter --member IRRS --to <40 hex> -python3 scripts/family.py remove --family-root ../InkRouter --member IRRS +python3 scripts/family.py init --org InkRouter --family InkRouter --into .. --reuse-empty-repo +python3 scripts/family.py add --family-root ../InkRouter/InkRouter --member InkRouter/IRRS +python3 scripts/family.py bump --family-root ../InkRouter/InkRouter --member IRRS --to <40 hex> +python3 scripts/family.py remove --family-root ../InkRouter/InkRouter --member IRRS +python3 scripts/family.py siblings --family-root ../InkRouter/InkRouter ``` `add`, `bump` and `remove` each write ONE commit, with explicit pathspecs, diff --git a/scripts/family.py b/scripts/family.py index 25518311..63c1fab0 100755 --- a/scripts/family.py +++ b/scripts/family.py @@ -2,10 +2,11 @@ # SPDX-License-Identifier: Apache-2.0 """Create and maintain a FAMILY: a holder that pins member assembly roots. - ./scripts/family.py init --org --family [--dry-run] - ./scripts/family.py add --family-root --member / - ./scripts/family.py bump --family-root --member --to - ./scripts/family.py remove --family-root --member + ./scripts/family.py init --org --family [--into ] + ./scripts/family.py add --family-root --member / + ./scripts/family.py bump --family-root --member --to + ./scripts/family.py remove --family-root --member + ./scripts/family.py siblings --family-root A FAMILY IS NOT A PROJECT, and this is a ruling (Brett Heap, 2026-09-04), made about InkRouter: @@ -41,7 +42,18 @@ "deploy separately" means. The family is the cheapest thing that answers the only question the estate actually had: how does somebody clone all of them. +THE WORKSTATION LAYOUT (Brett Heap, 2026-09-09, issue #76). `init` lands the +holder at `//`: a PLAIN FOLDER named after the family, +holding the holder clone and — once `siblings` has run — the members' working +clones beside it. The doubled name is kept on purpose: the holder is a +repository and keeps its own name, and `family.yaml` is what tells the two +apart. `siblings` is the holder's own `make siblings`, run from a checkout of +this standard so it works before the holder is anywhere else; there is ONE +implementation of it, `templates/family-root/scripts/siblings.py`, and this +subcommand runs that file rather than a second copy of its logic. + EXIT CODES: 0 done · 1 nothing (a dry run prints and exits 0) · 2 a refusal. +`siblings` exits with the template script's own code, unchanged. """ from __future__ import annotations @@ -51,9 +63,9 @@ import json import os import re +import runpy import subprocess import sys -import tempfile from pathlib import Path SHAPE_ROOT = Path(__file__).resolve().parents[1] @@ -363,6 +375,48 @@ def _local_remote_is_empty(bare: Path) -> bool | None: # --------------------------------------------------------------------------- +SIBLINGS_SCRIPT = (SHAPE_ROOT / "templates" / "family-root" / "scripts" / + "siblings.py") + + +def _landing(args, family: str) -> tuple[Path | None, Path]: + """`(the family folder, the holder)` — WHERE `init` PUTS THINGS. + + THE DEFAULT LANDS WHERE YOU ARE STANDING, which is the rule #39 gave + projects and the ruling of 2026-09-09 gives families: `--into` is the + PARENT directory (the invocation directory by default), the family FOLDER + `/` is created in it, and the holder is cloned INSIDE that at + `//`. The folder is a plain directory and never a + repository; the doubled name is the holder keeping its own repository + name, and `family.yaml` is what tells the two apart. + + STANDING IN THE FAMILY FOLDER ALREADY is the one exception: when the + directory is itself named `` and holds no `/family.yaml`, + a second `/` would nest a folder inside the folder the person is + already in, so the holder lands at `/`. When it DOES hold + one, the family exists and this is not an init — refused by name below. + + `--work-dir` keeps today's semantics untouched (`/`, no + folder made) because every test and every rehearsal in this suite passes + it, and a flag that quietly changed meaning is worse than a second flag. + """ + if args.work_dir is not None: + return None, args.work_dir.expanduser().resolve() / family + into = (args.into or Path.cwd()).expanduser().resolve() + if into.name == family: + if (into / family / MANIFEST).is_file(): + raise Refusal( + "family-holder-already-here", + f"{into} is already the {family} family folder and its holder " + f"is at {into / family}, so there is nothing to init", + "Remediation: `family.py add --family-root " + f"{into / family} --member /` grows it, and " + "`make siblings` in the holder puts each member's working " + "clone beside it. There is no --force.") + return into, into / family + return into / family, into / family / family + + def cmd_init(args) -> int: # noqa: C901 family = checked_value("--family", args.family) args.org = checked_value("--org", args.org) @@ -422,11 +476,26 @@ def cmd_init(args) -> int: # noqa: C901 "DIGEST_DEFINITION": TREE_DIGEST_DEFINITION, } + if args.into is not None and args.work_dir is not None: + raise Refusal( + "family-two-landings", + "--into and --work-dir both name where the holder goes, and they " + "do not mean the same thing", + "Remediation: pass --into for the workstation layout " + f"(/{family}/{family}), or --work-dir for the bare " + f"override (/{family}, no family folder). Not both.") + folder, holder = _landing(args, family) + print(f"family {values['FAMILY_NAME']} ({family_id})") print(f"shape {SHAPE_REPOSITORY} @ {shape_commit[:12]} " f"(tree {shape_tree[:12]}…)") print(f"created by {values['CREATED_BY']} on {values['CREATED_ON']}") print(f" holder {repository:<28} -> {url}") + if folder is None: + print(f"landing {holder} (--work-dir: no family folder)") + else: + print(f"folder {folder} a plain folder, NOT a repository") + print(f"landing {holder} the holder clone, inside it") print(f"members mounted under {MEMBERS_DIR}/; none yet — " "`family.py add` puts one there") print("remotes " + ("a bare repository on disk (no network)" if local @@ -440,14 +509,17 @@ def cmd_init(args) -> int: # noqa: C901 print("\n--dry-run: nothing was created.") return 0 - work_root = (args.work_dir.resolve() if args.work_dir - else Path(tempfile.mkdtemp(prefix="openreposhape-family-"))) - work_root.mkdir(parents=True, exist_ok=True) - work = work_root / family + # `mkdir -p` on the FOLDER, which is fine if it is already there — a + # family folder somebody made by hand, or one a scaffolded member already + # landed in, is exactly the case this is supposed to join. + work = holder + (folder or work.parent).mkdir(parents=True, exist_ok=True) if work.exists() and any(work.iterdir()): raise Refusal( "family-target-exists", f"{work} already exists and is not empty", - "Remediation: choose an empty --work-dir. There is no --force.") + "Remediation: choose another --into (or an empty --work-dir); if " + "that IS the holder, it is already a family and `family.py add` " + "is what grows it. There is no --force.") # THE ONE REPOSITORY THAT MAY ALREADY EXIST is the holder, and only with # --reuse-empty-repo, and only with ZERO commits. `InkRouter` in the @@ -525,13 +597,20 @@ def cmd_init(args) -> int: # noqa: C901 {PYTHON} {Path(__file__).name} add --family-root {work} \\ --member {args.org}/ - git -C {work} push + cd {work} && make siblings `add` mounts the member at {MEMBERS_DIR}/, records its pin, and -writes ONE commit. Then `make bootstrap` in the holder fetches every member -and its legs and runs each member's own bootstrap. +writes ONE commit; land it as a pull request. `make siblings` then puts each +member's WORKING clone beside the holder, in the family folder, on its +tracking branch. + +TWO COPIES OF EVERY MEMBER, on purpose: pinned and DETACHED inside the holder +under {MEMBERS_DIR}/ (what `make bootstrap` and `make validate` read), and a +working clone BESIDE the holder — that second one is where you work, and +`family.py bump` is how the family follows it. -Working tree at {work} +family folder {folder if folder is not None else work.parent} +holder {work} """) return 0 @@ -744,6 +823,49 @@ def cmd_remove(args) -> int: return 0 +# --------------------------------------------------------------------------- +# siblings +# --------------------------------------------------------------------------- + + +def cmd_siblings(args) -> int: + """The holder's `make siblings`, run from a checkout of the standard. + + ONE IMPLEMENTATION, TWO ENTRY POINTS (Brett Heap, 2026-09-09, ruling 1b: + *"One implementation, two entry points… No second code path."*). The + implementation is `templates/family-root/scripts/siblings.py`, the file a + holder CARRIES and its Makefile calls; this subcommand runs that same file + against `--family-root`, exactly as the root `bootstrap` shim runs the + assembly root's `scripts/bootstrap.py`. That is what makes it work before + the holder is cloned anywhere else — the standard has the utility, the + holder is only where it usually lives. + + `runpy` rather than an import: the script is a `__main__` program that + ends in `sys.exit(main())`, so its exit code is the one this process + exits with, unchanged and unrewritten. + """ + root = Path(args.family_root).expanduser().resolve() + if not (root / MANIFEST).is_file(): + raise Refusal( + "family-root-missing", + f"{root / MANIFEST} does not exist, so this is not a family root", + "Remediation: `family.py init --org --family ` creates " + "one; point --family-root at the HOLDER (the `/` " + "clone), not at the family folder around it.") + if not SIBLINGS_SCRIPT.is_file(): + raise Refusal( + "siblings-script-missing", + f"{SIBLINGS_SCRIPT} is missing from this checkout of the standard", + "Remediation: this subcommand runs the holder's own copy of that " + "file and has no second implementation to fall back on. Re-clone " + "openRepoShape, or run `make siblings` in the holder, which uses " + "its own copy.") + sys.argv = ["siblings", "--root", str(root), + *(["--dry-run"] if args.dry_run else [])] + runpy.run_path(str(SIBLINGS_SCRIPT), run_name="__main__") + return 0 # pragma: no cover - the script exits before returning here + + # --------------------------------------------------------------------------- @@ -773,7 +895,15 @@ def build_parser() -> argparse.ArgumentParser: init.add_argument("--local-remote-dir", type=Path, default=None, help="create a bare repository here instead of calling " "`gh repo create` (the TEST path; no network)") - init.add_argument("--work-dir", type=Path, default=None) + init.add_argument("--into", type=Path, default=None, metavar="DIR", + help="the PARENT directory the family folder is made " + "in (default: the directory you are standing in). " + "The holder lands at //, " + "beside where `make siblings` clones the members.") + init.add_argument("--work-dir", type=Path, default=None, + help="THE OVERRIDE: land the holder at " + "/ and make no family folder at " + "all. Mutually exclusive with --into.") init.add_argument("--no-push", action="store_true") init.add_argument("--dry-run", action="store_true") init.set_defaults(func=cmd_init) @@ -803,6 +933,17 @@ def build_parser() -> argparse.ArgumentParser: remove.add_argument("--family-root", required=True) remove.add_argument("--member", required=True, metavar="PROJECT") remove.set_defaults(func=cmd_remove) + + siblings = subparsers.add_parser( + "siblings", + help="clone every member BESIDE the holder, on its tracking branch") + siblings.add_argument("--family-root", required=True, + help="the HOLDER (`/`); the " + "siblings are placed in the folder around it") + siblings.add_argument("--dry-run", action="store_true", + help="say what would be cloned and fetched; clone " + "nothing, fetch nothing, move nothing") + siblings.set_defaults(func=cmd_siblings) return parser diff --git a/scripts/shape_materialize.py b/scripts/shape_materialize.py index cbf80698..b98dc7bf 100644 --- a/scripts/shape_materialize.py +++ b/scripts/shape_materialize.py @@ -216,6 +216,12 @@ def default_reference(elected_on: str) -> str: FAMILY_COPIED_VERBATIM = ( "scripts/validate-family.py", "scripts/bootstrap.py", + # The workstation utility (2026-09-09, #76): it clones each member BESIDE + # the holder, on its tracking branch, and warns about the parent folder + # without moving anything. Copied and pinned like the other two, which is + # what carries it to an existing holder through `update-shape.py check` -> + # `upstream-added` -> `apply --add scripts/siblings.py`. + "scripts/siblings.py", "Makefile", ".gitignore", # The holder carries the same copy pin, so it carries the same statement @@ -237,7 +243,11 @@ def default_reference(elected_on: str) -> str: ("scripts/repo_shape.py", "scripts/repo_shape.py"), ("contracts/repository-naming.yaml", "contracts/repository-naming.yaml"), ) -FAMILY_EXECUTABLE = ("scripts/validate-family.py", "scripts/bootstrap.py") +FAMILY_EXECUTABLE = ( + "scripts/validate-family.py", + "scripts/bootstrap.py", + "scripts/siblings.py", +) PLACEHOLDER_RE = re.compile(r"\{\{[A-Z_]+\}\}") diff --git a/templates/family-root/AGENTS-shape.md b/templates/family-root/AGENTS-shape.md index 91f0e9af..05684948 100644 --- a/templates/family-root/AGENTS-shape.md +++ b/templates/family-root/AGENTS-shape.md @@ -37,10 +37,11 @@ not hand-edit the `members:` block — the tool rewrites it wholesale. Land the commit as a pull request; the member must already be scaffolded or adopted, because a family cannot make a project out of a repository. -## The three commands +## The four commands ``` make bootstrap every member and its legs, then each member's own bootstrap +make siblings each member cloned BESIDE this holder, on its tracking branch make validate the family validator, then each member's `make validate` make pins the family's lockstep check alone ``` @@ -49,6 +50,25 @@ make pins the family's lockstep check alone pinned commits, by running the member's own bootstrap rather than reimplementing it. +`make siblings` places the WORKSTATION layout: a plain folder named after the +family, this holder inside it, and each member's working clone beside the +holder (`../`). Two copies of every member is correct and intended — +`members/` here is pinned and DETACHED, for `bootstrap` and +`validate`; the sibling is where a human works, and `family.py bump` is how +this family follows it. What it will NOT do, and you must not do either: + +* **It moves nothing.** When the parent folder is not named after the family + it WARNS and prints the exact `mv` for a human to run. Do not run that `mv` + on your own initiative: relocating a checkout breaks an open shell, an + editor and any LINKED WORKTREE, whose `.git` file carries an absolute path. +* **It never touches an existing clone beyond `git fetch`.** No checkout, no + reset, no pull, no stash, no branch switch — somebody is working in there. + A directory that is a clone of a different repository is reported and + SKIPPED, and the exit is to move it aside by hand, never to overwrite it. +* Run from a checkout of openRepoShape instead, when this holder is not yet + on the machine twice: `python3 scripts/family.py siblings --family-root + ` runs the very same file — one implementation, two entry points. + ## Never edit a file that has a row in `contracts/shape-pin.yaml` Those rows are the copies of openRepoShape this holder carries — the family diff --git a/templates/family-root/Makefile b/templates/family-root/Makefile index 1d17f0c4..c5ac4a69 100644 --- a/templates/family-root/Makefile +++ b/templates/family-root/Makefile @@ -1,6 +1,10 @@ -# The three targets a family holder ships. Copied verbatim from +# The four targets a family holder ships. Copied verbatim from # openRepoShape and digest-pinned in `contracts/shape-pin.yaml`. # +# `siblings` is the WORKSTATION half and the only target that writes outside +# this repository: it clones each member BESIDE the holder, in the family +# folder, and it never touches a clone that is already there beyond a fetch. +# # `bootstrap` and the member half of `validate` are done by # `scripts/bootstrap.py` rather than by a shell loop here: it must resolve a # credential for PRIVATE members before `git submodule update --init @@ -9,17 +13,22 @@ PYTHON ?= python3 .DEFAULT_GOAL := help -.PHONY: help bootstrap validate pins +.PHONY: help bootstrap siblings validate pins help: @echo 'make bootstrap fetch every member and its legs, then run each' @echo ' member'\''s own make bootstrap' + @echo 'make siblings clone every member BESIDE the holder, on its' + @echo ' tracking branch; fetch the ones already there' @echo 'make validate the family validator, then each member'\''s make validate' @echo 'make pins the family lockstep check alone (gitlink == pin)' bootstrap: $(PYTHON) scripts/bootstrap.py +siblings: + $(PYTHON) scripts/siblings.py + validate: $(PYTHON) scripts/validate-family.py $(PYTHON) scripts/bootstrap.py --no-fetch --members-make validate diff --git a/templates/family-root/README.md b/templates/family-root/README.md index 4e4d3667..c55324e2 100644 --- a/templates/family-root/README.md +++ b/templates/family-root/README.md @@ -19,9 +19,61 @@ pinned commits and runs its validators. | target | what it does | |---|---| | `make bootstrap` | fetch every member and its legs, then each member's `make bootstrap` | +| `make siblings` | clone every member BESIDE this holder, on its tracking branch; fetch the ones already there | | `make validate` | the family validator, then each member's `make validate` | | `make pins` | the family's own lockstep check alone: gitlink == `members[].pin.commit` | +## Workstation layout + +This holder lives inside a **plain folder named after the family** — not a +repository, not a submodule, just a directory — with the members' working +clones beside it: + +``` +{{FAMILY_NAME}}/ a PLAIN FOLDER, not a git repository + {{FAMILY_NAME}}/ THE HOLDER (this repository) + family.yaml the members and their pins + members/ each member, PINNED and DETACHED + / a WORKING clone, on its tracking branch + / …one per row in family.yaml + -worktrees/… whatever the member's own work needs + session-handoff-*.md the estate's own files live in the folder +``` + +`make siblings` is what places it: for each row in `family.yaml` it clones the +member at `../` from the same url `make bootstrap` uses, then runs +that member's own `scripts/bootstrap.py` so its legs are on their tracking +branches at their pins. A member that is already there is **fetched and +nothing else** — no checkout, no reset, no pull — because somebody is working +in it. A directory that is a clone of a DIFFERENT repository is reported and +skipped, never written into. It is idempotent: a second run is all `present`. + +**TWO COPIES OF EVERY MEMBER, on purpose.** `members/` inside this +holder is a pinned, detached checkout, and it is what `make bootstrap` and +`make validate` read — detached because a pin is a commit and not a branch. +The clone BESIDE the holder is **where you work**: its own tracking branch, +its own worktrees, its own pull requests. The family follows it afterwards +with `family.py bump`, which is the only thing that moves a pin. + +**The doubled `{{FAMILY_NAME}}/{{FAMILY_NAME}}` is deliberate** (ruled +2026-09-09). The holder is a repository and keeps its own name; the folder is +named after the family because that is what the folder holds. `family.yaml` +is the only thing that tells the two apart, and it is inside the holder. + +**Nothing ever moves a checkout.** When this holder's parent folder is not +named after the family, `make siblings` WARNS and prints the exact `mv` for a +human to run — it moves nothing itself. Relocating a directory out from under +a shell, an editor or an agent lane breaks linked worktrees, whose `.git` +files carry absolute paths, and this standard does not act by surprise. + +From a checkout of `{{SHAPE_REPOSITORY}}` the same utility runs against a +holder anywhere on disk, which is what to use before this repository is +cloned twice: + +```sh +python3 scripts/family.py siblings --family-root +``` + ## What a family is, and what it is not Ruled by **Brett Heap on 2026-09-04**: *"InkRouter is a set of microservices diff --git a/templates/family-root/scripts/siblings.py b/templates/family-root/scripts/siblings.py new file mode 100755 index 00000000..c80c820f --- /dev/null +++ b/templates/family-root/scripts/siblings.py @@ -0,0 +1,551 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Clone every member of this family BESIDE the holder, and check the layout. + + cd / + make siblings # or: python3 scripts/siblings.py + +THE WORKSTATION LAYOUT THIS PLACES, and the one the ruling asked for (Brett +Heap, 2026-09-04: *"like a holder FOLDER and some utilities for the family of +services"*, and the layout ruling of 2026-09-09): + + / a PLAIN FOLDER, not a git repository + / THE HOLDER: family.yaml, members/ (pinned) + / a WORKING clone, on its tracking branch + / …one per row in family.yaml + session-handoff-*.md the estate's own files live in the folder + +TWO COPIES OF EVERY MEMBER, ON PURPOSE. `members/` inside the holder +is a PINNED, DETACHED checkout: it is what `make bootstrap` and `make +validate` read, and it is detached because a pin is a commit and not a branch. +The clone this command puts BESIDE the holder is where a human works — its own +`origin`, its own tracking branch, its own worktrees — and the family follows +it afterwards with `family.py bump`. Neither copy is the other's mirror, and +this command never touches the pinned one. + +WHAT IT DOES, per member row in `family.yaml`: + + ABSENT `git clone --recurse-submodules` from the SAME url `make bootstrap` + resolves — `.gitmodules` first, `family.yaml`'s `repository:` as + the fallback — with the same credential, resolved the same way, and + then the member's OWN `scripts/bootstrap.py`, which puts its legs + on their tracking branches AT their pins. The result is a working + clone on its branch, not a detached one. + + PRESENT `git fetch`, AND NOTHING ELSE. No checkout, no reset, no pull, no + stash, no branch switch. Somebody is working in that clone — that + is what it is for — and a utility that "helpfully" moved their HEAD + would be the single most destructive thing in this standard. The + clone is first verified to BE that member (its `origin` names the + row's repository and its own `project.yaml` id is the row's `id`); + when it is not, that is reported and SKIPPED, never overwritten. + + THEN THE PARENT-FOLDER CHECK. When the holder's parent directory is not + named after the family, this WARNS and prints the exact `mv` — and + MOVES NOTHING. Relocating a checkout out from under somebody's + shell, editor and agent lanes breaks linked worktrees, whose `.git` + files carry absolute paths, and is the one act this standard + refuses everywhere else: done by surprise, with no hand on it. The + doctor pattern, not a mover. + +IDEMPOTENT. A second run is every member `present`, one `git fetch` each, and +exit 0. That is what makes it safe to put in a handoff document. + +IT CONFERS NOTHING, and neither does the layout. A folder is navigation. + +EXIT CODES + 0 every member is beside the holder; nothing to report + 1 a FINDING a human can act on: a sibling that is a DIFFERENT repository, + a clone that failed, a member's own bootstrap that went red + 2 a REFUSAL: the question could not be asked — no `family.yaml`, no + `git`. An unanswerable question is never an implicit pass. +""" + +from __future__ import annotations + +import argparse +import subprocess +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from repo_shape import ( # noqa: E402 + PYTHON, Refusal, find_repo_root, load_yaml, repo_basename, +) +# THE CREDENTIAL AND THE MEMBER ROWS ARE READ BY `bootstrap.py` ALREADY, and a +# second definition of either is how the two start disagreeing about which +# token they used or where a member is mounted. Both files are copies of +# openRepoShape that travel together in `contracts/shape-pin.yaml`, so the +# import is not a dependency this holder can lose. +from bootstrap import ( # noqa: E402 + REWRITE_TARGETS, credential, member_path, members_of, +) + +MANIFEST = "family.yaml" + + +# --------------------------------------------------------------------------- +# where a member comes from +# --------------------------------------------------------------------------- + + +def submodule_urls(root: Path) -> dict[str, str]: + """`{submodule path: url}` out of `.gitmodules`. + + THE SAME URL `make bootstrap` USES. `git submodule update` reads + `.gitmodules`, so a family whose members are mounted from an SSH remote, + a mirror or (in this suite) a bare repository on disk must produce + siblings from that same string — deriving `https://github.com/.git` + from the manifest instead would clone a different remote from the one this + holder is pinned against, silently, in exactly the estates that do not use + the default spelling. + """ + path = root / ".gitmodules" + if not path.is_file(): + return {} + proc = subprocess.run(["git", "config", "-f", str(path), "--list"], + capture_output=True, text=True, check=False) + if proc.returncode != 0: + return {} + paths: dict[str, str] = {} + urls: dict[str, str] = {} + for line in proc.stdout.splitlines(): + key, _, value = line.partition("=") + parts = key.split(".") + if len(parts) < 3 or parts[0] != "submodule": + continue + name, field = ".".join(parts[1:-1]), parts[-1] + if field == "path": + paths[name] = value + elif field == "url": + urls[name] = value + return {paths.get(name, name): url for name, url in urls.items()} + + +def resolve_relative(url: str, root: Path) -> str: + """A `.gitmodules` url spelled `../.git`, against THIS remote. + + Git reads a relative submodule url relative to the SUPERPROJECT'S REMOTE, + not to any directory — one path component dropped per `..` — so a clone + that took the string literally would fetch from wherever the process + happens to be standing. `family.py add` writes absolute urls and never + produces one of these; a hand-mounted member can, and the honest answer + is git's own rule rather than a guess. + """ + if not url.startswith(("./", "../")): + return url + base = (git_text(["remote", "get-url", "origin"], root) or "").rstrip("/") + if not base: + return url + for part in url.split("/"): + if part == "..": + base = base.rsplit("/", 1)[0] + elif part not in (".", ""): + base = f"{base}/{part}" + return base + + +def clone_url(root: Path, row: dict, urls: dict[str, str]) -> tuple[str, str]: + """`(url, where it came from)` for one member.""" + path = str(row.get("path") or "") + if path in urls: + return resolve_relative(urls[path], root), ".gitmodules" + repository = str(row.get("repository") or "") + return f"https://github.com/{repository}.git", f"{MANIFEST} repository:" + + +def same_repository(one: str, two: str) -> bool: + """Do two remote spellings name the SAME repository? + + A HUMAN'S OWN CLONE IS NOT REQUIRED TO SPELL THE REMOTE THE WAY + `.gitmodules` DOES: `git@github.com:Org/Repo.git`, + `https://github.com/Org/Repo` and `ssh://git@github.com/Org/Repo.git` are + one repository, and refusing to fetch somebody's existing clone over a + punctuation difference would send them to delete it. So the comparison is + on the normalised `host/owner/repo`, and on the trailing `owner/repo` + where the hosts are spelled differently (a mirror, an enterprise host). + """ + def normalise(url: str) -> str: + text = url.strip().replace("\\", "/").rstrip("/") + for prefix in ("git+ssh://", "ssh://", "https://", "http://", + "git://"): + if text.lower().startswith(prefix): + text = text[len(prefix):] + break + head = text.split("/", 1)[0] + if "@" in head: # git@github.com:Org/Repo.git + text = text.split("@", 1)[1] + head = text.split("/", 1)[0] + if ":" in head: # the scp-like spelling's colon + text = text.replace(":", "/", 1) + if text.lower().endswith(".git"): + text = text[:-4] + return text.lower().strip("/") + + left, right = normalise(one), normalise(two) + if left == right: + return True + tail = right.split("/") + return len(tail) >= 2 and left.endswith("/".join(tail[-2:])) + + +# --------------------------------------------------------------------------- +# git, with the credential the family already resolves +# --------------------------------------------------------------------------- + + +def git_prefix() -> tuple[list[str], str, str]: + """`(the -c rewrite args, the credential source, the token)`. + + Resolved by `bootstrap.py`'s own `credential()`, in the same order, and + used ONLY through a `url.<...>.insteadOf` rewrite for the duration of one + command: never written into `.git/config`, never persisted, never printed. + """ + source, token = credential() + args: list[str] = [] + if token: + for target in REWRITE_TARGETS: + args += ["-c", f"url.https://x-access-token:{token}@github.com/" + f".insteadOf={target}"] + return args, source, token + + +def scrub(text: str, token: str) -> str: + """git puts the url it tried into its error text, token and all.""" + return text.replace(token, "") if token else text + + +def git_text(args: list[str], cwd: Path) -> str | None: + proc = subprocess.run(["git", *args], cwd=str(cwd), capture_output=True, + text=True, check=False) + return proc.stdout.strip() if proc.returncode == 0 else None + + +def branch_of(path: Path) -> str: + """The checked-out branch, or `detached` — READ ONLY, like everything + this command does to a clone somebody else made.""" + name = git_text(["rev-parse", "--abbrev-ref", "HEAD"], path) + if not name: + return "?" + return "detached" if name == "HEAD" else name + + +# --------------------------------------------------------------------------- +# one member +# --------------------------------------------------------------------------- + + +class Sibling: + """One row of the layout table: what is beside the holder, and its state.""" + + def __init__(self, project: str, path: Path): + self.project = project + self.path = path + self.state = "?" + self.branch = "-" + self.finding: str | None = None + + def row(self, kind: str = "sibling") -> str: + return (f" {kind:<9} {self.project:<18} {self.state:<22} " + f"{self.branch}") + + +def project_id(path: Path) -> str | None: + manifest = path / "project.yaml" + if not manifest.is_file(): + return None + try: + data = load_yaml(manifest) + except Refusal: + return None + return str(data.get("id")) if isinstance(data, dict) else None + + +def run_member_bootstrap(path: Path, sibling: Sibling) -> None: + """The member's OWN bootstrap, which is what puts its legs on branches. + + Not reimplemented here for the same reason the family's `bootstrap.py` + does not reimplement it: each member is a whole project and knows how. + """ + script = path / "scripts" / "bootstrap.py" + if not script.is_file(): + print(f" [{sibling.project}] no scripts/bootstrap.py; cloned and " + "left as it is. A member without one is not misconfigured — it " + "simply has nothing for this command to call.") + return + print(f" --- {sibling.project}: its own bootstrap ---") + sys.stdout.flush() + proc = subprocess.run([sys.executable or PYTHON, str(script)], + cwd=str(path), check=False) + sys.stdout.flush() + if proc.returncode != 0: + sibling.finding = (f"{sibling.project}: its own bootstrap exited " + f"{proc.returncode}; the clone is there and its " + "legs may not be placed") + + +def place(root: Path, row: dict, urls: dict[str, str], prefix: list[str], + token: str, dry_run: bool) -> Sibling: + """Clone or fetch ONE member beside the holder. Never moves a HEAD.""" + project = str(row.get("project") or "?") + repository = str(row.get("repository") or "") + target = root.parent / project + sibling = Sibling(project, target) + url, source = clone_url(root, row, urls) + + if target == root: + # A MEMBER NAMED AFTER ITS OWN FAMILY. `/` is the + # holder, so this member's sibling would BE the holder — and this + # command would then be asked to verify the holder against a member's + # row. Named rather than attempted: the exit is a folder whose name + # is not the member's, and that is the human's choice to make. + sibling.state = "IS THE HOLDER" + sibling.branch = branch_of(root) + sibling.finding = ( + f"{project}: its sibling would be {target}, which is this holder " + "itself — the member is named after its own family. Nothing was " + "done. Rename the family FOLDER (the holder keeps its name) so " + f"the two are not the same path, then re-run.") + return sibling + + present = (target / ".git").exists() + if not present and target.exists() and any(target.iterdir()): + sibling.state = "NOT A CLONE" + sibling.finding = ( + f"{project}: {target} exists, is not empty and is not a git " + "clone, so this command will not write into it. Move it aside or " + "name the directory something else; nothing here overwrites a " + "directory somebody made.") + return sibling + + if not present: + if dry_run: + sibling.state = "would clone" + print(f" [{project}] would clone {url} ({source}) -> {target}") + return sibling + print(f" [{project}] cloning {url} ({source})") + sys.stdout.flush() + proc = subprocess.run( + ["git", *prefix, "clone", "-q", "--recurse-submodules", "--", + url, str(target)], + capture_output=True, text=True, check=False) + if proc.returncode != 0: + sibling.state = "CLONE FAILED" + sibling.finding = ( + f"{project}: git clone exited {proc.returncode}:\n" + + scrub((proc.stderr + proc.stdout).strip(), token)) + return sibling + sibling.state = "cloned" + run_member_bootstrap(target, sibling) + sibling.branch = branch_of(target) + return sibling + + # PRESENT. Verify it is this member, then FETCH ONLY. + origin = git_text(["remote", "get-url", "origin"], target) or "" + identity = project_id(target) + if not same_repository(origin, url) and not ( + repository and same_repository(origin, repository)): + sibling.state = "WRONG ORIGIN" + sibling.branch = branch_of(target) + sibling.finding = ( + f"{project}: {target} is a clone of {origin or '(no origin)'}, " + f"not of {repository or url}, so it is not this member and is " + "left exactly as it is. Rename it, or move it aside and re-run; " + "this command never fetches into a repository it cannot identify.") + return sibling + if identity is not None and str(row.get("id")) != identity: + sibling.state = "WRONG PROJECT" + sibling.branch = branch_of(target) + sibling.finding = ( + f"{project}: {target}/project.yaml declares id {identity!r} where " + f"the family's row records {str(row.get('id'))!r}. A repository at " + "the right url is not by itself the project the row claims, so " + "this is reported and skipped rather than fetched.") + return sibling + + if dry_run: + sibling.state = "present, would fetch" + sibling.branch = branch_of(target) + print(f" [{project}] present at {target}; would fetch, nothing else") + return sibling + print(f" [{project}] present at {target}; fetching only") + fetched = subprocess.run(["git", *prefix, "fetch", "--quiet", "origin"], + cwd=str(target), capture_output=True, text=True, + check=False) + # NO CHECKOUT, NO RESET, NO PULL: somebody is working in here. + sibling.state = "present, fetched" if fetched.returncode == 0 \ + else "present, fetch failed" + if fetched.returncode != 0: + print(f" [{project}] fetch exited {fetched.returncode}; the clone is " + "untouched:\n" + + scrub((fetched.stderr + fetched.stdout).strip(), token), + file=sys.stderr) + sibling.branch = branch_of(target) + return sibling + + +# --------------------------------------------------------------------------- +# the parent-folder check +# --------------------------------------------------------------------------- + + +def family_folder_name(manifest: dict, root: Path) -> str: + """What the family FOLDER is called. + + THE HOLDER'S REPOSITORY NAME, not `name:`. `family.py init` creates + `//` out of `--family `, which is the + repository's name; `name:` is a DISPLAY name and may carry a space. Where + a family never set one the two are the same string. + """ + repository = str(manifest.get("repository") or "") + if repository: + return repo_basename(repository) + return str(manifest.get("name") or root.name) + + +def parent_folder_warning(root: Path, family: str) -> str | None: + """The WARNING and the exact `mv`, or None when the layout is right. + + IT MOVES NOTHING, and this is the ruling (issue #76, 2026-09-09): a tool + that relocated the checkout would do it out from under the shell, the + editor and every agent lane standing in it, and would break linked + worktrees, whose `.git` files carry ABSOLUTE paths. The warning plus the + command gives the same outcome with the human's hand on it. + """ + folder = root.parent + if folder.name == family: + return None + lines = [ + f"WARNING the holder's parent folder is {folder.name!r}, not " + f"{family!r}.", + f" {root} should sit in a plain folder named {family}, beside " + "the members'", + " working clones — that is the layout `make siblings` places. " + "The siblings", + f" go beside the holder either way, so they are in {folder}.", + "", + " NOTHING WAS MOVED. Relocating a checkout breaks an open " + "shell, an editor", + " and any LINKED WORKTREE (its `.git` file carries an absolute " + "path), so it", + " is yours to run, when nothing is standing in it:", + "", + ] + wanted = folder.parent / family + if root.name == family and not wanted.exists(): + # The common case: the holder was cloned straight into ~/projects, so + # `/` IS the holder itself and the move needs a + # staging name — `mv A A/A` cannot be spelled in one command. + staged = folder.parent / f"{family}.holder" + lines += [f" mv {root} {staged}", + f" mkdir -p {wanted}", + f" mv {staged} {wanted / family}"] + else: + lines += [f" mkdir -p {wanted}", + f" mv {root} {wanted / family}"] + lines += ["", + " Then re-run `make siblings` from the moved holder; the " + "members are", + " cloned beside it, in the folder."] + return "\n".join(lines) + + +# --------------------------------------------------------------------------- + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("--root", type=Path, default=None, + help="the family holder (default: the enclosing " + "repository)") + parser.add_argument("--dry-run", action="store_true", + help="say what would be cloned and fetched; clone " + "nothing, fetch nothing, move nothing") + args = parser.parse_args(argv) + + try: + root = find_repo_root(args.root or Path(__file__).resolve().parents[1]) + # `members_of` is `bootstrap.py`'s reading of the rows and stays the + # only one; the manifest is then read again for the two fields the + # LAYOUT needs and bootstrap has no use for. + name, rows = members_of(root) + manifest = load_yaml(root / MANIFEST) + if not isinstance(manifest, dict): + raise Refusal("family-manifest-unreadable", + f"{root / MANIFEST}: not a mapping") + except Refusal as exc: + print(str(exc), file=sys.stderr) + return 2 + + family = family_folder_name(manifest, root) + print(f"siblings: {name} ({root}) — {len(rows)} member(s)") + print(f" family folder {root.parent}") + print(" each member is cloned BESIDE the holder and stays PINNED inside " + "it under") + print(" members/ — the pinned copy is detached for bootstrap and " + "validate, the") + print(" sibling is where you work.") + + prefix, source, token = git_prefix() + urls = submodule_urls(root) + print(f"\n(a) the members, beside the holder (credential source: " + f"{source})") + siblings: list[Sibling] = [] + if not rows: + print(" no members declared. A family with none is empty, not wrong: " + "`family.py add` is what puts one here.") + for row in rows: + siblings.append(place(root, row, urls, prefix, token, args.dry_run)) + + print("\n(b) the layout") + print(f" {'what':<9} {'name':<18} {'state':<22} branch") + holder = Sibling(root.name, root) + holder.state = "the holder" + holder.branch = branch_of(root) + print(holder.row("holder")) + for sibling in siblings: + print(sibling.row()) + # THE SECOND COPY, said out loud in the same table: the pinned mount is + # what `make bootstrap` and `make validate` read, and it is detached + # because a pin is a commit and not a branch. + for row in rows: + mounted = member_path(root, row) + # The row's OWN `path:` as the label, never a path computed back out + # of the mount: a hand-edited manifest can carry something that is not + # under this root, and a table is not the place to discover it. + pinned = Sibling(str(row.get("path") + or f"members/{row.get('project')}"), mounted) + if (mounted / ".git").exists(): + pinned.state = "pinned in the holder" + pinned.branch = branch_of(mounted) + else: + pinned.state = "not checked out" + pinned.branch = "make bootstrap" + print(pinned.row("pinned")) + + warning = parent_folder_warning(root, family) + if warning: + print() + sys.stdout.flush() # so the warning lands where it belongs + print(warning, file=sys.stderr) + sys.stderr.flush() + + findings = [s.finding for s in siblings if s.finding] + print() + sys.stdout.flush() + if findings: + for finding in findings: + print(f"FINDING {finding}", file=sys.stderr) + print(f"siblings: {len(findings)} finding(s); nothing was moved and " + "no existing clone was touched", file=sys.stderr) + return 1 + if args.dry_run: + print("--dry-run: nothing was cloned, fetched or moved.") + return 0 + print(f"siblings ok: {len(siblings)} member(s) beside the holder in " + f"{root.parent}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/test_family.py b/tests/test_family.py index ffb485f0..2bf8df49 100644 --- a/tests/test_family.py +++ b/tests/test_family.py @@ -21,6 +21,7 @@ from __future__ import annotations +import os import shutil import subprocess import sys @@ -154,7 +155,14 @@ def test_init_carries_the_shape_pin_over_its_own_copies(family): # statement about its own bytes — and the file is pinned rather than # merely shipped, because one that can be edited without the pin noticing # says nothing. + # + # 2026-09-09: `scripts/siblings.py` joined them (#76). The holder's + # WORKSTATION utility is a copy like the validator and the bootstrap, and + # it is PINNED for the same reason: it is the file `update-shape.py` then + # offers to an existing holder as `upstream-added`, and a copy nothing + # digests is a copy nobody can tell has been edited. assert rows == {"scripts/validate-family.py", "scripts/bootstrap.py", + "scripts/siblings.py", "Makefile", ".gitignore", ".gitattributes", ".github/workflows/validate.yml", "AGENTS-shape.md", "scripts/repo_shape.py", @@ -163,6 +171,35 @@ def test_init_carries_the_shape_pin_over_its_own_copies(family): "a family has no legs, so it does not carry the leg validator") +def test_init_copies_the_siblings_utility_and_the_makefile_target(family): + """THE HOLDER CARRIES THE UTILITY, not a second implementation of it. + + `make siblings` runs the copy; `family.py siblings` runs the very same + file out of the standard (ruling 1b, 2026-09-09: one implementation, two + entry points). So the copy must be byte-identical to the template, pinned + by digest, executable like the other two scripts, and named by the + Makefile the holder ships. + """ + sys.path.insert(0, str(REPO / "scripts")) + from repo_shape import file_sha256 + root = family["root"] + copied = root / "scripts" / "siblings.py" + template = (REPO / "templates" / "family-root" / "scripts" / + "siblings.py") + assert copied.read_bytes() == template.read_bytes() + rows = {row["path"]: row["sha256"].lower() + for row in load_yaml(root / "contracts" / "shape-pin.yaml")["files"]} + assert rows["scripts/siblings.py"] == file_sha256(copied) + if os.name != "nt": + assert copied.stat().st_mode & 0o111, ( + "`FAMILY_EXECUTABLE` names it, so the materializer chmods it") + makefile = (root / "Makefile").read_text() + assert "siblings:" in makefile + assert "$(PYTHON) scripts/siblings.py" in makefile + assert "make siblings" in (root / "README.md").read_text() + assert "make siblings" in (root / "AGENTS-shape.md").read_text() + + def test_init_writes_the_holders_agent_files(family): """The holder gets its OWN pinned rules — a family has no legs, no leg pins and no lockstep workflow refs, so half of the assembly root's file @@ -837,3 +874,391 @@ def test_pins_and_no_members_together_ask_for_nothing(holder): result = validate(holder, "--pins", "--no-members") assert result.returncode == 2 assert "family-nothing-to-check" in result.stderr + + +# --- init lands in a family FOLDER (#76, 2026-09-09) ------------------------ +# +# THE LAYOUT IS PLACED, NEVER MOVED. `init` used to materialize the holder in +# a TEMP directory unless `--work-dir` said otherwise, so every family folder +# on every workstation was arranged by hand afterwards — which is why the +# doubled `/` had to be explained to each person who met it. +# The default now lands where the person is standing, in a plain folder named +# after the family, and `--work-dir` keeps exactly its old meaning. + + +def init_local(tmp_path, *extra, family: str = "Contoso", cwd=None): + return run_script(FAMILY, "init", "--org", ORG, "--family", family, + "--created-by", "Test Human", "--created-on", + "2026-09-09", "--local-remote-dir", + str(tmp_path / "remotes"), *extra, cwd=cwd) + + +def test_init_lands_the_holder_in_a_family_folder_where_you_are_standing( + tmp_path): + """`//`, with `` the invocation directory. + + The same "lands where you were standing" rule #39 gave projects. The + folder is a PLAIN DIRECTORY — not a repository, not a submodule — and the + holder keeps its own repository name inside it. + """ + stand = tmp_path / "projects" + stand.mkdir() + result = init_local(tmp_path, cwd=stand) + assert result.returncode == 0, result.stderr + result.stdout + folder, holder = stand / "Contoso", stand / "Contoso" / "Contoso" + assert folder.is_dir() and not (folder / ".git").exists(), ( + "the family folder is a plain directory and never a repository") + assert (holder / "family.yaml").is_file() + assert (holder / ".git").is_dir() + assert f"folder {folder}" in result.stdout + assert f"landing {holder}" in result.stdout + # The next block names the folder, the holder and BOTH next commands, and + # says which of the two copies of a member you work in. + assert f"family folder {folder}" in result.stdout + assert f"holder {holder}" in result.stdout + assert "add --family-root" in result.stdout + assert "make siblings" in result.stdout + assert "TWO COPIES OF EVERY MEMBER" in result.stdout + assert "BESIDE the holder" in result.stdout + + +def test_init_into_names_the_parent_directory(tmp_path): + """`--into` is the PARENT, not the landing spot: the folder goes in it.""" + into = tmp_path / "elsewhere" + result = init_local(tmp_path, "--into", str(into)) + assert result.returncode == 0, result.stderr + result.stdout + assert (into / "Contoso" / "Contoso" / "family.yaml").is_file() + + +def test_init_standing_in_the_family_folder_does_not_nest_another(tmp_path): + """A person standing in `Contoso/` means THAT folder, not one inside it. + + Otherwise the first thing anybody does — `mkdir Contoso && cd Contoso` — + produces `Contoso/Contoso/Contoso`, and the layout that has to be + explained once has to be explained twice. + """ + stand = tmp_path / "Contoso" + stand.mkdir() + result = init_local(tmp_path, cwd=stand) + assert result.returncode == 0, result.stderr + result.stdout + assert (stand / "Contoso" / "family.yaml").is_file() + assert not (stand / "Contoso" / "Contoso").exists() + + +def test_init_refuses_when_that_folder_already_holds_its_holder(tmp_path): + """The second run in the same place is not an init of anything.""" + stand = tmp_path / "Contoso" + stand.mkdir() + assert init_local(tmp_path, cwd=stand).returncode == 0 + again = init_local(tmp_path, "--reuse-empty-repo", cwd=stand) + assert again.returncode == 2 + assert "family-holder-already-here" in again.stderr + assert "family.py add" in again.stderr + assert "make siblings" in again.stderr + assert "There is no --force" in again.stderr + + +def test_init_work_dir_keeps_its_old_meaning_and_makes_no_folder(tmp_path): + """`/`, exactly as before — every rehearsal in this + suite passes it, and a flag that quietly changed meaning is worse than a + second flag.""" + work = tmp_path / "work" + result = init_local(tmp_path, "--work-dir", str(work)) + assert result.returncode == 0, result.stderr + result.stdout + assert (work / "Contoso" / "family.yaml").is_file() + assert not (work / "Contoso" / "Contoso").exists() + assert "--work-dir: no family folder" in result.stdout + + +def test_init_refuses_into_and_work_dir_together(tmp_path): + result = init_local(tmp_path, "--into", str(tmp_path / "a"), + "--work-dir", str(tmp_path / "b")) + assert result.returncode == 2 + assert "family-two-landings" in result.stderr + assert not (tmp_path / "a").exists() and not (tmp_path / "b").exists() + + +def test_init_refuses_a_landing_that_exists_and_is_not_empty(tmp_path): + """As before, and with no --force: the holder target is somebody's + directory and nothing here writes into one.""" + stand = tmp_path / "projects" + (stand / "Contoso" / "Contoso").mkdir(parents=True) + (stand / "Contoso" / "Contoso" / "MINE.md").write_text("not yours\n") + result = init_local(tmp_path, cwd=stand) + assert result.returncode == 2 + assert "family-target-exists" in result.stderr + assert "There is no --force" in result.stderr + assert (stand / "Contoso" / "Contoso" / "MINE.md").is_file() + + +def test_init_dry_run_prints_the_landing_and_creates_nothing(tmp_path): + stand = tmp_path / "projects" + stand.mkdir() + result = init_local(tmp_path, "--dry-run", cwd=stand) + assert result.returncode == 0, result.stderr + assert f"landing {stand / 'Contoso' / 'Contoso'}" in result.stdout + assert "--dry-run: nothing was created." in result.stdout + assert not (stand / "Contoso").exists() + assert not (tmp_path / "remotes").exists() + + +# --- make siblings: the members, beside the holder --------------------------- + + +SIBLINGS = REPO / "templates" / "family-root" / "scripts" / "siblings.py" + + +@pytest.fixture +def estate(family, tmp_path): + """A CORRECT workstation layout: `/InkRouter/InkRouter` and nothing + beside it yet. Function-scoped, because these tests write clones.""" + folder = tmp_path / NAME + folder.mkdir() + holder = folder / NAME + shutil.copytree(family["root"], holder, symlinks=True) + return {"folder": folder, "holder": holder, "remotes": family["remotes"]} + + +def siblings(holder, *extra, env=None): + return run_script(holder / "scripts" / "siblings.py", *extra, cwd=holder, + env={**ALLOW_FILE_PROTOCOL, **(env or {})}) + + +def layout(stdout: str) -> list[str]: + """The layout table alone — the part with no absolute path in it, so two + runs in two directories can be compared line for line.""" + lines = stdout.splitlines() + start = lines.index("(b) the layout") + rows = [] + for line in lines[start + 1:]: + if not line.startswith(" "): + break + rows.append(" ".join(line.split())) + return rows + + +def test_siblings_clones_each_member_beside_the_holder_on_its_branch(estate): + """Through `make siblings`, which is the entry point a holder ships. + + The sibling is a WORKING clone: on its tracking branch, with its own + bootstrap already run, so its legs are on their branches at their pins. + """ + proc = subprocess.run(["make", "siblings"], cwd=str(estate["holder"]), + capture_output=True, text=True, check=False, + env={**os.environ, **ALLOW_FILE_PROTOCOL}) + assert proc.returncode == 0, proc.stderr + proc.stdout + for project in MEMBERS: + sibling = estate["folder"] / project + assert (sibling / ".git").is_dir(), f"{project} was not cloned" + assert git("rev-parse", "--abbrev-ref", "HEAD", + cwd=sibling).stdout.strip() == "main", ( + "a sibling is a working clone on its tracking branch, not a " + "detached checkout like the pinned copy") + assert (sibling / "spec" / ".git").exists(), ( + "the member's own bootstrap placed its legs") + assert git("rev-parse", "--abbrev-ref", "HEAD", + cwd=sibling / "spec").stdout.strip() == "main" + assert "siblings ok: 2 member(s)" in proc.stdout + assert layout(proc.stdout) == [ + "what name state branch", + f"holder {NAME} the holder main", + "sibling IRRS cloned main", + "sibling IRSS cloned main", + "pinned members/IRRS pinned in the holder main", + "pinned members/IRSS pinned in the holder main", + ] + + +def test_a_second_run_is_all_present_and_exits_zero(estate): + """IDEMPOTENT, which is what makes it safe to put in a handoff.""" + first = siblings(estate["holder"]) + assert first.returncode == 0, first.stderr + first.stdout + second = siblings(estate["holder"]) + assert second.returncode == 0, second.stderr + second.stdout + assert "cloned" not in " ".join(layout(second.stdout)) + assert second.stdout.count("present, fetched") == len(MEMBERS) + assert "fetching only" in second.stdout + + +def test_siblings_touches_nothing_in_a_clone_somebody_is_working_in(estate): + """FETCH ONLY: no checkout, no reset, no pull, no branch switch. + + A utility that "helpfully" moved somebody's HEAD would be the most + destructive thing in this standard, so the branch a person is on and the + file they have not committed are both still there afterwards. + """ + assert siblings(estate["holder"]).returncode == 0 + sibling = estate["folder"] / "IRRS" + git("checkout", "-q", "-b", "wip", cwd=sibling) + (sibling / "SCRATCH.md").write_text("uncommitted work\n") + was = git("rev-parse", "HEAD", cwd=sibling).stdout.strip() + + result = siblings(estate["holder"]) + assert result.returncode == 0, result.stderr + result.stdout + assert git("rev-parse", "--abbrev-ref", "HEAD", + cwd=sibling).stdout.strip() == "wip" + assert git("rev-parse", "HEAD", cwd=sibling).stdout.strip() == was + assert (sibling / "SCRATCH.md").read_text() == "uncommitted work\n" + assert "sibling IRRS present, fetched wip" in layout(result.stdout) + + +def test_siblings_reports_a_clone_of_another_repository_and_skips_it(estate): + """A directory at the right NAME is not by itself the member. + + Reported, skipped, left exactly as it is, and the run exits non-zero — the + exit is the human's hand, never an overwrite. + """ + wrong = estate["folder"] / "IRSS" + wrong.mkdir() + git("init", "-q", "-b", "main", ".", cwd=wrong) + git("remote", "add", "origin", "https://github.com/Somebody/Else.git", + cwd=wrong) + (wrong / "MINE.md").write_text("somebody else's work\n") + + result = siblings(estate["holder"]) + assert result.returncode == 1, result.stdout + result.stderr + assert "WRONG ORIGIN" in result.stdout + assert "FINDING IRSS:" in result.stderr + assert "Somebody/Else" in result.stderr + assert "no existing clone was touched" in result.stderr + assert (wrong / "MINE.md").read_text() == "somebody else's work\n" + assert not (wrong / "project.yaml").exists(), "nothing was written into it" + # The other member is still placed: one bad directory is not a reason to + # do nothing for the rest of the family. + assert (estate["folder"] / "IRRS" / ".git").is_dir() + + +def test_siblings_reports_a_directory_that_is_not_a_clone_at_all(estate): + result = siblings(estate["holder"]) + assert result.returncode == 0 + rmtree(estate["folder"] / "IRRS") + (estate["folder"] / "IRRS").mkdir() + (estate["folder"] / "IRRS" / "notes.md").write_text("mine\n") + again = siblings(estate["holder"]) + assert again.returncode == 1 + assert "NOT A CLONE" in again.stdout + assert (estate["folder"] / "IRRS" / "notes.md").read_text() == "mine\n" + + +def test_siblings_warns_about_the_parent_folder_and_moves_nothing(family, + tmp_path): + """THE DOCTOR PATTERN, NOT A MOVER (#76). The holder here sits under a + folder that is not named after the family, which is what a hand-arranged + workstation looks like — and the answer is a warning, the exact `mv`, and + nothing moved.""" + wrong = tmp_path / "somewhere-else" + wrong.mkdir() + holder = wrong / NAME + shutil.copytree(family["root"], holder, symlinks=True) + + result = siblings(holder, "--dry-run") + assert result.returncode == 0, result.stderr + result.stdout + assert "WARNING the holder's parent folder is 'somewhere-else', not " \ + "'InkRouter'." in result.stderr + assert "NOTHING WAS MOVED" in result.stderr + assert "LINKED WORKTREE" in result.stderr + # The exact commands, and the staging name the one-step `mv A A/A` cannot + # be spelled without. + assert f"mv {holder} {tmp_path / 'InkRouter.holder'}" in result.stderr + assert f"mkdir -p {tmp_path / 'InkRouter'}" in result.stderr + assert f"mv {tmp_path / 'InkRouter.holder'} " \ + f"{tmp_path / 'InkRouter' / 'InkRouter'}" in result.stderr + assert holder.is_dir() and (holder / "family.yaml").is_file(), ( + "it moves nothing, and that is the whole point") + assert not (tmp_path / "InkRouter").exists() + + +def test_siblings_dry_run_clones_nothing(estate): + result = siblings(estate["holder"], "--dry-run") + assert result.returncode == 0, result.stderr + result.stdout + assert "would clone" in result.stdout + assert "--dry-run: nothing was cloned, fetched or moved." in result.stdout + for project in MEMBERS: + assert not (estate["folder"] / project).exists() + + +def test_family_py_siblings_runs_the_holders_own_file(family, tmp_path): + """ONE IMPLEMENTATION, TWO ENTRY POINTS (ruling 1b, 2026-09-09). + + `family.py siblings` runs `templates/family-root/scripts/siblings.py` + through `runpy`, the way the root `bootstrap` shim runs the assembly + root's bootstrap — so a person with a checkout of the standard and a + holder on disk gets the same run, and its exit code, without a second + code path to keep in step. The proof is the layout table: byte for byte + the same as `make siblings`, which is the part of the output that carries + no absolute path. + """ + def one(name: str, runner) -> dict: + folder = tmp_path / name / NAME + folder.mkdir(parents=True) + holder = folder / NAME + shutil.copytree(family["root"], holder, symlinks=True) + result = runner(holder) + assert result.returncode == 0, result.stderr + result.stdout + return {"folder": folder, "result": result} + + theirs = one("target", lambda holder: siblings(holder)) + ours = one("shim", lambda holder: run_script( + FAMILY, "siblings", "--family-root", str(holder), + env=ALLOW_FILE_PROTOCOL)) + + assert layout(ours["result"].stdout) == layout(theirs["result"].stdout) + for project in MEMBERS: + assert (ours["folder"] / project / ".git").is_dir() + assert git("rev-parse", "--abbrev-ref", "HEAD", + cwd=ours["folder"] / project).stdout.strip() == "main" + + +def test_family_py_siblings_refuses_a_path_that_is_not_a_holder(project): + result = run_script(FAMILY, "siblings", "--family-root", str(project)) + assert result.returncode == 2 + assert "family-root-missing" in result.stderr + assert "family.py init" in result.stderr + + +def test_siblings_accepts_a_clone_whose_remote_is_spelled_differently(estate): + """`git@github.com:Org/Repo.git` and this family's mounted url are ONE + repository, and a utility that refused to fetch somebody's clone over a + punctuation difference would send them to delete it.""" + assert siblings(estate["holder"]).returncode == 0 + sibling = estate["folder"] / "IRRS" + git("remote", "set-url", "origin", + f"git@github.com:{ORG}/IRRS.git", cwd=sibling) + result = siblings(estate["holder"]) + # The fetch itself cannot succeed against a remote that is not there — + # that is a network fact, not an identity one — but the clone is + # RECOGNISED, left alone, and never reported as another repository. + assert result.returncode == 0, result.stderr + result.stdout + assert "WRONG ORIGIN" not in result.stdout + assert "present" in " ".join(layout(result.stdout)) + + +def test_siblings_skips_a_clone_whose_project_id_is_not_the_rows(estate): + """A repository at the right url is not by itself the project the row + claims: `project.yaml` inside the clone is the source.""" + assert siblings(estate["holder"]).returncode == 0 + sibling = estate["folder"] / "IRSS" + manifest_path = sibling / "project.yaml" + manifest_path.write_text( + manifest_path.read_text().replace("id: irss", "id: something-else", 1)) + result = siblings(estate["holder"]) + assert result.returncode == 1, result.stdout + result.stderr + assert "WRONG PROJECT" in result.stdout + assert "something-else" in result.stderr + + +def test_siblings_resolves_a_relative_submodule_url(estate): + """A `../.git` in `.gitmodules` is relative to THIS repository's + REMOTE, which is git's own rule — a clone that took the string literally + would fetch from wherever the process happened to be standing.""" + modules = estate["holder"] / ".gitmodules" + modules.write_text( + "\n".join(f'[submodule "members/{p}"]\n\tpath = members/{p}\n' + f"\turl = ../{p}.git" for p in MEMBERS) + "\n") + result = siblings(estate["holder"]) + assert result.returncode == 0, result.stderr + result.stdout + for project in MEMBERS: + sibling = estate["folder"] / project + assert (sibling / ".git").is_dir() + assert git("remote", "get-url", "origin", cwd=sibling).stdout.strip() \ + == str(estate["remotes"] / f"{project}.git") diff --git a/tests/test_repo_hygiene.py b/tests/test_repo_hygiene.py index c2b5e0d7..80b8d4fa 100644 --- a/tests/test_repo_hygiene.py +++ b/tests/test_repo_hygiene.py @@ -33,9 +33,16 @@ REPO / "templates" / "assembly-root" / "scripts" / "bootstrap.py", REPO / "templates" / "family-root" / "scripts" / "validate-family.py", REPO / "templates" / "family-root" / "scripts" / "bootstrap.py", + REPO / "templates" / "family-root" / "scripts" / "siblings.py", ] +#: `bootstrap` joins them on 2026-09-09 (#76): the holder's `siblings.py` +#: imports the credential resolution and the member rows from the +#: `bootstrap.py` beside it rather than defining either a second time, and the +#: two files travel together in `contracts/shape-pin.yaml`. It is a local +#: module in exactly the sense the other four are — a file this standard +#: ships, never a package anybody installs. LOCAL_MODULES = {"repo_shape", "shape_materialize", "path_classify", - "conftest"} + "conftest", "bootstrap"} #: The two bash scripts a person runs BEFORE they have a checkout: the front #: door itself, and the command that fetches it. Both are shipped executables @@ -256,7 +263,16 @@ def test_agents_md_is_short_enough_to_be_read(): a bigger act than the one already forbidden — so the rule is written down beside it rather than left to be inferred.""" lines = (REPO / "AGENTS.md").read_text().splitlines() - assert len(lines) <= 277, f"AGENTS.md is {len(lines)} lines; the cap is 277" + # 2026-09-09: 277 -> 303 — the family procedure gained the WORKSTATION + # layout (#76). Seven lines say where `init` LANDS, because the doubled + # `/` reads like a mistake until somebody explains it and + # an assistant that "fixed" it would move a checkout. The rest is `make + # siblings` / `family.py siblings` and, mostly, what it NEVER does: it + # moves nothing (and neither may the assistant, `mv` in hand), and it + # touches no existing clone beyond a fetch. A rule about not moving + # somebody's checkout is the one an assistant cannot infer from the + # others, so it is written down beside them. + assert len(lines) <= 303, f"AGENTS.md is {len(lines)} lines; the cap is 303" def test_claude_md_points_at_agents_md(): @@ -554,8 +570,22 @@ def test_readme_is_short_enough_to_be_read(): # `contracts/-pin.yaml`), and the shape pin to # `opensoft/openRepoShape`. Wrappers cost lines and no prose; a cap that # refused them would be a cap on legibility rather than on words. - assert len(lines) <= 1014, ( - f"README.md is {len(lines)} lines; the cap is 1014") + # 2026-09-09: 1014 -> 1059 — the family WORKSTATION layout (#76), ruled by + # Brett Heap the same day. The Families section described a holder and + # said nothing about the folder around it, so the first question anyone + # asked — "is `InkRouter/InkRouter` the family, and what is the parent + # folder?" — had no answer in this file, and the layout existed anyway, + # arranged by hand. Its own `###` subsection, not a paragraph, because + # three of its four halves are things a reader gets wrong separately: + # where `init` LANDS (`//`, and why the doubled + # name is kept), that there are TWO COPIES of every member on purpose + # and which one you work in, and that nothing ever MOVES a checkout — + # the warning-plus-`mv` instead, for the reason the deliberately-not- + # proposed mover would have broken (linked worktrees carry absolute + # paths). Six more lines extend the InkRouter tree to show the folder + # and name `family.py siblings` in the command block. + assert len(lines) <= 1059, ( + f"README.md is {len(lines)} lines; the cap is 1059") @pytest.mark.parametrize("name", SHIPPED_BASH) diff --git a/tests/test_update_shape_add.py b/tests/test_update_shape_add.py index 1a33af78..24b8bab5 100644 --- a/tests/test_update_shape_add.py +++ b/tests/test_update_shape_add.py @@ -19,13 +19,17 @@ named these files must report NO addition at all, because the lists have to be read at the TARGET commit rather than out of the checkout running the command. -C2 ALSO CARRIES THE REAL ADDITION, `.gitattributes` (2026-09-05, #51), and C1 -is made to lack it — the file is deleted out of the CLONE and its two copy-list -entries stripped, which is this change inverted. Three synthetic files prove -the machinery; one real one proves the machinery is pointed at the standard. -The strip is a no-op on a checkout that has not landed #51 yet, so the fixture -tells the same story on both sides of that commit, and nothing here writes to -the checkout it was cloned from. +C2 ALSO CARRIES THE REAL ADDITIONS — `.gitattributes` (2026-09-05, #51) for +both roots and `scripts/siblings.py` (2026-09-09, #76) for the FAMILY holder +alone — and C1 is made to lack them: each file is deleted out of the CLONE and +its copy-list entries stripped, which is those changes inverted. Three +synthetic files prove the machinery; the real ones prove the machinery is +pointed at the standard, and `siblings.py` proves it a second way — a holder +that already exists is owed the new utility and a project is offered nothing, +because only the family's list names it. Each strip is a no-op on a checkout +that has not landed its change yet, so the fixture tells the same story on +both sides of either commit, and nothing here writes to the checkout it was +cloned from. """ from __future__ import annotations @@ -57,11 +61,18 @@ #: offered a family's addition, nor a family a project's. FAMILY_ADDED = "FAMILY-shape-test.md" -#: THE REAL ADDITION (#51). Both roots gain it, out of the same list each -#: synthetic file uses, so every project and every holder cut before -#: 2026-09-05 is owed it — which is the case this module exists to cover and -#: the only one anybody will actually run. +#: THE REAL ADDITIONS, out of the same lists each synthetic file uses, so +#: every root cut before them is owed them — which is the case this module +#: exists to cover and the only one anybody will actually run. +#: +#: `.gitattributes` (#51, 2026-09-05) is in BOTH roots' lists. ATTRIBUTES = ".gitattributes" +#: `scripts/siblings.py` (#76, 2026-09-09) is in the FAMILY's alone: it is the +#: holder's workstation utility, and a project has no members to place. +SIBLINGS = "scripts/siblings.py" +#: `(path inside the root, the template roots whose lists name it)`. +REAL_ADDITIONS = ((ATTRIBUTES, ("assembly-root", "family-root")), + (SIBLINGS, ("family-root",))) MATERIALIZER = "scripts/shape_materialize.py" #: An existing row, for the "--add what is already pinned" refusal. @@ -121,26 +132,31 @@ def test_unname_everywhere_strips_a_crlf_entry_line(): ')\r\n') -def strip_the_real_addition(upstream) -> list: - """Undo #51 in the CLONE, so C1 is the standard as it was before it. +def strip_the_real_additions(upstream) -> list: + """Undo #51 and #76 in the CLONE, so C1 is the standard before them. Returns the paths it changed, empty when the checkout under test predates - #51 — in which case C1 already lacks the file and there is nothing to undo. + both — in which case C1 already lacks the files and there is nothing to + undo. The copy-list entries go together with the files: a list that names + a path the tree does not have reports nothing (`upstream.blob` is None), + so leaving one behind would make the fixture silently prove less. """ touched = [] - for template in ("assembly-root", "family-root"): - rel = f"templates/{template}/{ATTRIBUTES}" - if (upstream / rel).is_file(): - (upstream / rel).unlink() - touched.append(rel) source = upstream / MATERIALIZER text = source.read_text(encoding="utf-8") - stripped = unname_everywhere(text, ATTRIBUTES) - if stripped != text: - source.write_text(stripped, encoding="utf-8") + for entry, templates in REAL_ADDITIONS: + for template in templates: + rel = f"templates/{template}/{entry}" + if (upstream / rel).is_file(): + (upstream / rel).unlink() + touched.append(rel) + text = unname_everywhere(text, entry) + assert f'"{entry}"' not in text, ( + f"the copy lists still name {entry}, which this fixture just " + "removed") + if text != source.read_text(encoding="utf-8"): + source.write_text(text, encoding="utf-8") touched.append(MATERIALIZER) - assert f'"{ATTRIBUTES}"' not in stripped, ( - "the copy lists still name the file this fixture just removed") return touched @@ -153,9 +169,10 @@ def standard(tmp_path_factory) -> dict: capture_output=True, text=True, check=False) assert proc.returncode == 0, proc.stderr - # ---- C1: the standard as it stood before #51 -------------------------- - undone = strip_the_real_addition(upstream) - c1 = (commit_all(upstream, "The standard before .gitattributes", undone) + # ---- C1: the standard before #51 and #76 ------------------------------ + undone = strip_the_real_additions(upstream) + c1 = (commit_all(upstream, "The standard before .gitattributes and " + "siblings.py", undone) if undone else git("rev-parse", "HEAD", cwd=upstream).stdout.strip()) result = run_script( @@ -186,14 +203,16 @@ def standard(tmp_path_factory) -> dict: "#!/usr/bin/env python3\nprint('a new check')\n", encoding="utf-8") (upstream / "templates" / "family-root" / FAMILY_ADDED).write_text( "# The holder's copy of the same idea.\n", encoding="utf-8") - # The REAL file, byte for byte out of this checkout: a test that invented + # The REAL files, byte for byte out of this checkout: a test that invented # its own `.gitattributes` would prove the machinery and say nothing about # what a project actually receives. - attributes = [] - for template in ("assembly-root", "family-root"): - rel = f"templates/{template}/{ATTRIBUTES}" - (upstream / rel).write_bytes((REPO / rel).read_bytes()) - attributes.append(rel) + real = [] + for entry, templates in REAL_ADDITIONS: + for template in templates: + rel = f"templates/{template}/{entry}" + (upstream / rel).parent.mkdir(parents=True, exist_ok=True) + (upstream / rel).write_bytes((REPO / rel).read_bytes()) + real.append(rel) source = upstream / MATERIALIZER text = source.read_text(encoding="utf-8") text = name_in_list(text, "COPIED_VERBATIM", ADDED) @@ -202,13 +221,17 @@ def standard(tmp_path_factory) -> dict: text = name_in_list(text, "FAMILY_COPIED_VERBATIM", FAMILY_ADDED) text = name_in_list(text, "COPIED_VERBATIM", ATTRIBUTES) text = name_in_list(text, "FAMILY_COPIED_VERBATIM", ATTRIBUTES) + # The family's alone, and executable there like the two scripts beside it. + text = name_in_list(text, "FAMILY_COPIED_VERBATIM", SIBLINGS) + text = name_in_list(text, "FAMILY_EXECUTABLE", SIBLINGS) source.write_text(text, encoding="utf-8") - c2 = commit_all(upstream, "Add AGENTS-shape and .gitattributes to both roots", [ + c2 = commit_all(upstream, "Add AGENTS-shape, .gitattributes and the " + "holder's siblings utility", [ MATERIALIZER, f"templates/assembly-root/{ADDED}", f"templates/assembly-root/{ADDED_SCRIPT}", f"templates/family-root/{FAMILY_ADDED}", - *attributes, + *real, ]) assert c1 != c2 return {"upstream": upstream, "clone": clone, "family": base / "fam" / FAMILY, @@ -479,6 +502,50 @@ def test_every_existing_project_is_owed_the_attributes_file(root, standard): assert not (root / ATTRIBUTES).exists(), "`check` writes nothing" +def test_an_existing_holder_is_owed_the_siblings_utility(holder, standard): + """THE REAL ADDITION #76 MAKES, and the route it reaches a holder by. + + InkRouter/InkRouter exists today and was cut before `make siblings` + existed. `check` reports the file by name and exits 1; `apply --add + scripts/siblings.py` copies the standard's own bytes, writes the row, + chmods it because `FAMILY_EXECUTABLE` names it, and leaves the holder + green against its own validator. Nothing lands until a human names the + path, which is the whole of the `--add` rule. + """ + checked = run_script(UPDATE, "check", "--root", str(holder), "--upstream", + str(standard["upstream"]), "--at", standard["c2"]) + assert checked.returncode == 1, checked.stdout + checked.stderr + assert verdicts(checked.stdout)[SIBLINGS] == "upstream-added" + assert f"--add {SIBLINGS}" in checked.stdout + assert not (holder / SIBLINGS).exists(), "`check` writes nothing" + + result = run_script(UPDATE, "apply", "--root", str(holder), "--yes", + "--upstream", str(standard["upstream"]), + "--at", standard["c2"], "--add", SIBLINGS, + "--branch", "shape/add-siblings") + assert result.returncode == 0, result.stdout + result.stderr + copied = holder / SIBLINGS + assert copied.read_bytes() == ( + REPO / "templates" / "family-root" / SIBLINGS).read_bytes(), ( + "the holder receives the standard's file, not a paraphrase of it") + assert pin_rows(holder)[SIBLINGS] == file_sha256(copied) + if os.name != "nt": + assert copied.stat().st_mode & 0o111, ( + "`FAMILY_EXECUTABLE` names it, so `--add` chmods it too") + green = run_script(holder / "scripts" / "validate-family.py", cwd=holder) + assert green.returncode == 0, green.stdout + green.stderr + + +def test_a_project_is_not_offered_the_siblings_utility(root, standard): + """It is in the FAMILY's list alone. A project has no members to place, + and offering it one root's addition out of another root's list is how the + two tables would start disagreeing.""" + result = check(root, standard, "--at", standard["c2"]) + assert result.returncode == 1 + assert SIBLINGS not in result.stdout + assert not (root / SIBLINGS).exists() + + def test_a_family_holder_is_owed_it_as_well(holder, standard): """It is in BOTH lists, so it is owed to both kinds of root — a holder carries the same digest pin and is broken by CRLF the same way.""" From af116e461e9ec62d4f732f9681c58ef07716cdf9 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 9 Sep 2026 12:11:27 -0400 Subject: [PATCH 2/3] siblings.py: the url normaliser strips any scheme by pattern rather than a literal list (SonarCloud S5332 on #79) (#76) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SonarCloud failed #79's quality gate on one finding, python:S5332 ("Using HTTP protocol is insecure. Use HTTPS instead.") against the literal list of schemes `same_repository`'s normaliser stripped before comparing two remote urls. Nothing there fetches anything — the prefix is removed from BOTH sides so that `git@github.com:Org/Repo.git` and `https://github.com/Org/Repo` are one answer — so the finding is about the literal and not about a transport. It is still worth taking, and not by an exemption: a list of schemes has to be kept in step with git's transports, and this file was already carrying a list that named five of them and would have to grow for the sixth. One `re.sub` on `^[A-Za-z][A-Za-z0-9+.-]*://` replaces the list, which is more general as well as quieter: `ssh://`, `git://`, `git+ssh://`, `file://` and anything an estate's own remote helper registers all normalise the same way now, and `file:///srv/mirrors/Repo.git` and `/srv/mirrors/Repo.git` became one answer rather than two. The scp-like `host:owner/repo` handling, the `.git` suffix, the case folding and the trailing `owner/repo` comparison are untouched. The "spelled differently" test becomes six spellings of one repository — scp-like, https, ssh, git+ssh, `file://` over this family's own mounted bare repository, and that same path with a trailing slash — and asserts each is recognised as the member rather than reported as another repository. It now runs the second pass with `--dry-run`: `place` verifies the origin and the member's own `project.yaml` id BEFORE the dry run decides not to fetch, so the identity is what is under test and the run reaches for no network, which the previous single case did when it re-spelled the origin as a github.com url and then fetched it. 110 passed (tests/test_family.py, tests/test_repo_hygiene.py). Lane: xfactory-2 (xFactory-2) Co-Authored-By: Claude Fable 5.1 --- templates/family-root/scripts/siblings.py | 20 +++++++---- tests/test_family.py | 44 +++++++++++++++++------ 2 files changed, 48 insertions(+), 16 deletions(-) diff --git a/templates/family-root/scripts/siblings.py b/templates/family-root/scripts/siblings.py index c80c820f..0aa4edb8 100755 --- a/templates/family-root/scripts/siblings.py +++ b/templates/family-root/scripts/siblings.py @@ -64,6 +64,7 @@ from __future__ import annotations import argparse +import re import subprocess import sys from pathlib import Path @@ -83,6 +84,16 @@ MANIFEST = "family.yaml" +#: ANY `://` prefix, by PATTERN rather than by a list of the schemes +#: git happens to speak. Two reasons, in that order: a list has to be kept in +#: step with git's transports — `ssh`, `git`, `file`, `https`, `git+ssh` and +#: whatever an estate's own helper registers — and a list is also a list of +#: LITERALS, one of them the clear-text HTTP scheme, which a scanner reads as +#: a transport this file chose rather than as a string it strips (SonarCloud +#: python:S5332, on PR #79). Nothing here fetches anything at all: the prefix +#: is removed from both sides before two urls are compared. +SCHEME_RE = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*://") + # --------------------------------------------------------------------------- # where a member comes from @@ -164,14 +175,11 @@ def same_repository(one: str, two: str) -> bool: punctuation difference would send them to delete it. So the comparison is on the normalised `host/owner/repo`, and on the trailing `owner/repo` where the hosts are spelled differently (a mirror, an enterprise host). + Any scheme at all is dropped by `SCHEME_RE`, which is what makes + `file:///srv/mirrors/Repo.git` and `/srv/mirrors/Repo.git` one answer too. """ def normalise(url: str) -> str: - text = url.strip().replace("\\", "/").rstrip("/") - for prefix in ("git+ssh://", "ssh://", "https://", "http://", - "git://"): - if text.lower().startswith(prefix): - text = text[len(prefix):] - break + text = SCHEME_RE.sub("", url.strip().replace("\\", "/").rstrip("/")) head = text.split("/", 1)[0] if "@" in head: # git@github.com:Org/Repo.git text = text.split("@", 1)[1] diff --git a/tests/test_family.py b/tests/test_family.py index 2bf8df49..36c2b97d 100644 --- a/tests/test_family.py +++ b/tests/test_family.py @@ -1216,21 +1216,45 @@ def test_family_py_siblings_refuses_a_path_that_is_not_a_holder(project): assert "family.py init" in result.stderr -def test_siblings_accepts_a_clone_whose_remote_is_spelled_differently(estate): - """`git@github.com:Org/Repo.git` and this family's mounted url are ONE - repository, and a utility that refused to fetch somebody's clone over a - punctuation difference would send them to delete it.""" +#: SPELLINGS OF ONE REPOSITORY. `{remotes}` is this family's own mounted url, +#: which is a bare repository on disk here and a GitHub url in the world; +#: `{org}` reaches the same member through `family.yaml`'s `repository:`, the +#: way a person's own clone of a PRIVATE member is usually spelled. Every +#: scheme is dropped by pattern rather than by a list, which is why +#: `file://` and `` are one answer as well (#79's S5332: a list of +#: schemes is also a list of literals, and one of them read as a transport +#: choice). +SPELLINGS = ( + "git@github.com:{org}/IRRS.git", + "https://github.com/{org}/IRRS.git", + "ssh://git@github.com/{org}/IRRS.git", + "git+ssh://git@github.com/{org}/IRRS.git", + "file://{remotes}/IRRS.git", + "{remotes}/IRRS.git/", +) + + +@pytest.mark.parametrize("spelling", SPELLINGS) +def test_siblings_accepts_a_clone_whose_remote_is_spelled_differently( + estate, spelling): + """One repository, six spellings, and a utility that refused to fetch + somebody's clone over a punctuation difference would send them to delete + it. + + `--dry-run` for the second run, so the IDENTITY is what is under test and + nothing reaches for a network: `place` verifies the origin and the + `project.yaml` id BEFORE the dry run decides not to fetch, and this suite + creates nothing and contacts nothing. + """ assert siblings(estate["holder"]).returncode == 0 sibling = estate["folder"] / "IRRS" git("remote", "set-url", "origin", - f"git@github.com:{ORG}/IRRS.git", cwd=sibling) - result = siblings(estate["holder"]) - # The fetch itself cannot succeed against a remote that is not there — - # that is a network fact, not an identity one — but the clone is - # RECOGNISED, left alone, and never reported as another repository. + spelling.format(org=ORG, remotes=estate["remotes"]), cwd=sibling) + + result = siblings(estate["holder"], "--dry-run") assert result.returncode == 0, result.stderr + result.stdout assert "WRONG ORIGIN" not in result.stdout - assert "present" in " ".join(layout(result.stdout)) + assert "sibling IRRS present, would fetch main" in layout(result.stdout) def test_siblings_skips_a_clone_whose_project_id_is_not_the_rows(estate): From c33a8a98efaa2a5e0fba7c4974d1a6d429be28d5 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 9 Sep 2026 12:36:46 -0400 Subject: [PATCH 3/3] siblings.py: a relative submodule url resolves against a backslash-separated remote too (Windows, #79) (#76) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR #79's `windows-latest` job, at the merged head, on one test: FINDING IRRS: git clone exited 128: fatal: 'D:\a\_temp\t\family0\remotes\InkRouter.git/IRRS.git' does not appear to be a git repository A `.gitmodules` url spelled `../IRRS.git` is resolved against the SUPERPROJECT'S REMOTE, which is git's own rule and a TEXTUAL one: one trailing component dropped per `..`. The remote on that runner is a bare repository on disk, and `git remote get-url` hands it over as `D:\a\_temp\t\family0\remotes\InkRouter.git` — with no forward slash anywhere in it. The walk split on `/` alone, so the `..` consumed nothing, `InkRouter.git` stayed, and `IRRS.git` was appended to the whole string. On Linux the same test passes because the same path has slashes; the assumption was invisible on the platform it was written on, which is the same shape as the three cases `tests/test_windows_paths.py` already exists for. THE SEPARATOR IS NOW THE BASE'S OWN, decided by what the base IS. A FILESYSTEM PATH — no `://`, and any colon in its first segment is a DRIVE LETTER rather than git's `host:path` scp syntax, the same distinction `repo_shape.SAFE_PATH_RE` draws and for the same reason — is normalised to `/` for the walk and handed back in the spelling it arrived in. A URL keeps what it had: a url's separator is `/` on every platform, so an scp url's slashes must never be rewritten into backslashes on a Windows workstation. Backslashes rather than forward slashes on the way out because git on Windows accepts either and a human comparing the value against `git remote -v` or `.gitmodules` should not have to translate it — and `same_repository` folds both spellings to one answer, so identity does not depend on the choice. The arithmetic moved into `join_relative`, a PURE STRING function over two strings that touches no filesystem and asks nothing about this platform, which is what lets the Windows answer be asserted from Linux — the trick `family_landing` and `root_key` are already held to. `_one_level_up` carries the edges: a `..` too many consumes NOTHING, so a scheme (`https:/`) and an scp host (`git@host:`) survive it, while the parent of `/Repo.git` is the POSIX root and a Windows DRIVE is a component of its own (`D:/Repo.git` -> `D:` -> `D:/Other.git`). The drive is tested BEFORE the scheme, because `D:` is also what a one-letter scheme looks like. Eight spellings are asserted from Linux in `tests/test_windows_paths.py`, starting with the runner's own remote verbatim: the backslash path, a forward-slash Windows path, a UNC share, POSIX, `https://`, `file:///`, `ssh://` and the scp form, each resolving `../IRRS.git` beside the holder's own remote. Two more cover the path-or-url decision and the edges, and one covers the case that is every family this standard has actually made: an ABSOLUTE url goes through untouched, without asking git anything. The end-to-end `test_siblings_resolves_a_relative_submodule_url` is unchanged — it is the test that caught this, and it stays the one that runs the real clone. 148 passed (tests/test_family.py, tests/test_windows_paths.py, tests/test_repo_hygiene.py). Lane: xfactory-2 (xFactory-2) Co-Authored-By: Claude Fable 5.1 --- templates/family-root/scripts/siblings.py | 94 +++++++++++++-- tests/test_windows_paths.py | 137 ++++++++++++++++++++++ 2 files changed, 223 insertions(+), 8 deletions(-) diff --git a/templates/family-root/scripts/siblings.py b/templates/family-root/scripts/siblings.py index 0aa4edb8..5d05b935 100755 --- a/templates/family-root/scripts/siblings.py +++ b/templates/family-root/scripts/siblings.py @@ -93,6 +93,13 @@ #: python:S5332, on PR #79). Nothing here fetches anything at all: the prefix #: is removed from both sides before two urls are compared. SCHEME_RE = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*://") +#: The same scheme with NOTHING AFTER IT, which is what a `..` too many would +#: leave behind if it were allowed to keep trimming. +SCHEME_ONLY_RE = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*:/*$") +#: A Windows drive, and the only colon a FILESYSTEM PATH may carry — the +#: distinction `repo_shape.SAFE_PATH_RE` draws for the same reason: without +#: it, `D:\remotes\Repo.git` reads as git's `host:path` scp syntax. +DRIVE_RE = re.compile(r"^[A-Za-z]:$") # --------------------------------------------------------------------------- @@ -133,6 +140,81 @@ def submodule_urls(root: Path) -> dict[str, str]: return {paths.get(name, name): url for name, url in urls.items()} +def base_is_a_path(base: str) -> bool: + """Is this remote a FILESYSTEM PATH rather than a url? + + THE COLON IS A DRIVE LETTER'S OR IT IS SCP SYNTAX, which is the same + distinction `repo_shape.SAFE_PATH_RE` draws for a value on a git command + line: `D:\\a\\remotes\\InkRouter.git` is a path, `git@host:org/Repo.git` + and `host:path/Repo.git` are urls in git's scp spelling, and anything + with a `://` is a url outright. Only the path answer may be + walked with a backslash in it — a url's separator is always `/`, on + every platform. + """ + if SCHEME_RE.match(base): + return False + head = base.replace("\\", "/").split("/", 1)[0] + return ":" not in head or DRIVE_RE.match(head) is not None + + +def _one_level_up(flat: str) -> str | None: + """`flat` with its last component dropped, or None when there is none. + + A `..` TOO MANY CONSUMES NOTHING. What is left after the last component + can be a scheme (`https:/`), an scp host (`git@host:`) or nothing at all, + and none of those is a directory a `..` may eat — such a url is wrong + wherever it is read, and the clone that fails prints it. The one case + where nothing left IS an answer is a POSIX root: the parent of + `/Repo.git` is `/`, so it returns the empty string and the next name + appended makes `/Other.git`. A Windows DRIVE is a component of its own + for the same reason (`D:/Repo.git` -> `D:` -> `D:/Other.git`), and is the + one colon that is not scp syntax. + """ + if "/" not in flat: + return None + head = flat.rsplit("/", 1)[0] + # THE DRIVE IS ASKED FIRST, because `D:` is also what a one-letter scheme + # would look like and `SCHEME_ONLY_RE` cannot tell them apart. On Windows + # it is a component and the answer is yes. + if DRIVE_RE.match(head): + return head + if SCHEME_ONLY_RE.match(head) or head.endswith(":"): + return None + return head + + +def join_relative(base: str, url: str) -> str: + """`base` with `url`'s `../` applied — PURE STRING ARITHMETIC. + + NO FILESYSTEM AND NO PLATFORM, so `tests/test_windows_paths.py` can ask + it the Windows question from Linux the way it asks `family_landing` and + `root_key`. Git's rule for a relative submodule url is textual: one + trailing component dropped per `..`, appended per name. + + THE SEPARATOR IS THE BASE'S OWN. A local path on Windows arrives from + `git remote get-url` as `D:\\a\\_temp\\remotes\\InkRouter.git` — no + forward slash anywhere in it — so a walk that split on `/` alone dropped + nothing and appended `/IRRS.git` to the whole thing, which is the + `windows-latest` failure on PR #79 (`fatal: '…\\InkRouter.git/IRRS.git' + does not appear to be a git repository`). The walk therefore normalises + to `/`, and hands the result BACK in the spelling the remote used: git on + Windows accepts either, and a human comparing this against `git remote + -v` or `.gitmodules` should not have to translate it. `same_repository` + folds both spellings to one answer anyway, so identity does not depend + on the choice. + """ + native_backslash = base_is_a_path(base) and "\\" in base + flat = (base.replace("\\", "/") if native_backslash else base).rstrip("/") + for part in url.split("/"): + if part == "..": + up = _one_level_up(flat) + if up is not None: + flat = up + elif part not in (".", ""): + flat = f"{flat}/{part}" + return flat.replace("/", "\\") if native_backslash else flat + + def resolve_relative(url: str, root: Path) -> str: """A `.gitmodules` url spelled `../.git`, against THIS remote. @@ -141,19 +223,15 @@ def resolve_relative(url: str, root: Path) -> str: that took the string literally would fetch from wherever the process happens to be standing. `family.py add` writes absolute urls and never produces one of these; a hand-mounted member can, and the honest answer - is git's own rule rather than a guess. + is git's own rule rather than a guess. The arithmetic itself is + `join_relative`, which knows nothing about this machine. """ if not url.startswith(("./", "../")): return url - base = (git_text(["remote", "get-url", "origin"], root) or "").rstrip("/") + base = (git_text(["remote", "get-url", "origin"], root) or "").strip() if not base: return url - for part in url.split("/"): - if part == "..": - base = base.rsplit("/", 1)[0] - elif part not in (".", ""): - base = f"{base}/{part}" - return base + return join_relative(base, url) def clone_url(root: Path, row: dict, urls: dict[str, str]) -> tuple[str, str]: diff --git a/tests/test_windows_paths.py b/tests/test_windows_paths.py index 2fe3061f..896df44b 100644 --- a/tests/test_windows_paths.py +++ b/tests/test_windows_paths.py @@ -16,6 +16,14 @@ * a backslash eaten by the YAML reader's escape handling, which turned the `\\t` of a directory named `t` into a TAB and made a plan's own `source:` a path that does not exist. + +A FOURTH joined them on 2026-09-09 (PR #79's `windows-latest` run): a +relative submodule url resolved against the holder's remote by dropping one +`/`-separated component per `..`, where the remote was +`D:\\a\\_temp\\t\\family0\\remotes\\InkRouter.git` and had no `/` in it at all — so +nothing was dropped, the name was appended to the whole string, and `git +clone` refused `…\\InkRouter.git/IRRS.git`. Same shape as the three above: a +separator assumed, on the one platform that spells it differently. """ from __future__ import annotations @@ -57,6 +65,38 @@ def entry(): return module +@pytest.fixture(scope="module") +def siblings(): + """The HOLDER's `scripts/siblings.py`, for its url arithmetic. + + Loaded by path, under its own module name, with the family's + `bootstrap.py` pre-registered as `bootstrap` — that import is the one + `siblings.py` makes for the credential resolution and the member rows, and + in a materialized holder the two files sit side by side. `sys.path` and + `sys.modules` are put back afterwards, so nothing here leaves a module + named `bootstrap` importable for the rest of the session. + """ + scripts = REPO / "templates" / "family-root" / "scripts" + saved_path, saved_module = list(sys.path), sys.modules.get("bootstrap") + try: + spec = importlib.util.spec_from_file_location( + "family_root_bootstrap", scripts / "bootstrap.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + sys.modules["bootstrap"] = module + spec = importlib.util.spec_from_file_location( + "family_root_siblings", scripts / "siblings.py") + loaded = importlib.util.module_from_spec(spec) + spec.loader.exec_module(loaded) + yield loaded + finally: + sys.path[:] = saved_path + if saved_module is None: + sys.modules.pop("bootstrap", None) + else: + sys.modules["bootstrap"] = saved_module + + @pytest.fixture(scope="module") def adopt(): """`adopt-project.py` as a module, for its YAML writer.""" @@ -153,6 +193,103 @@ def test_no_family_leaves_the_landing_exactly_where_it_was(entry): assert entry.family_landing(parent, "") == parent +# --------------------------------------------------------------------------- +# A relative submodule url, against a remote spelled the Windows way +# --------------------------------------------------------------------------- + +#: The runner's own remote in the `windows-latest` run of PR #79, verbatim, +#: and the `..` that was resolved against it. `\` throughout and not one `/`: +#: that is the whole of the bug, so the fixture is the string rather than a +#: paraphrase of it. +WINDOWS_HOLDER_REMOTE = r"D:\a\_temp\t\family0\remotes\InkRouter.git" + + +@pytest.mark.parametrize("base,expected", [ + # THE FAILURE ITSELF. The result comes back in the spelling the remote + # used — git on Windows accepts either, and a human comparing this + # against `git remote -v` or `.gitmodules` should not have to translate. + (WINDOWS_HOLDER_REMOTE, r"D:\a\_temp\t\family0\remotes\IRRS.git"), + # A Windows path a person typed with forward slashes: git accepts it, so + # it comes back the way it went in. + ("D:/x/remotes/InkRouter.git", "D:/x/remotes/IRRS.git"), + # A UNC share, where the leading `\\` must survive the walk. + (r"\\server\share\remotes\InkRouter.git", + r"\\server\share\remotes\IRRS.git"), + # POSIX, which is what every green run had been asserting. + ("/srv/mirrors/InkRouter.git", "/srv/mirrors/IRRS.git"), + # A URL keeps git's own rule: one component of the URL dropped. + ("https://host/org/InkRouter.git", "https://host/org/IRRS.git"), + ("file:///srv/mirrors/InkRouter.git", "file:///srv/mirrors/IRRS.git"), + ("ssh://git@host/org/InkRouter.git", "ssh://git@host/org/IRRS.git"), + # The scp spelling, whose colon is NOT a drive letter: the separator is + # still `/` and the host is not a component a `..` may eat. + ("git@host:org/InkRouter.git", "git@host:org/IRRS.git"), +]) +def test_a_relative_member_url_resolves_against_any_remote_spelling( + siblings, base, expected): + r"""`../IRRS.git` against the holder's own remote, on every spelling. + + `join_relative` is PURE STRING ARITHMETIC over the two strings, which is + what lets the Windows answer be asserted from Linux — the same trick + `root_key` and `family_landing` above are held to. The end-to-end test in + `tests/test_family.py` is what caught this on the runner; this is what + keeps it caught between Windows runs. + """ + assert siblings.join_relative(base, "../IRRS.git") == expected + + +def test_a_windows_remote_is_a_path_and_an_scp_url_is_not(siblings): + r"""Only a FILESYSTEM PATH may be walked with a backslash in it. + + `D:\...` is a path whose colon is a drive letter; `git@host:org/Repo.git` + and `host:path/Repo.git` are urls in git's scp spelling, and a url's + separator is `/` on every platform. Getting this backwards would rewrite + an scp url's slashes into backslashes on a Windows workstation. + """ + assert siblings.base_is_a_path(WINDOWS_HOLDER_REMOTE) + assert siblings.base_is_a_path("D:/x/remotes/InkRouter.git") + assert siblings.base_is_a_path("/srv/mirrors/InkRouter.git") + assert not siblings.base_is_a_path("git@host:org/InkRouter.git") + assert not siblings.base_is_a_path("host:path/InkRouter.git") + assert not siblings.base_is_a_path("https://host/org/InkRouter.git") + assert not siblings.base_is_a_path("file:///srv/mirrors/InkRouter.git") + + +@pytest.mark.parametrize("base,url,expected", [ + # A drive is a component of its own, and is the one colon that is not scp + # syntax: `D:` is also what a one-letter SCHEME looks like. + (r"D:\InkRouter.git", "../IRRS.git", r"D:\IRRS.git"), + # The parent of a repository at the POSIX root is the root. + ("/InkRouter.git", "../IRRS.git", "/IRRS.git"), + # `..` TOO MANY CONSUMES NOTHING: not the scheme, not the scp host. + ("https://host/InkRouter.git", "../../IRRS.git", "https://host/IRRS.git"), + ("git@host:InkRouter.git", "../IRRS.git", + "git@host:InkRouter.git/IRRS.git"), + # Several levels, and a `./` that means this directory. + ("/srv/a/b/InkRouter.git", "../../mirrors/IRRS.git", + "/srv/a/mirrors/IRRS.git"), + (r"D:\a\remotes\InkRouter.git", "./sub/IRRS.git", + r"D:\a\remotes\InkRouter.git\sub\IRRS.git"), +]) +def test_the_walk_stops_where_there_is_nothing_left_to_consume( + siblings, base, url, expected): + """The edges of the same arithmetic, each one a shape a hand-mounted + member can carry. A url with a `..` too many is wrong wherever it is + read; what this must not do is invent `https:/` out of a scheme.""" + assert siblings.join_relative(base, url) == expected + + +def test_an_absolute_member_url_is_returned_untouched(siblings): + """Only `./` and `../` are relative. Everything `family.py add` writes — + every url in every family this standard has made — goes through + unchanged, and without asking git anything: the guard is the first line + of `resolve_relative`, which is why the root passed here does not exist. + """ + for url in (WINDOWS_HOLDER_REMOTE, "https://github.com/InkRouter/IRRS.git", + "git@github.com:InkRouter/IRRS.git", "/srv/mirrors/IRRS.git"): + assert siblings.resolve_relative(url, Path("/nonexistent")) == url + + # --------------------------------------------------------------------------- # The guard on a value that becomes a command argument # ---------------------------------------------------------------------------