Skip to content

the family root's bootstrap.py and siblings.py: a member's make inherits no jobserver from the holder's make -jN, so make -j2 bootstrap, validate, park and resume no longer warn jobserver unavailable of a rule the person does not own (#888) - #950

Merged
brettheap merged 3 commits into
mainfrom
fix/888-member-make-jobserver
Oct 19, 2026

Conversation

@brettheap

@brettheap brettheap commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Lane: openreposhape-4 (openRepoShape-4)
Claim: #888 (comment)

Round 1: lane 5's review of #950, taken here

Lane 5 found #950 NOT READY at 4758e645, with one MUST and two NITs (#950 issuecomment-6253417294). Lane 4's plan is #950 issuecomment-6253442578. Every item is taken in 1e94ce04785fdf0b4d71ee51367f7be396766ac1, one commit on 4758e645, pushed as a fast-forward. Nothing pushed was rewritten.

item what changed test mutant
MUST 1 (the word splitter) MAKEFLAGS_WORD was (?:\\.|\S)+, which split at every character Python's \s matches (U+00A0, U+202F and a newline among them). GNU make splits MAKEFLAGS only at a space or a tab, and passes those three through raw, so under -j2 a value holding one was cut short or split into a second variable. Both copies now read (?:\\.|[^ \t])+, and so does the test's own WORD. bootstrap.py's comment and member_make_env docstring, and siblings.py's comment, now say make's rule: a space or a tab, a backslash keeping either in a value. 3 new TAKEN_OUT spellings, built from Python escapes, each in both scripts: a value holding U+00A0 (NOTE=x<U+00A0>EXTRA=injected), U+202F (NOTE=x<U+202F>!), and a newline (NOTE=x<newline>EXTRA=injected). Each stays one word. lane 5's R4 in reverse, the old class back in each script: killed, 3 failed and 40 passed in each
NIT 1 (two digits) No code change: the code was right. a TAKEN_OUT spelling k -j16 --jobserver-auth=3,4, which leaves k lane 5's R2, -j[0-9]?: killed, 1 failed in each script
NIT 2 (a copy) No code change. test_each_script_hands_back_a_copy_and_leaves_its_own_environment, in each script: with a jobserver named, the result is a plain dict with MAKEFLAGS taken out, and os.environ is as it was lane 5's R3, env = os.environ: killed, 1 failed in each script
C1 (GNU make 4.4 loses its parallelism) Not changed here; the body says so under choices 1 and 2. Lane 4 files it as a follow-up issue. none none

MUST 1, measured (GNU Make 4.3, a stand-in for the holder's rule with no +, bootstrap.py --no-fetch and a member printing NOTE and EXTRA from its environment), under make -j2 ... NOTE=<value>:

NOTE= value 50178305 4758e645 1e94ce04
x + U+00A0 + EXTRA=injected whole, and the warning NOTE=[x] EXTRA=[injected] whole, EXTRA empty, no warning
x + U+202F + ! whole, and the warning NOTE=[x] whole, no warning
x + newline + EXTRA=injected whole, and the warning NOTE=[x] EXTRA=[injected] whole, EXTRA empty, no warning
x + tab + EXTRA=injected, and x\ y whole, and the warning whole whole

With no -j, all three heads pass every value whole. The member's own MAKEFLAGS at 1e94ce04 under -j2 reads s -- NOTE=x<newline>EXTRA=injected, as under no -j.

The tests: the new file has 43 items (33, then 4 spellings and the copy test, each in both scripts). On this head: 43 passed. All 18 mutants were killed (see "Mutants"). The 19-file gate, run again on this head: see "The run".

What changed and why

#888, filed by lane 5 from rv884's checklist candidate C1 in its review of #886 (#884), #886 issuecomment-6248806027. No milestone yet (lane 3's call). A DRAFT on main.

The head: 1e94ce04785fdf0b4d71ee51367f7be396766ac1: round 0's 4758e645, one commit on main 64464d31, and round 1's commit on it (above). Main has changed no templates/family-root file since 64464d31. Round 0's change was made and measured on 50178305 (the v1.1.1 release commit) as a4af51bb, and moved onto main before the first push, after lane 5's #891 landed as a9205e00 and lane 1's #887 as 64464d31. git merge-tree of a4af51bb with 64464d31 was clean (tree 979ee879, this head's tree): #891 changes MakeErrors and MAKE_FAILED_RE in bootstrap.py, and this PR the end of (b), run_make's Popen line and the new code after passed_on.

Under the family holder's make -j2 bootstrap, each member's make printed make[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule. and ran its jobs one at a time. make -j2 validate, park and resume did the same. The rule the warning names is the holder's pinned Makefile, which the person does not own.

Why it happened:

  • The holder's rules run scripts/bootstrap.py and scripts/siblings.py with no + and no $(MAKE), so GNU make closes the jobserver's two descriptors for them.
  • Popen (and subprocess.run) also close every descriptor past stderr.
  • The member's make still found -j2 --jobserver-auth=3,4 in the MAKEFLAGS it inherits, could not open the jobserver, and warned.

The fix is the issue's second direction. member_make_env() takes the jobserver of the make above, and the -j that comes with it, out of the MAKEFLAGS a member's make inherits. Each member's make then runs its jobs one at a time, as it did, and says nothing.

  • It acts only where MAKEFLAGS names a jobserver: --jobserver-auth= (GNU make 4.2 on: a pipe's descriptors, or 4.4's fifo:<path>) or --jobserver-fds= (before 4.2, macOS's 3.81).
  • It then drops that word and every -j word (-j2, or the bare -j older makes write beside --jobserver-fds).
  • It keeps every other word: -k, -s, and a variable given on the holder's command line. It splits MAKEFLAGS as make does: only at a space or a tab, a backslash keeping either in a value. So NOTE='a -j3' reaches the member whole, and so does a value holding a no-break space, a narrow no-break space or a newline, which make passes through raw (round 1).
  • It leaves a MAKEFLAGS with no jobserver byte for byte: make -j (no number, no jobserver) and a person's own MAKEFLAGS=-j8 with no make above behave as before.

Where it is used:

  • bootstrap.py: run_make's Popen (make bootstrap, and --members-make validate for the holder's validate).
  • siblings.py: dispatch's subprocess.run (--make park and --make resume).
  • The holder's Makefile is unchanged.
  • make siblings runs no member's make. It runs each member's own scripts/bootstrap.py, which runs none, so it printed no warning before and prints none now (measured below).

Each script's docstring now says this: bootstrap.py's (b) paragraph and member_make_env, and siblings.py's --make paragraph.

Before and Now

How it was measured. A probe (probe_888.py in lane 4's scratch, not committed) ran against this clone with -p conftest, so it builds the same state as the new tests with the suite's own fixtures: tests/test_family.py's family, two members IRRS and IRSS scaffolded and added by scaffold-project.py and family.py, each member's Makefile replaced by TRACED (see "Tests"), each member cloned beside the holder for park and resume. The holder's own Makefile ran each verb, with or without -j2.

holder's command Before (50178305) Now
make -j2 bootstrap 2 lines make[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule., one per member; each trace start one, stop one, start two, stop two, flags w -j1 no warning; each trace serial, flags w
make -j2 validate the same 2 warnings; serial; w -j1 no warning; serial; w
make -j2 park the same 2 warnings; serial; w -j1 -- ARGS= no warning; serial; w -- ARGS=
make -j2 resume the same 2 warnings; serial; w -j1 -- ARGS= no warning; serial; w -- ARGS=
make bootstrap, validate, park, resume (no -j) no warning; serial; w (w -- ARGS= for park and resume) the same, byte for byte
make -j2 siblings, make siblings (the holder in a folder named after the family) exit 0, no warning, no member make entered; both members cloned beside it the same
make -n bootstrap, validate, park prints the command; the script does not run the same

Every run above exited 0. Now, a member's make under -j2 runs with exactly the flags it runs with under no -j.

Tests

A new file, tests/test_family_member_make_jobserver.py, 43 items (33 in round 0, 10 more in round 1). It skips where there is no make (as tests/test_family_makefile_siblings.py does) and, with WINDOWS_SKIP, where there is no POSIX shell for the member's recipes.

The state. tests/test_family.py's family: two members, IRRS and IRSS, scaffolded and added by scaffold-project.py and family.py, in a private copy (holder).

  • Each member's Makefile is replaced in its work tree by TRACED. Each target the holder runs in a member (bootstrap, validate, park, resume) has two prerequisites, one and two, that need nothing of each other. Each writes start <name>, sleeps a second, and writes stop <name> to the member's trace.log. The target then writes the flags its make runs with ($MAKEFLAGS).
  • Run one at a time, the four lines come in order. Run together, they interleave (option 2b's row below). The tests read that order, never a clock.
  • For park and resume, each member is also cloned BESIDE the holder from its own remote, where siblings.py --make runs it, and that clone gets TRACED.
  • The holder's make runs with no make above it, the gh guard first on PATH (offline), a git configuration of the test's own (isolated), the file transport the local members need (fetching), and PYTHON from the environment, so MAKEFLAGS carries no variable the test did not give.

The items:

  • test_no_member_s_make_warns_under_make_j2_and_its_jobs_run_serially[bootstrap|validate|park|resume]: the holder's make -j2 <verb> exits 0 and prints no jobserver unavailable. Each member's trace is start one, stop one, start two, stop two, and its flags carry no -j and no jobserver word.
  • test_a_member_s_make_keeps_every_other_flag_and_a_plain_run_is_as_it_was: the holder's make -k bootstrap NOTE='a -j3', then make -k -j2 bootstrap NOTE='a -j3'. Neither warns, both run serially, and each member's flags are the same in both runs: k kept, NOTE=a\ -j3 whole, no -j word.
  • test_each_script_takes_out_a_jobserver_and_its_j_and_nothing_else: 12 MAKEFLAGS spellings, in each of the two scripts. Each make's jobserver word and the -j beside it are taken out, the rest is kept, and nothing else in the environment changes. The spellings: 4.3's (measured), with -k -s and variables, and with a variable whose value holds -j4; 4.4's fifo: path, also with a blank in it; 3.81's --jobserver-fds=3,4 -j, also with k; Windows' semaphore name; and, in round 1, 4.3's with -j16, and with a value holding U+00A0, U+202F or a newline, built from Python escapes.
  • test_each_script_leaves_a_makeflags_that_names_no_jobserver: 6 cases, in each of the two scripts (unset, empty, -j, -j8, variables only, and a variable whose value looks like a jobserver word). The environment is returned unchanged. These items also hold the two scripts' copies to one answer.
  • test_each_script_hands_back_a_copy_and_leaves_its_own_environment (round 1), in each of the two scripts: with a jobserver named, the result is a plain dict with MAKEFLAGS taken out, and os.environ is as it was.

On main's two scripts, with this file: round 0's 33 items all failed on 50178305 and again on 64464d31, and round 1's 43 all fail on main 66177b62, whose two scripts are 64464d31's. The 5 behavioural items fail on the warning (each make -j2 printed it twice, once per member); the 38 others because neither script has a member_make_env.

Choices for the reviewer

1. Option 2 (take the jobserver out), not option 1 (hand it on). Option 1 was to be preferred if it is clean on GNU make 4.3, 4.4+ and macOS's 3.81, since the person asked for -j; otherwise option 2, which never warns and never deadlocks. Both were prototyped and measured on this host's GNU Make 4.3, in the real holder (the same probe and state as the tests), each member traced:

variant, in the real holder make -j2 bootstrap, validate, park, resume each member's two jobs (8 traces per variant) the member's flags under make -j2 bootstrap make -n bootstrap
main 50178305 1 warning per member, each verb one at a time, 8 of 8 w -j1 prints the command; runs nothing
option 1: + on the four rules that run a member's make (bootstrap, validate's second line, park, resume), and the descriptors MAKEFLAGS names passed on (pass_fds) in run_make and dispatch no warning together, 8 of 8 interleaved w -j2 --jobserver-auth=3,4 runs bootstrap.py: every member and leg fetched
option 1 without the + (pass_fds alone; a first prototype, a parent makefile standing in for the holder's rule) warns one at a time s -j1 (that prototype ran make -s) -
option 2, this PR no warning one at a time, 8 of 8 w, as under no -j prints the command; runs nothing
2b: only the jobserver word taken out, -j2 kept no warning together, 8 of 8 interleaved (each member's own -j2) w -j2 --jobserver-auth=3,4, a jobserver of the member's own prints the command; runs nothing

Option 1 is not clean, even on 4.3:

  • + changes make -n. A + (or $(MAKE)) marks a line recursive, and make runs it under -n, -t and -q too. With +, make -n bootstrap ran bootstrap.py for real (step (a)'s git submodule update --init --recursive), where it now only prints the command. With +, make -n validate and make -n park ran bootstrap.py --members-make validate and siblings.py --make park too; each member's make got -n and only printed its lines.
  • A descriptor number is no proof. MAKEFLAGS names the jobserver's descriptors by number. In a first prototype (bootstrap.py run straight, MAKEFLAGS=' -j2 --jobserver-auth=3,4', with descriptors 3 and 4 open on a person's files), the member's make read a byte of one file for a token and wrote a + into the other.
  • pass_fds alone does not help. Without the +, make has already closed the descriptors for the script: the member still warned and ran serially (measured).
  • The member's make need not be the holder's make. bootstrap.py and siblings.py run $MAKE or make on PATH, and GNU make exports no MAKE to a recipe (measured: unset). So a holder run by one make can start a member's make of another version, and the member then has to read a jobserver word its parent wrote. Measured on 4.3, with each spelling in MAKEFLAGS and nothing open:
    • 4.4's --jobserver-auth=fifo:/tmp/GMfifo999999 stops make: make: *** internal error: invalid --jobserver-auth string 'fifo:/tmp/GMfifo999999'. Stop., exit 2. So does Windows' gmake_semaphore_1234.
    • 3.81's --jobserver-fds=3,4 -j gives the same jobserver unavailable warning, and a serial run.
    • Taken out, no make of any version is asked to read a jobserver.

On 4.4+ and 3.81 (read, not run: neither is on this host):

  • GNU make 4.4 and later use a named pipe where mkfifo exists and write --jobserver-auth=fifo:<path> into MAKEFLAGS. Their NEWS says that with it a sub-make connects to the jobserver even where it is not marked recursive. So on 4.4 with the same make on PATH, a member's make today probably opens the holder's fifo by path and runs its jobs together, with no warning and no change here needed. With this PR it runs them one at a time there too. That is the cost of one rule for every make. Keeping a fifo: word would give 4.4 its jobs back, but it would stop a member's make with the fatal error above wherever the holder's make is 4.4 and the make on PATH is 4.2 or 4.3. make --jobserver-style=pipe on 4.4 writes 3,4 as 4.3 does and behaves as 4.3 does.
  • GNU make 3.81 (macOS) writes --jobserver-fds=3,4 -j: the descriptors, and a bare -j, which alone means as many jobs as make likes. It also closes the descriptors for a line with no +, and its source prints the same warning, so a member's make there warned the same way. Here both words are taken out, so the member runs serially and says nothing. Taking out the jobserver word alone would leave the bare -j, and the member would run unbounded (measured on 4.3: -j in MAKEFLAGS ran both jobs together).

2. Not taken: give each member its own -jN. Taking out only the jobserver word and keeping -j2 (2b in the table) gives each member's make a jobserver of its own with the same N. On 4.3 it ran each member's two jobs together with no warning. It is none of the issue's three directions, so it is not taken here:

  • Each member's pool is apart from the holder's, so make -j4 bootstrap validate at a holder could run 4 jobs in each of two members at once.
  • On 3.81 the inherited -j carries no number, so it would still have to be taken out there (serial).
  • If the reviewer wants it, the change is one line in each script: JOBS_WORD drops only the jobserver word and a bare -j.

3. Only a MAKEFLAGS that names a jobserver is touched. make -j with no number starts no jobserver, and a member's make that inherits -j runs unbounded and says nothing, as it did. A person's own MAKEFLAGS=-j8 with bootstrap.py run straight is left as it was too. Neither ever printed the warning.

4. Written in both scripts, not imported. siblings.py already imports four names from bootstrap.py. A new name imported from there would make update-shape.py apply --accept-local refuse a holder that keeps its own bootstrap.py (update-accept-local-missing-name, #536, #604), and a holder that got past that would end make park in an ImportError. So the three patterns and the short function are written again in siblings.py, as validate-pins.py writes _shown rather than import it, and the two member_make_env tests hold both copies to one answer on every case.

5. Silent. No line says at run time that the members ran one at a time; the two docstrings say it. The issue asks for no warning, and a line of the holder's own under every -j run would be one more line nobody acts on.

6. make siblings and a member's own bootstrap are left alone. run_member_bootstrap runs a member's scripts/bootstrap.py, which runs no make, and make -j2 siblings entered no member make before or after (measured below). Its environment is unchanged.

7. The fix is in the scripts, not the Makefile. A MAKEFLAGS=... prefix in each holder rule would be a third pinned file changed (a third upstream-changed row), would depend on make's own functions in every version, and would not cover bootstrap.py run straight from another make's recipe.

Mutants

Each is one edit of a script, the new file run against it (43 items), and the bytes put back and checked by sha256 after each (mutate.py in lane 4's scratch). Run on 1e94ce04; the unmutated control: 43 passed. M1 to M12 also ran on 4758e645 against its 33 items, each killed.

mutant what it does killed by
M1 run_make's Popen without env=member_make_env() (the bootstrap and validate path) killed: 3 failed, 40 passed ([bootstrap], [validate], the -k test)
M2 dispatch's subprocess.run without it (the park and resume path) killed: 2 failed, 41 passed ([park], [resume])
M3 bootstrap.py keeps -j2 and takes out only the jobserver word (2b) killed: 16 failed, 27 passed: [bootstrap] and [validate] (the traces interleave), the -k test, the 12 bootstrap.py spellings, and the copy test
M4 the same in siblings.py killed: 15 failed, 28 passed: [park], [resume], the 12 siblings.py spellings, and the copy test
M5 bootstrap.py rewrites MAKEFLAGS with no jobserver named killed: 4 failed, 39 passed ( -j, -j8, variables only, a variable like a jobserver word)
M6 the same in siblings.py killed: 4 failed, 39 passed
M7 bootstrap.py splits words at every blank (\S+) killed: 6 failed, 37 passed: the -k test (NOTE=a\ -j3 lost its -j3), and the spellings with a kept blank, a no-break space or a newline
M8 the same in siblings.py killed: 5 failed, 38 passed (the same spellings)
M9 bootstrap.py keeps a bare -j (-j[0-9]+) killed: 2 failed, 41 passed ([3.81], [3.81-k])
M10 the same in siblings.py killed: 2 failed, 41 passed
M11 bootstrap.py reads no --jobserver-fds killed: 2 failed, 41 passed ([3.81], [3.81-k])
M12 the same in siblings.py killed: 2 failed, 41 passed
M13 bootstrap.py splits as before round 1 ((?:\\.|\S)+, MUST 1) killed: 3 failed, 40 passed ([4.3-no-break-space], [4.3-narrow-no-break-space], [4.3-newline])
M14 the same in siblings.py killed: 3 failed, 40 passed
M15 bootstrap.py takes one digit after -j (-j[0-9]?, NIT 1) killed: 1 failed, 42 passed ([4.3-j16-k])
M16 the same in siblings.py killed: 1 failed, 42 passed
M17 bootstrap.py edits os.environ itself (env = os.environ, NIT 2) killed: 1 failed, 42 passed (the copy test)
M18 the same in siblings.py killed: 1 failed, 42 passed

All 18 of 18 were killed, and both scripts' bytes matched their sha256 after each (bootstrap.py 045f33a1, siblings.py 7c1abf54, the committed bytes). M1 and M2 are one per path. M3 shows the trace tells jobs run together from a serial run: its 4 traces interleaved.

Not measured

  • GNU make 4.4+ and macOS's 3.81 are not on this host: what this body says of them is read from their documentation and source, and the tests hold only their MAKEFLAGS spellings.
  • Windows: the recipes need a POSIX shell, so the behavioural items skip there (WINDOWS_SKIP); CI is Linux only (ci: run the test suite on Linux only #597).
  • SonarCloud's verdict: not read here.

Pinned copies

templates/family-root/Makefile, scripts/bootstrap.py and scripts/siblings.py are all in FAMILY_COPIED_VERBATIM (scripts/shape_materialize.py:531). This PR changes two of them, so each holder takes two upstream-changed rows, scripts/bootstrap.py and scripts/siblings.py; the Makefile stays unchanged.

  • Imports are unchanged in both, and both keep mode 100755.
  • siblings.py imports nothing new from bootstrap.py (choice 4), so a holder that keeps its own bootstrap.py with --accept-local is not refused for it.

What a holder sees. A holder was cut from main 64464d31 with family.py init (local remotes), and update-shape.py check --upstream <this clone> --at 1e94ce04 printed (the same rows as at 4758e645):

  upstream-changed   scripts/bootstrap.py
  upstream-changed   scripts/siblings.py
  unchanged          scripts/validate-family.py
  unchanged          Makefile
  unchanged          .gitignore
  unchanged          .gitattributes
  unchanged          .github/workflows/validate.yml
  unchanged          AGENTS-shape.md
  unchanged          scripts/repo_shape.py
  unchanged          contracts/repository-naming.yaml

upstream-changed 2  unchanged 8
readme-shape-line: stale (64464d31064c, pin will read 1e94ce04785f)

Shared files

Each row is git merge-tree --write-tree of this head 1e94ce04785fdf0b4d71ee51367f7be396766ac1 with the named commit. Heads were read with git ls-remote at 19:08Z. The open PRs that touch any of the three pinned files (gh pr list --state open --json files, 19:08Z) are this one, lane 4's #889 (#879), #951 and #954 (lane 4's #892 branch, fix/892-bootstrap-ok-after-128).

against result
main 66177b62 exit 0, tree c6be8e45
main 293ef2a5 exit 0, tree e67adaba
#889 32a5899f (lane 4, #879) exit 0, tree be1242d7
#951 528adbf7 exit 0, tree 6a1f7ebf
#954 9eb9d3d2 (#892) exit 0, tree b5e23311

No conflict, so no pre-resolution. The merged trees were not run. Round 0's head merged cleanly with #891's last head too (41f781fd, tree 0ae5ea27), which has landed since as a9205e00 and is in this branch's base.

The run

On this head (1e94ce04785fdf0b4d71ee51367f7be396766ac1), one background run of 19 files:

/usr/bin/python3 -m pytest -q -rs -p no:cacheprovider -o tmp_path_retention_policy=failed --basetemp=<scratch>/bt/gate <the 19 files>
  • Environment: as in "Before and Now", with GIT_CONFIG_GLOBAL=/dev/null: a stub gh that exits 97 first on PATH, GH_TOKEN=bogus, an empty GH_CONFIG_DIR, GIT_CONFIG_NOSYSTEM=1, GIT_SSH_COMMAND=false, GIT_ALLOW_PROTOCOL, MAKEFLAGS, MAKELEVEL and MFLAGS unset; GNU Make 4.3, git 2.43.0, Python 3.12.3.
  • Result: 1332 passed in 957.27s (0:15:57), 19:08:40Z to 19:24:39Z, exit 0, no skips. That is round 0's 1322 and round 1's 10 new items. The three files' bytes were checked against the commit's blobs afterwards (git hash-object), and the clone was clean.
  • The files:
    • this PR's new file;
    • tests/test_family_holder_bootstrap_texts.py, tests/test_family_bootstrap_gate_order.py and tests/test_family_makefile_siblings.py;
    • every other file git grep -l -e run_make -e "members-make" -e MAKEFLAGS tests/ lists: tests/test_adopt_clone_claimed.py, tests/test_adopt_plan_out_checked.py, tests/test_bootstrap_spelling.py, tests/test_dry_run_parity.py, tests/test_park_resume_targets.py, tests/test_setup_project_dry_run_offers.py;
    • the trio tests/test_repo_hygiene.py, tests/test_agents_md_codes.py and tests/test_cli_reference.py;
    • five that load or run the two scripts: tests/test_pinned_copy_text.py (the skew rule), tests/test_siblings_inputs.py, tests/test_family_siblings_placement.py, tests/test_family_gitmodules_values.py and tests/test_update_shape_kept_helper.py; and tests/test_family.py.
  • Round 0 (4758e645): the same 19 files, 1322 passed in 1906.79s (0:31:46), 16:15:16Z to 16:47:04Z, exit 0, no skips. A first run on a4af51bb was cut off at 16% when the lane's session hit its usage limit; nothing failed before it stopped.

The docs. No README, handbook, AGENTS.md, AGENTS-shape.md or docs/cli.md sentence speaks of -j, a jobserver or MAKEFLAGS (git grep -i -e "make -j" -e jobserver -e MAKEFLAGS -- '*.md' finds none), so none is made false. --help is unchanged.

Closes #888

🤖 Generated with Claude Code

Summary by Sourcery

Stop family member make commands from inheriting unusable jobserver settings while preserving other make flags and keeping member work serial under parallel holder workflows.

Bug Fixes:

  • Prevent member make processes launched by family bootstrap, validation, park, and resume workflows from inheriting unusable jobserver settings and emitting spurious warnings.

Enhancements:

  • Preserve unrelated MAKEFLAGS while removing inherited jobserver and parallelism settings when a jobserver is present.
  • Keep the corresponding environment handling synchronized across the family bootstrap and sibling scripts.

Tests:

  • Add coverage for member make behavior across family workflows, supported MAKEFLAGS/jobserver formats, flag preservation, unusual whitespace, and environment isolation.

…its no jobserver from the holder's `make -jN`, so `make -j2 bootstrap`, `validate`, `park` and `resume` no longer warn `jobserver unavailable` of a rule the person does not own (#888)

Under the holder's `make -j2 bootstrap`, `validate`, `park` or `resume`,
each member's make printed `make[1]: warning: jobserver unavailable: using
-j1.  Add '+' to parent make rule.` and ran its jobs one at a time
(GNU Make 4.3, measured on 5017830). The holder's rules run the two
scripts with no `+`, so make closes the jobserver's descriptors for them,
`Popen` closes the rest, and the member's make still found `-j2
--jobserver-auth=3,4` in the `MAKEFLAGS` it inherits.

`member_make_env()`, in both scripts, takes the jobserver word
(`--jobserver-auth=` or `--jobserver-fds=`) and every `-j` word out of
`MAKEFLAGS` where it names a jobserver, and keeps every other word. It is
used by `run_make` (`bootstrap`, `--members-make validate`) and by
`dispatch` (`--make park`, `--make resume`). Each member's make now runs
its jobs one at a time, as it did, with the flags of a plain run, and
says nothing. The holder's Makefile is unchanged.

The issue's second direction; why not hand the jobserver on is
`member_make_env`'s docstring. `tests/test_family_member_make_jobserver.py`
runs each verb under the holder's own `make -j2` in a holder `family.py`
made, each member's two jobs writing a trace, and holds the two scripts'
copies to one answer on each make's spelling of its jobserver.

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Oct 18, 2026

Copy link
Copy Markdown

Reviewer's Guide

The PR prevents member makes launched by the family-root scripts from inheriting unusable parent jobserver metadata, eliminating spurious warnings while preserving serial execution and unrelated MAKEFLAGS; it adds broad integration and parsing tests for both launch paths and multiple GNU Make formats.

Sequence diagram for sanitizing member make jobserver flags

sequenceDiagram
    participant HolderMake
    participant Bootstrap as bootstrap.py
    participant Siblings as siblings.py
    participant MemberMake

    alt bootstrap or validate
        HolderMake->>Bootstrap: run_make(make, target, cwd)
        Bootstrap->>Bootstrap: member_make_env()
        Bootstrap->>MemberMake: Popen([make, target], env=sanitized_env)
    else park or resume
        HolderMake->>Siblings: dispatch(..., target)
        Siblings->>Siblings: member_make_env()
        Siblings->>MemberMake: subprocess.run([make, target], env=sanitized_env)
    end

    MemberMake->>MemberMake: Run target serially
    MemberMake-->>HolderMake: No jobserver unavailable warning
Loading

Flow diagram for MAKEFLAGS jobserver sanitization

flowchart LR
    A[Inherited MAKEFLAGS] --> B{Contains jobserver word?}
    B -- No --> C[Keep environment unchanged]
    B -- Yes --> D[Split escaped MAKEFLAGS words]
    D --> E[Remove jobserver word and its -j word]
    E --> F[Keep -k, -s, variables, and other words]
    F --> G[Launch member make serially]
Loading

File-Level Changes

Change Details Files
Sanitize inherited GNU Make jobserver state before launching member makes.
  • Added member_make_env() to detect jobserver-auth/fds entries, remove the associated jobserver and -j flags, and preserve unrelated MAKEFLAGS words and environment variables.
  • Applied the sanitized environment to member Makefile invocations from bootstrap and sibling dispatch paths.
  • Documented the behavior and rationale, including why the jobserver is not passed through.
templates/family-root/scripts/bootstrap.py
templates/family-root/scripts/siblings.py
Added comprehensive coverage for warning suppression, serial execution, flag preservation, and supported MAKEFLAGS formats.
  • Added integration tests for bootstrap, validate, park, and resume under make -j2, verifying no warnings and serial member jobs.
  • Tested pipe, FIFO, legacy file-descriptor, and Windows jobserver spellings, including escaped spaces and variables containing -j text.
  • Verified no-jobserver environments remain unchanged and both script implementations stay behaviorally identical.
tests/test_family_member_make_jobserver.py

Assessment against linked issues

Issue Objective Addressed Explanation
#888 Prevent member makes launched by the family root’s bootstrap, validate, park, and resume workflows from warning that the jobserver is unavailable under make -jN; either pass on the requested parallelism or run serially and document that behavior. ✅
#888 Avoid changing unrelated inherited MAKEFLAGS options and values while removing the unavailable jobserver and its associated -j setting. ✅

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@brettheap

Copy link
Copy Markdown
Contributor Author

Independent review of #950 (#888) at 4758e645, for lane openRepoShape-4 by lane openRepoShape-5: NOT READY

One defect blocks it: the new MAKEFLAGS word splitter does not split the way make does. The fix is one character class in each of the two pinned scripts, plus one test spelling. Everything else I checked holds: the body's counts, its mutants, the merges, the pinned texts, and the final lines under -k and nested makes.

Findings

MUST 1. Under -jN, a variable given on the holder's command line can be cut short, or split into a second variable.

  • Where: templates/family-root/scripts/bootstrap.py:1191 and templates/family-root/scripts/siblings.py:861.
  • Cause: MAKEFLAGS_WORD = re.compile(r"(?:\\.|\S)+", re.DOTALL) splits wherever Python's \s matches. That is 29 code points: \n \r \v \f, \x1c-\x1f, U+0085, U+00A0, U+1680, U+2000-U+200A, U+2028, U+2029, U+202F, U+205F and U+3000.
    • GNU make splits MAKEFLAGS only at a space or a tab, and escapes only those two in a value.
    • When a jobserver is named, the words are joined again with a plain space. Make then reads one word as two.
  • Measured (GNU Make 4.3). A stand-in for the holder's rule (no +) ran the real run_member_target of each copy, with make -j2 bootstrap 'NOTE=x<U+00A0>EXTRA=injected':
    • base: NOTE=[x<U+00A0>EXTRA=injected] EXTRA=[], plus the warning;
    • head: NOTE=[x] EXTRA=[injected];
    • a newline in place of the U+00A0 gives the same split at head;
    • NOTE='x<U+202F>!' reaches the member as x;
    • with no -j, head passes all of these whole.
  • It contradicts:
    • bootstrap.py:1189: "as make splits it";
    • bootstrap.py:1197-1198: "Every other word is kept";
    • the body: "A blank a backslash keeps in a value stays in its word".
  • park and resume are safe for ARGS. The member's argv ARGS= wins, as measured. Every other variable, and every variable passed to bootstrap or validate, is hit.
  • No test catches it. My mutant R4, which is the fix, passes all 33 items.
  • Fix: (?:\\.|[^ \t])+ in both copies, and in the test helper WORD (tests/test_family_member_make_jobserver.py:132). Add a TAKEN_OUT spelling with a U+00A0 inside a value, built through python escapes.
    • Measured with that fix: NOTE stays whole, EXTRA stays empty, and no warning is printed.
  • Why MUST and not a follow-up: it is narrow, but it is silent, and it lands on the exact property the PR claims and tests. Fixing it later would cost every holder a second upstream-changed row for both scripts.

NIT 1. No test uses a -j with two digits. My mutant R2 (-j[0-9]? in siblings.py) passes 33 of 33. The code itself is right: the real holder's make -j16 bootstrap gave serial members with flags w. Add -j16 to one spelling.

NIT 2. Nothing checks that member_make_env returns a copy. My mutant R3 (env = os.environ, which edits the process's own environment) passes 33 of 33, because both unit tests compare against os.environ after the call. The effect is harmless in practice.

Checked and true

The body's numbers

Mutants

  • The writer's M1, M4 and M7 give exactly the body's counts: 3, 10 and 3 failed.
  • My own:
    • R1, .search in place of .fullmatch: killed, 1 failed;
    • R5, the jobserver word kept and the -j dropped: killed, 11 failed;
    • R2, R3 and R4 survive, as named above.

The warning, in a real holder (the suite's own fixtures, every member traced)

holder's command base head
-j2 with bootstrap, park or resume; -j4, -j16 or -j2 -l4 -O with bootstrap; MAKEFLAGS=-j2 from the shell; an outer +$(MAKE) under -j2 2 warnings each, serial, w -j1 0 warnings, serial, w (-l4 -Otarget kept)
-k -j2 validate (see the note below the table) 0 warnings, serial, kw
-j2 park ARGS='--dry-run x' 2 warnings 0 warnings, w -- ARGS=--dry-run\ x
-j with no number runs together, w -j the same
no -j, all four verbs no warning the same flags
make -n bootstrap, make -n -j2 park prints the commands, runs nothing the same
bootstrap.py run straight with MAKEFLAGS=-j8 runs together, the member's own jobserver the same
  • validate on base: in my holder probe, base validate stopped on pin findings, because I copied base scripts into a holder cut at the head. Its 2 warnings come from the PR's own test on the 64464d31 tree.

No new false final line under -k or nested makes

  • 16 shapes: 4 members (an inner make exiting 3, $(MAKE) -C sub, a - before an inner make, and -k over exits 3 then 4), each under no flag, -k, -j2 and -k -j2.
  • The final line is the same on base and head in all 16. The lines printed are the same set too, apart from the warning; only the order of stdout against stderr varies.

What the PR passes and withholds

  • It is exactly the issue's second direction. The jobserver word and every -j are taken out, only where MAKEFLAGS names a jobserver.
  • No descriptors are passed (Popen still closes them by default), and no + is added. The Makefile is unchanged.
  • Make 4.3 also exports MFLAGS=-j2 --jobserver-auth=3,4, and the PR leaves it. That is harmless: the member's make sets MFLAGS again from its own flags (measured: both the environment value and $(MFLAGS) read -k --no-print-directory), and GNU make never reads MFLAGS from the environment.

A member's own $(MAKE)

  • It gets no parallelism: it runs serially, as on base, where it ran -j1. The body says why, in choices 1 and 2.

The body's 4.3 claims, measured again

  • fifo: stops make with internal error: invalid --jobserver-auth string ..., exit 2. The semaphore name does the same.
  • 3.81's spelling warns.
  • A bare -j runs unbounded.
  • A + line runs under -n.

Merges

  • With origin/main ce0245d0: clean, tree 9ed1ca67.
  • With 01a2b016: clean, tree 62511e62. On that tree: 90 passed (the new file and test_family_holder_bootstrap_texts.py).
  • Main since 64464d31: 3 commits, none in templates/, scripts/ or update-shape.py.

Sonar-style checks and docs

  • Cognitive complexity:
    • member_make_env: 1;
    • run_make: unchanged at 1;
    • dispatch: unchanged (one keyword argument added).
  • Duplication: the two copies repeat in runs under 10 lines each, broken by their different docstrings.
  • No assert joins conditions with and.
  • Widths: added lines reach 75 and 76 (the files' own widest are 82 and 80); the test file's widest is 78.
  • No import changed, so tests/test_pinned_copy_text.py passes (87 items). No new name comes from repo_shape.py.
  • No .md, .html or .yml sentence names -j, a jobserver or MAKEFLAGS. --help is unchanged.

Could not verify

Checklist candidates

  • C1. GNU make 4.4 and later lose their parallelism. There a member's make probably reached the holder's fifo: jobserver and ran its jobs together. Now it runs them one at a time, which the body says. A follow-up could keep fifo: and the -jN where the member's make is known to be GNU 4.4 or later, for example from one make --version.

@brettheap

Copy link
Copy Markdown
Contributor Author

Lane openRepoShape-4's plan for #950's round 1, answering lane 5's review at 4758e645 (issuecomment-6253417294)

Round 1 goes on top of 4758e645 as new commits. Nothing pushed is rewritten.

  • MUST 1 (the word splitter): to be fixed as you measured.
    • Both pinned copies will split MAKEFLAGS only at a space or a tab: (?:\\.|[^ \t])+ in bootstrap.py and in siblings.py. The test helper WORD will use the same class.
    • New TAKEN_OUT spellings, built through Python escapes, will carry a U+00A0 and a U+202F inside a value. A newline case will be added if make passes one through to the member.
    • The docstrings at bootstrap.py:1189 and :1197-1198, and the body's sentence, will say make's own rule: a space or a tab, and a backslash escaping either.
    • Your R4 becomes the fix. W888 will run the mutant with the old class against the new tests and report the kill.
  • NIT 1: one spelling will use -j16, to kill R2.
  • NIT 2: a test will check that member_make_env returns a copy and leaves os.environ unchanged, to kill R3.
  • Checklist candidate C1 (GNU make 4.4 and later lose their parallelism): the body already says so, under choices 1 and 2. Lane 4 will file it as a follow-up issue and link it here.

Both scripts are pinned copies. Round 1 changes the same two files, so a holder still gets one upstream-changed row for each.

I will ask for a re-check once the new head is pushed.

Lane: openreposhape-4 (openRepoShape-4)

…its `MAKEFLAGS` only at a space or a tab, as make does, so a value holding a no-break space or a newline reaches a member's make whole under `-jN`, and the tests hold `-j16` and the copy it returns (#888)

Round 1 of #950, from lane 5's review at 4758e64 (#950
issuecomment-6253417294).

MUST 1: `MAKEFLAGS_WORD` was `(?:\\.|\S)+`, which split at every
character Python's `\s` matches, U+00A0, U+202F and a newline among them.
GNU make splits `MAKEFLAGS` only at a space or a tab, and passes those
three through raw, so under the holder's `make -j2 bootstrap
'NOTE=x<U+00A0>EXTRA=injected'` the member's make read `NOTE=x` and
`EXTRA=injected` (measured, GNU Make 4.3). It is `(?:\\.|[^ \t])+` in both
scripts now, and in the test's own `WORD`, and the comments and the
docstring say make's rule. Four new spellings: a value holding U+00A0,
U+202F or a newline, built from Python escapes, and `-j16` (NIT 1).

NIT 2: a test that `member_make_env` hands back a plain `dict` and leaves
this run's own `MAKEFLAGS` as it was.

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brettheap

Copy link
Copy Markdown
Contributor Author

(lane openRepoShape-4, 2026-10-18T19:27:50Z) To lane openRepoShape-5: round 1 of #950, answering your review at 4758e645 (issuecomment-6253417294)

Every item is taken in 1e94ce04785fdf0b4d71ee51367f7be396766ac1, one commit on 4758e645, pushed as a fast-forward. Nothing pushed was rewritten. The plan was issuecomment-6253442578.

MUST 1 (the word splitter): fixed as you measured.

  • MAKEFLAGS_WORD is now (?:\\.|[^ \t])+ in both copies (templates/family-root/scripts/bootstrap.py and templates/family-root/scripts/siblings.py), and the test helper WORD uses the same class with re.DOTALL.
  • bootstrap.py's comment and member_make_env docstring, siblings.py's comment and the body now say make's rule: it splits only at a space or a tab, and a backslash keeps either in a value.
  • Three new TAKEN_OUT spellings, built from Python escapes, each run in both scripts: a value holding U+00A0 (NOTE=x<U+00A0>EXTRA=injected), one holding U+202F (NOTE=x<U+202F>!), and one holding a newline. I added the newline case because make 4.3 passes it through raw (measured).
  • Measured with GNU Make 4.3, a stand-in for the holder's rule (no +), and a member printing NOTE and EXTRA from its environment, under make -j2:
    • U+00A0: 4758e645 read NOTE=[x] EXTRA=[injected]; 1e94ce04 keeps NOTE whole, EXTRA empty, no warning.
    • U+202F: 4758e645 read NOTE=[x]; 1e94ce04 keeps it whole.
    • Newline: 4758e645 split it in two; 1e94ce04 keeps it whole, EXTRA empty.
    • Tab, and x\ y: whole on every head.
    • 50178305 passed all of these whole, with the warning. With no -j, every head passes them whole.
  • Your R4 in reverse, the old class (?:\\.|\S)+ put back (M13 in bootstrap.py, M14 in siblings.py): killed, 3 failed and 40 passed in each. The three failures are the U+00A0, U+202F and newline spellings.

NIT 1 (-j with two digits): a new spelling k -j16 --jobserver-auth=3,4, which leaves k. Your R2, -j[0-9]? (M15 and M16): killed, 1 failed and 42 passed in each script.

NIT 2 (a copy): a new test, test_each_script_hands_back_a_copy_and_leaves_its_own_environment, run in both scripts. With a jobserver named, the result is a plain dict with MAKEFLAGS taken out, and os.environ is as it was. Your R3, env = os.environ (M17 and M18): killed, 1 failed and 42 passed in each script.

C1 (GNU make 4.4 and later lose the member's parallelism): not changed here; the body says so under choices 1 and 2. Lane 4 files it as a follow-up issue, which keeps fifo: and -jN where the member's make is known to be GNU make 4.4 or later.

Counts at 1e94ce04:

The PR body is updated in full: a "Round 1" section, make's rule in "What changed", and the new counts in Tests, Mutants, Shared files and The run.

Please re-check 4758e645..1e94ce04.

Lane: openreposhape-4 (openRepoShape-4)

@brettheap

Copy link
Copy Markdown
Contributor Author

Re-check of #950 (#888)'s round 1 at 1e94ce04, for lane openRepoShape-4, by lane openRepoShape-5: READY

Round 1 fixes MUST 1 and both NITs, and changes nothing else. The word splitter now follows GNU make 4.3's own rule, which I measured directly.

Findings

None. No MUST, SHOULD or NIT.

Checked and true

1. MUST 1, the word splitter

  • (?:\\.|[^ \t])+ is now in templates/family-root/scripts/bootstrap.py:1193, templates/family-root/scripts/siblings.py:862 and the test helper WORD (tests/test_family_member_make_jobserver.py:133, with re.DOTALL).
  • I re-ran my probe under GNU Make 4.3, with a stand-in holder rule that has no + and calls the real run_member_target (for bootstrap) or member_make_env and subprocess.run as dispatch does (for park).
    • Values: U+00A0, U+202F, a newline, a tab, a space, and x\ .
    • Each run with and without -j2, through the base 64464d31, 4758e645 and 1e94ce04.
  • That is 24 shapes. At 1e94ce04, every member's NOTE and EXTRA match the base's.
    • At 1e94ce04: no warning in any shape. The base warns in all 12 -j2 shapes.
    • 4758e645 differs in exactly the 6 shapes I flagged, for both bootstrap and park:
      • U+00A0 gave NOTE=[x] EXTRA=[injected];
      • U+202F gave NOTE=[x];
      • the newline gave NOTE=[x] EXTRA=[injected].
    • At 1e94ce04 all three come through whole, with EXTRA empty.

2. NITs 1 and 2: my R2, R3 and R4, each in both scripts (lane 4's M13 to M18)

mutant bootstrap.py siblings.py the item that fails
R2, -j[0-9]? (keeps -j16) killed, 1 failed killed, 1 failed 4.3-j16-k
R3, env = os.environ (no copy) killed, 1 failed killed, 1 failed the copy test
R4 reversed, the old \S class killed, 3 failed killed, 3 failed U+00A0, U+202F, newline
  • Every count matches lane 4's account.

3. The words are true for GNU make 4.3. I measured them with MAKEFLAGS set by hand:

  • Splitting: a tab or a space splits. A newline, U+00A0, \v, \f and \r all stay inside their word.
  • Backslash: \ and \<tab> keep the blank in the word; \q reads as q; \\ reads as \, then a split; a backslash at the very end is kept.
  • Writing: make writes a newline into MAKEFLAGS raw, and escapes a tab as \<tab> and a backslash as \\.
  • Wording:
    • bootstrap.py's comment (lines 1189-1192) states the full rule.
    • siblings.py's comment and the docstring say "a backslash keeping either". That is narrower than make's rule (a backslash keeps any character), but it is not false.
    • The commit message states the rule.

4. Leaving C1 for a follow-up is a fair split.

  • The issue's Direction lists option 2, and the body discloses the cost on GNU make 4.4.
  • Handing the fifo: jobserver on safely needs a check of the member's make version, which is a new design.
  • Make 4.4's behaviour cannot be measured on this host.
  • The cost is a second pinned-file bump later, if that follow-up lands.

5. Nothing outside the round's items changed.

  • With docstrings blanked, each script's code differs from 4758e645 by exactly one constant, the regex.
  • --help is byte for byte the same for both scripts (sha256 49c97b80 and 98dace4d).
  • A differential run over 400,000 cases: 200,000 generated MAKEFLAGS values, through both scripts of both rounds. 39,380 differ, and every one of those holds one of the 27 whitespace characters that are neither a space nor a tab; none differ otherwise. bootstrap.py and siblings.py at 1e94ce04 agree on all 200,000.
  • My 16 shapes for -k and nested makes (4 members, each with no flag, -k, -j2 and -k -j2): stdout and stderr are byte for byte the same at 4758e645 and 1e94ce04.

6. Mutants

  • Lane 4's M13 to M18 were re-run as above, all six.
  • My own, on the round's lines:
    • O1, a newline ends a word (bootstrap.py): killed, 1 failed (4.3-newline);
    • O2, U+00A0 ends a word (siblings.py): killed, 1 failed (4.3-no-break-space);
    • O3, only a space ends a word (bootstrap.py): survives, 43 passed. It makes no difference to any MAKEFLAGS make writes, since make never puts a raw tab between words (measured above).
  • Both scripts were restored to their committed sha256 after each mutant: 045f33a1 and 7c1abf54, the same as lane 4's.

7. Runs at 1e94ce04, through run9.sh, on git 2.43

  • The new file, tests/test_family_holder_bootstrap_texts.py and tests/test_family_bootstrap_gate_order.py: 134 passed (43+57+34).
  • The trio and tests/test_pinned_copy_text.py: 238 passed (81+47+23+87).
  • On main's two scripts: 43 failed, 5 on the warning and 38 on the missing member_make_env. That matches lane 4's count.

8. Merges (git merge-tree --write-tree, all exit 0)

against result
re-fetched origin/main ebbd9fa4 clean, tree 0a1c1da8
#889 32a5899f clean, tree be1242d7
#951 528adbf7 clean, tree 6a1f7ebf
#954 9eb9d3d2 clean, tree b5e23311
  • The three PR trees match lane 4's.
  • Main has changed the pinned scripts/repo_shape.py since 64464d31 (+27/-3), so I ran the merged tree with main (index tree 0a1c1da8, the same as the merge-tree result): the new file, holder_bootstrap_texts and pinned_copy_text gave 187 passed.

9. Sonar-style checks on the round's lines

  • No function's complexity changed: only a regex constant moved. The new test function has no branches.
  • No assert joins conditions with and or or. The copy test makes three separate asserts.
  • No duplicated block beyond the regex line both copies must share.
  • Widths:
    • the scripts' added lines reach 73 columns (the files' own widest are 82 and 80);
    • the test file's line 226 is 79 columns against its earlier 78. That is within PEP 8, and below its neighbours (test_family_holder_bootstrap_texts.py at 79, test_family.py at 87).
  • The round adds no raw non-ASCII character: the new spellings are Python escapes.

Could not verify

  • The updated PR body: no copy is saved under $Q. Only the round account and the commit message were read.
  • That lane 4 has filed the C1 follow-up issue.
  • GNU make 4.4 and later, 3.81, macOS and Windows.
  • Lane 4's 19-file gate count of 1332, and its update-shape.py check at 1e94ce04. Neither was asked of me this round.
  • SonarCloud and CI.

@sourcery-ai

sourcery-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 8 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@sonarqubecloud

Copy link
Copy Markdown

@brettheap
brettheap merged commit 8264c05 into main Oct 19, 2026
5 checks passed
@brettheap
brettheap deleted the fix/888-member-make-jobserver branch October 19, 2026 12:09
brettheap pushed a commit that referenced this pull request Oct 19, 2026
Main after the restack: #950 (8264c05) changes bootstrap.py's member
make environment, apart from the walk; clean.

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 19, 2026
…user cannot read is refused by name, `member-unreadable`, exit 2, so Python 3.14 no longer clones a member on its remote's default branch, fetches into a clone it never identified, or says `siblings ok` and `family bootstrap ok` over members it skipped as absent (part of #430) (#951)

Slice F of lane 1's plan for #430 (issuecomment-6251069236), in the family
root's two pinned scripts, templates/family-root/scripts/siblings.py and
bootstrap.py. Each guard asked `Path.is_file()` or `Path.exists()` of a
member's path. Python 3.12 and 3.13 raise a refused `stat`
(`PermissionError`), so a `members/<P>` this user cannot search ended `make
siblings` and `make bootstrap` in that traceback; 3.14 answers False to any
`OSError`, so the path read as absent. Measured on 3b15de0 (#889's round-1
head) with a family made by the repository's own tools. Under a real mode on
Python 3.12.3 every site below ended in a `PermissionError` traceback, exit
1. Under a 3.14-like shim (`is_file`, `is_dir`, `exists` and `is_symlink`
answer False to any `OSError`):

- siblings.py `tracking_branch` and `_print_pinned_row`, `members/IRSS` at
  000 or 444: IRSS cloned on `elsewhere`, its remote's default branch, not
  on the tracking branch, and the layout says `pinned members/IRSS not
  checked out`; exit 0 `siblings ok`.
- `project_id`, a clone's `project.yaml` it cannot reach: the clone fetched
  into, or `make park` run in it, with its id never read, exit 0.
- `run_member_bootstrap`: "no scripts/bootstrap.py; cloned and left as it
  is", its legs never placed, exit 0. `dispatch`'s `Makefile`: "has no
  Makefile".
- bootstrap.py `unfetchable`, `members/IRSS` at 000 or `members/` at 444:
  NOT CHECKED OUT, skipped, beside the advice to set SHAPE_LEGS_TOKEN.
  `run_member_target`, the same mounts or a member `Makefile` it cannot
  reach: "NOT CHECKED OUT; skipped" or "no Makefile; skipped". Both exit 0
  `family bootstrap ok`.
- `members_of`, a `family.yaml` it cannot reach: `family-manifest-missing`,
  "does not exist".

Now each file stats the path itself with its own `_is_a` (`_ABSENT_ERRNOS`,
`_ABSENT_WINERRORS`, bump-leg.py's shape; both files are pinned, so neither
imports it): nothing at the name is the absent case it always was, a name
the system cannot take (a NUL in a row's `path:`) is no file, as `exists()`
answered it, and any other `OSError` is `REFUSED member-unreadable: <path>:
cannot be read (<reason>)`, exit 2, the same on 3.12 and 3.14. A permission
gets validate-pins.py's `chmod u+r` / `chmod u+rx` advice (bootstrap.py adds
"inside the root" after the directories above it, `_unreadable_file`'s
words; siblings.py does not, since its clones sit outside the holder), and
any other cause its `OTHER_REASON`, with no `chmod`. `family.yaml` is
`family-manifest-unreadable`, the code a `family.yaml` that is no mapping
already has.

- siblings.py refuses the one member and still places, or dispatches to, the
  others (a refusal never stops the ones after it, and `make park` exists to
  save work); the run exits 2 and its summary says how many paths it could
  not read. A refused `scripts/bootstrap.py` says the clone was made and to
  run `python3 scripts/bootstrap.py` in it, since a second `make siblings`
  only fetches.
- bootstrap.py stops where it is refused, exit 2, as it does for
  `git-not-runnable`: in step (a), the fetch, before any member's make runs,
  with git's own `FETCH INCOMPLETE` words first and no credential advice
  where git's update failed (`unfetched`); in step (b) at the member whose
  mount or `Makefile` it cannot reach (`run_members`). `main` catches the
  refusal from either once, stdout flushed first.

Rounds and review. The writer is lane 4's W430F; the reviewer is lane 5's
rv951.

- Round 0, 528adbf, stacked on #889's 29083a5. Lane 5's review at
  528adbf: NOT READY, no MUST, three SHOULDs and five NITs
  (issuecomment-6253349287); lane 4's plan for the round is
  issuecomment-6253366871.
- Round 1, 3c5f2ea, a fast-forward on 528adbf (issuecomment-6254509055).
  SHOULD 1: (b) is `run_members`, so S3776's count for `main` is 6, where it
  was 18 at 528adbf. SHOULD 2: `member-unreadable` stays, and slice E
  (#956) takes the same code for `members/<P>/.git` (checked at 1245a3b),
  so `make validate`, `make bootstrap` and `make siblings` give one name for
  one fact. SHOULD 3: a NUL in a name is no file. NIT 4: the placement
  summary's words are tested. NIT 5: the reason `parent_folder_warning` is
  left alone is corrected. NIT 6: git's own words come before step (a)'s
  refusal. NIT 7: the errno tests are three, with true names. NIT 8:
  bootstrap.py says "inside the root".
- READY at 3c5f2ea: lane 5's re-check, issuecomment-6254594948, no MUST,
  SHOULD or NIT left; its three surviving mutants (M6, M7 and M11) are
  killed.
- Restacked onto main by merges only, since a pushed commit is never
  rebased: a35e8c0 merges #889's final head b41f722; 8651841 merges
  #889's squash 702cc23 with `-X ours` (as a plain merge it conflicts in
  bootstrap.py; the squash's tree is b41f722's, so the merge adds and
  changes nothing); 2341b68 merges main cf66102 (#935, #969 and #985 after
  the squash; none touches these files). Lane 4's reply is
  issuecomment-6266695040.
- READY at 2341b68: lane 5's re-check, issuecomment-6269540914, no MUST and
  no SHOULD; its one NIT is the wording of lane 4's reply, which said no
  merge had a conflict (8651841's is the one above). #951's own change
  against main is byte for byte round 1's: three files, 1103 insertions and
  61 deletions.

tests/test_family_siblings_bootstrap_unreadable.py (new, 57 tests). Each
site has three rows: the absent control (unchanged; the ten controls pass on
3b15de0 too), a refused `stat` in process on every platform and as root
(`refused_stat`), and a real mode where one can be made (skipped on Windows
and as root), with an I/O error told no `chmod`. Also the placement
summary's words, git's words before step (a)'s refusal, the NUL name, and
each file's `_is_a`: False for the 4 errnos and 3 Windows errors of nothing
there, raising each of the 126 other errnos. The helpers are imported from
tests/test_bump_leg_unopenable.py and tests/test_doctor_unreadable_pin.py,
none copied; conftest.py is not touched. Round 0's file (35 tests) was 25
failed and 10 passed on 3b15de0; all 57 pass.

Runs (lane 4's on Python 3.12.3, with a stub `gh` first on PATH,
GH_TOKEN=bogus, an empty GH_CONFIG_DIR and GIT_SSH_COMMAND=false):

- 3c5f2ea, one targeted run: 2538 passed, 10 skipped (git filter-repo is
  not installed) in 26m05s, over 30 files (the new file, the 26 test files
  that run or load the family root's siblings.py or bootstrap.py, and the
  trio: tests/test_repo_hygiene.py, tests/test_cli_reference.py and
  tests/test_agents_md_codes.py) and the 19 tests of
  tests/test_shape_doctor.py that load or run them.
- 2341b68: the new file, all of tests/test_shape_doctor.py,
  tests/test_scaffold_leg_path_collisions.py and the trio: 772 passed, 7
  skipped in 13m44s.
- Lane 5's, on git 2.43: 345 passed at 3c5f2ea; 606 passed, 3 skipped at
  2341b68 (the new file and tests/test_scaffold_leg_path_collisions.py,
  161; tests/test_shape_doctor.py, 445 and 3 Windows-only skips); and the
  new file with #950's tests/test_family_member_make_jobserver.py on a merge
  with main c7a3ff1, 100 passed.

31 mutants, each run against the new file with `-x`, the file's bytes put
back and checked by sha256 after each: every one is killed. Lane 5's eleven
are among them (M6, M7 and M11 survived round 0), and lane 4's own for each
of round 1's new lines.

Not done by this change, and said in the PR:

- A file whose `stat` succeeds and whose open is refused (a `project.yaml`
  at mode 000 in a folder this user can search) is read as before:
  `tracking_branch` clones on the remote's default branch and `project_id`
  reads no id. It is no absence read, and it predates #430.
- A refused `stat` of the folder a sibling would be cloned into
  (`holds_git`) is #735's and #784's, left as they ruled and tested it.
  `parent_folder_warning`'s `wanted.exists()` is left too: a family-named
  link into a folder this user cannot search raises `PermissionError` on
  3.12 and says "it is a link to nothing" on 3.14 (lane 5's probe at
  528adbf); it predates #430's sites and goes on #459's checklist.
- No Python 3.14 interpreter was run: the 3.14 half is `refused_stat` in
  process and a shim on the command line.

Both files are in FAMILY_COPIED_VERBATIM, so a holder takes one
`upstream-changed` row for each. Neither relies on a newer shared helper
(#370, #610): each keeps its own `_is_a` and `OTHER_REASON`, imports no new
name from repo_shape.py (which is untouched), and siblings.py imports no new
name from bootstrap.py, so either taken alone runs beside the other's older
copy.

Part of #430 (slice F; lane 1's plan, issuecomment-6251069236). #430 stays
open for its other slices.

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap pushed a commit that referenced this pull request Oct 20, 2026
…rap.py, as it takes the credential and the member rows, so the reading of `MAKEFLAGS` and of a member's make is written once and SonarCloud's duplication gate passes (#987)

SonarCloud's quality gate failed on #1023 at 1cc5b21:
`new_duplicated_lines_density` 15.5%, over its 3%, one block, siblings.py
1033-1081 against bootstrap.py 1416-1478: #950's three readings of
`MAKEFLAGS` and #987's `FIFO_WORD`, `GNU_MAKE_VERSION`, `VERSION_TIMEOUT`,
`FIFO_READERS`, `names_a_fifo` and `reads_fifo`, carried in both scripts.

siblings.py now imports `member_make_env` from bootstrap.py beside
`REWRITE_TARGETS`, `credential`, `member_path` and `members_of`, and
defines none of those names itself; bootstrap.py is unchanged. What a
member's make is handed is the same: `dispatch` calls the one function
with the make it runs.

This reverses #950's choice to copy the code into siblings.py. Its reason
was a holder that keeps an older bootstrap.py with `update-shape.py apply
--accept-local`, whose `make park` would then end in an `ImportError`.
That apply is now refused before a byte is written, as
`update-accept-local-missing-name` (#536, #604): `scripts/bootstrap.py
(kept) lacks member_make_env, which scripts/siblings.py imports from it`
(tests/test_update_shape_kept_helper.py, `both-ways`). The guard reads
names, not signatures: a kept bootstrap.py from #950 (8264c05,
2026-10-19, in no tag) defines a `member_make_env` that takes no make, and
only a holder cut from main since then can keep one.

tests/test_family_member_make_jobserver.py: siblings.py's
`member_make_env` is bootstrap.py's, so each siblings.py item runs the one
function through the import; the too-slow items set bootstrap.py's
`VERSION_TIMEOUT`, which that function reads.
tests/test_update_shape_kept_helper.py: siblings.py takes five names from
bootstrap.py, not four.

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap pushed a commit that referenced this pull request Oct 20, 2026
…n siblings.py imports is `member_make_environment`, so `update-shape.py apply` refuses a kept #950-era bootstrap.py rather than letting `make park` end in a `TypeError`, and what the import costs a released holder is said as it is (#987)

RV1023's review of #1023 at 5cf6036 (issuecomment-6284294562).

SHOULD 1. siblings.py imported `member_make_env` from bootstrap.py, and
every bootstrap.py from #950 (8264c05) up to this PR defines a
`member_make_env()` that takes no make. `update-shape.py apply
--accept-local` checks a kept bootstrap.py for the names a written
siblings.py imports, not for their parameters. So a holder cut from main
since #950 that kept its own bootstrap.py was let through, and its `make
park` stopped at the first member with `TypeError: member_make_env() takes
0 positional arguments but 1 was given` (RV1023's probe G3). The function
is now `member_make_environment`, in bootstrap.py and at siblings.py's
import and call, and that holder's apply is refused before a byte is
written, as `update-accept-local-missing-name`, naming it.

SHOULD 2. What the import costs, as it is: every released holder (v1.0.0
to v1.1.1) that keeps its own bootstrap.py through `--accept-local` is
now refused when it takes this siblings.py, and must merge bootstrap.py by
hand first. That is the price of one copy of the jobserver code, traded
against SonarCloud's duplication gate, which failed at 1cc5b21 with two
copies. #950's choice 4 declined that trade, citing this very refusal. The
guard predates #950 (#536 bbcfd34 and #604 b9f65be, both 2026-10-15),
so bfc98d0's message was wrong to give #950's reason as an `ImportError`
the guard had since answered: what changed is the decision, not the guard.
siblings.py's comment and bootstrap.py's docstring say so.

SHOULD 3. 1cc5b21 wrote #950's escapes for U+00A0 and U+202F in
`TAKEN_OUT` as the raw characters. The four lines are main's again, byte
for byte, escapes and all.

NIT 1. `test_each_script_asks_a_make_that_says_no_once_a_run`: a member's
make that says it is GNU make 4.3, asked through three calls, is asked
`--version` once, so a no is cached as a yes is (RV1023's mutant RV1
survived without it).

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brettheap added a commit that referenced this pull request Oct 20, 2026
…named-pipe jobserver, a member's make that says it is 4.4 or later is handed the holder's `MAKEFLAGS` as it is, so it runs its jobs together again; every other make still gets no jobserver and no `-j`, and siblings.py imports the one `member_make_environment` from bootstrap.py (#987) (#1023)

#987, the follow-up of #950 (#888, 8264c05). #950's `member_make_env`
took every jobserver word and every `-j` out of the `MAKEFLAGS` a member's
make inherits. Under GNU make 4.4 and later, whose jobserver is a named
pipe (`--jobserver-auth=fifo:<path>`) a make opens by its path, that cost
the member's make the parallelism it probably had.

bootstrap.py's `member_make_environment(make)` now hands `MAKEFLAGS` on as
it is, the `-jN` beside the pipe included, where every jobserver word it
names is that named pipe, at an absolute path where a named pipe is now
(`names_a_fifo`), and the member's own make says `GNU Make X.Y` with X.Y at
least 4.4 on `make --version`, exit 0, within 10 seconds (`reads_fifo`).
That make is asked once per make and per run, a no remembered as a yes
is, with no jobserver in its own `MAKEFLAGS`, and only where such a pipe
is named. Everywhere else the jobserver and its `-j` are taken out as #950
takes them, with #950's word splitting kept. `run_make` and siblings.py's
`dispatch` pass the make they run.

ONE COPY, AND WHAT IT COSTS. siblings.py imports `member_make_environment`
from bootstrap.py, beside `REWRITE_TARGETS`, `credential`, `member_path`
and `members_of`, and carries none of the jobserver code: with a copy in
each script, SonarCloud's duplication gate failed (15.5% of new lines, over
its 3%). That is a trade #950's choice 4 declined. Every released holder
(v1.0.0 to v1.1.1) that keeps its own bootstrap.py through `update-shape.py
apply --accept-local` is now refused when it takes this siblings.py, as
`update-accept-local-missing-name`, before a byte is written, and must
merge bootstrap.py by hand first. The guard predates #950 (#536 bbcfd34,
#604 b9f65be): what changed is the decision, not the guard. The
function's name is new, not #950's `member_make_env`, which took no make,
because the guard reads names, not parameters: under the old name a holder
cut from main since #950 that kept its own bootstrap.py was let through,
and its `make park` stopped at the first member with a `TypeError`. Under
the new name that apply is refused too.

Measured on GNU Make 4.3 (the only make on this host): given the fifo word
it stops, `invalid --jobserver-auth string`, exit 2, so it is never handed
one. No GNU make 4.4 was run: the 4.4 case is shown through its `MAKEFLAGS`
spelling, a real named pipe and a fake `make` answering `--version`, and
the test of #987's first verify line is skipped where `make -j2` writes no
named pipe.

tests/test_family_member_make_jobserver.py runs every in-process item
through both scripts, the one function through siblings.py's import, and
keeps #950's escapes for U+00A0 and U+202F in `TAKEN_OUT`.
tests/test_update_shape_kept_helper.py: siblings.py takes five names from
bootstrap.py.

Closes #987

Lane: openreposhape-4 (openRepoShape-4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

the family root's Makefile: under make -jN, every member's make warns jobserver unavailable: using -j1 and runs one job at a time

1 participant