[brettheap]

Lane: openreposhape-1 (openRepoShape-1)
Claim: #655 (comment)

What

This PR answers #655: two P2 lines of #457, L1-56 and L1-57, and the two riders on #617's answered_paths: rows, L1-58 and L1-59.

• L1-57: execute records a plan's answer against a rule. This is decision (A) of lane 1's triage, which Brett Heap approved on 2026-10-16T11:21:17Z.
  • execute writes an answered_paths: row for each plan entry kept in a leg that covers a path the policy's walk does not put in that leg. That is a path the walk leaves leg: null, as adopt-project.py, shape-doctor.py, validate-manifest.py: execute records the plan's answers under answered_paths: and a moved submodule names its row, so an answered examples/ is not asked again and a deliberate move names no plan (#525) #617 recorded, or a path a rule calls for the other leg or the root.
  • An entry answered as its rule says gets no row, and nor does one left as plan proposed it where a gitlink changed how plan folded its directory. So does one answered root or drop.
  • The doctor holds a walked path in the wrong leg to the record as adopt-project.py, shape-doctor.py, validate-manifest.py: execute records the plan's answers under answered_paths: and a moved submodule names its row, so an answered examples/ is not asked again and a deliberate move names no plan (#525) #617 holds a question: file by file, and only in the leg the row names.
  • An answered path is listed under --json's answered_paths and keeps its classified_as and rule. It is named in a clause of its own: project.yaml's `answered_paths:` answers 1 path(s) into the leg they are in, which a rule puts elsewhere (#655): code/docs/ (spec by rule spec-governance).
• L1-56: the doctor stops asking about an empty .gitmodules.
  • A .gitmodules that holds no byte registers no submodule. The question ambiguous-gitmodules asks of it ("which leg owns the submodules it registers") has no subject, so no answer could be wrong.
  • The doctor no longer asks it. The leg's entry names the file under empty_gitmodules, and the row says the policy asks nothing of 1 `.gitmodules` that hold(s) no byte and so register(s) no submodule (#655): code/.gitmodules.
  • A .gitmodules holding anything, even one blank line or a comment, stays the question it was.
• L1-58: a directory's row without its / is said.
  • The row still answers nothing under the directory: _covering reads it as a file's row, as check reads a plan entry.
  • Where the leg the row names holds a directory at that path, the leg's entry names it under unslashed, and the row says 1 row(s) of project.yaml's `answered_paths:` name a directory without the `/` a directory's row ends with, so each answers nothing under it (#655): spec/examples.
  • It is never a finding, and it never keeps the row off ok.
• L1-59: what a directory's row answers is said where the row is read. The comment execute writes above the block now says: "A directory's row answers for every file under it, one added after the adoption too: the doctor does not hold a file to source_commit:." The docstring of the project's validate-manifest.py says the same, ending "nothing holds a file to source_commit."
• AGENTS.md. The one line on the record now says answered_paths: holds a row "per other path the policy asked about or put elsewhere". The file stays at 492 of 492 lines.

Why (repro on main a524542; the same on 35133f2)

The source holds README.md, src/app/util.py, docs/index.md, a golden run under examples/ and a submodule at vendor/rules. The plan answered examples/ as spec, vendor/ and .gitmodules as code, and docs/ (rule spec-governance, spec) as code, each with a resolution:. Every probe ran with a stub gh first on PATH, GH_TOKEN set to a bogus value, an empty GH_CONFIG_DIR, GIT_CONFIG_GLOBAL=/dev/null and GIT_ALLOW_PROTOCOL=file.

• L1-57. check exited 0, and execute exited 0 with adoption verified. answered_paths: held examples/ only. The doctor's first run exited 1 with MISPLACED (1 path): code/docs/: spec by rule spec-governance.
• L1-56. git rm vendor/rules in the code leg left code/.gitmodules at 0 bytes, tracked. Three runs of the doctor in a row each listed code/.gitmodules under review_required. A hand-written answered_paths: row for .gitmodules changed nothing.
• L1-58. With the examples/ row edited to examples, validate-manifest.py said manifest ok. The doctor asked spec/examples/ again and said nothing about the row.
• L1-59. A file staged under spec/examples/later/ was answered by the examples/ row, by design. The block's comment said only that each row answers "the question it would ask again of that path in that leg".

On this branch, the same script gives:

• L1-57: the doctor's first run exits 0, COMPLIANT, with placement ok, and both spec/examples/ and code/docs/ under answered_paths;
• L1-56: review_required is empty on every run, and the clause above is said;
• L1-58: the question is still asked and the unslashed clause is said;
• L1-59: the comment above says what a directory's row answers.

How

• adopt-project.py.
  • answered_path_record runs the policy's walk over the source's paths as before, gitlinks left out. It keeps each entry _answered_legs reads as answered spec or code (not .gitmodules) that covers a path whose walked leg differs from that answer. The walked leg is read by _walked_leg, from the walk entry that covers the path. An answer is recorded against a rule only where it also differs from the leg plan showed, read from the same walk with the gitlinks in and ask_about_submodules and ask_about_refused_paths applied, so an entry left as plan wrote it is never recorded.
  • The new _covering_names(path) lists every name _covering can match for a path: each directory above it, with its /, and the path itself.
  • ANSWER_RECORD_PREAMBLE, the module docstring and two comments in execute say what the record now holds.
• shape-doctor.py.
  • answered_questions is called twice in audit_leg: once for the questions, as adopt-project.py, shape-doctor.py, validate-manifest.py: execute records the plan's answers under answered_paths: and a moved submodule names its row, so an answered examples/ is not asked again and a deliberate move names no plan (#525) #617 did, and once for the walked paths in the wrong leg.
  • Its rule is unchanged. A path is answered when every file under it is covered by exactly one row, and that row names this leg.
  • It now looks each file up by _covering_names rather than holding every path against every file. That is the same answer, without the quadratic cost (see Tests).
  • answers_clause names the two kinds of answer apart.
  • empty_gitmodules takes a .gitmodules that is a regular file of 0 bytes on disk (lstat, as leg_tracked_files reads a size) out of the questions. git refuses to index a symlinked .gitmodules ("invalid path"), and where the disk has none git reads the index's copy, so neither counts as empty.
  • unslashed_rows names the rows for this leg whose path is a directory of the leg's classified files.
  • Two keys on a leg's entry, empty_gitmodules and unslashed, are present only when non-empty, so every other report is unchanged.
  • The module docstring has two new paragraphs.
• templates/assembly-root/scripts/validate-manifest.py: docstring only.
  • This changes a pinned template's bytes. Each project's copy reads upstream-changed to update-shape.py check until apply. adopt-project.py, shape-doctor.py, validate-manifest.py: execute records the plan's answers under answered_paths: and a moved submodule names its row, so an answered examples/ is not asked again and a deliberate move names no plan (#525) #617 already moved this file in v1.1.0, so a project on v1.0.0 updates it once either way.
  • The checks themselves are unchanged. The first docstring line and --help are unchanged, so docs/cli.md is unchanged too.
  • I grepped tests/ for regexes over template lines: none reads this file's lines, apart from the new L1-59 test, which reads its docstring.

Why L1-56 stops asking instead of letting a record answer

• The question has no subject. Every leg is a right answer for a file that registers nothing, so a recorded answer would assert nothing.
• execute never writes such a row. The file is emptied after the adoption, by git rm, so the record route needs a hand edit of project.yaml on every project where it happens, made only to quiet the doctor. Checking rule 3 forbids that edit.
• adopt-project.py, shape-doctor.py, validate-manifest.py: execute records the plan's answers under answered_paths: and a moved submodule names its row, so an answered examples/ is not asked again and a deliberate move names no plan (#525) #617 keeps .gitmodules out of answered_paths: on purpose. submodules: answers it, and only where it places every submodule the file registers. Letting a row answer it would weaken that rule.
• "No byte" is exact and needs no parser. git's own reader is the only judge of a file holding a comment or a blank line. The doctor would have to run it, and adopt-project.py: a plan that sends .gitmodules to one leg and a submodule it registers to another passes check and execute, says adoption verified, and the assembly's recursive clone then fails: plan should ask, check should find, execute should refuse #166 rules that a read that failed is not an answer.

Why L1-58 is said and not read as the directory's row

• One spelling means one thing. check reads examples as a file's entry, and calls it plan-empty-entry beside a plan-uncovered for each file under it.
• A lenient reading would let examples and examples/a.json both pass the validator, which sees no overlap between them, and then both cover a.json in the doctor.
• The validator cannot say it, because it runs where the legs are not checked out. The doctor reads the leg, so it says it.

Tests

• New file: tests/test_doctor_plan_overrides.py, 22 tests.
  • 12 fail on main. The other 10 guard rules main already keeps: a row for the other leg and a row for one file of two settle nothing; entries answered as their rule, root or drop get no row; an entry left as plan wrote it, beside a gitlink that kept plan from folding its directory, gets no row (r655's F1); a blank line or a comment in .gitmodules is still asked; file rows and a row for the other leg are not named.
  • Two tests are end to end and need git filter-repo: the adoption of docs/ against spec-governance, and the removal of the code leg's only submodule.
• tests/test_doctor_recorded_answers.py.
  • adopt-project.py, shape-doctor.py, validate-manifest.py: execute records the plan's answers under answered_paths: and a moved submodule names its row, so an answered examples/ is not asked again and a deliberate move names no plan (#525) #617's a-confident-rule-overridden case now expects docs/ recorded. Its docstring said "(outside adopt-project.py: record the plan's answers in project.yaml, so the doctor's placement row and the submodules: rows follow them #525)", and it fails on main.
  • adopt_source takes overrides beside answers, so the end-to-end adoption above reuses it.
• Revert table. Each change was reverted alone in its own clone: R0 to R18 of fc00656, before the F1 fix, where each ran 150 tests, and R19 of 0e37d02, where it ran 151. The tests were the new file, tests/test_doctor_recorded_answers.py and tests/test_manifest_validator.py. The table counts failing tests.

revert	fails
R0: main's source (all four non-test files)	13 (12 new and #617's changed case)
R1: execute records the policy's questions only (#617's filter)	7
R2: the doctor reads no answer for a path in the wrong leg	3
R3: a row for either leg answers	3 (1 new, 2 of #617's)
R4: any one answered file answers the path	3 (1 new, 2 of #617's)
R5: an empty .gitmodules is still asked	2
R6: "empty" means whitespace only	1
R7: any .gitmodules asks nothing	3 (2 new, 1 of #617's)
R8: the empty .gitmodules is not said	1
R9: a slashless directory row is not said	2
R10: every slashless row is named, directory or not	2
R11: a slashless row is named in either leg	1
R12: the block's comment as on main	2
R13: the validator template as on main	1
R14: AGENTS.md as on main	1
R15: an answer against a rule worded as a question's	1
R16: answered_questions' body as #617 wrote it, both calls kept	0 (behaviour-preserving; see the probe)
R17: execute records every entry kept in a leg	16
R18: execute records no answer against a root rule	1
R19: an answer judged against the gitlink-free walk alone (r655's F1)	1

Every one of the 22 new tests fails under at least one of R1 to R19.

• The cost probe. Main's answered_questions body was compared with this branch's on synthetic legs, at a load average of about 50. Half the paths were questions and half were in the wrong leg, and every other one was answered.

files / paths	main, questions only	main's body, both calls	this branch, both calls
5,000 / 200	0.22 s	0.54 s	0.02 s
20,000 / 1,000	4.55 s	9.72 s	0.07 s
60,000 / 3,000	38.50 s	74.56 s	0.37 s

The answers were identical in every row.

• answered_path_record now reads every file of the source through _answered_legs, not only the questions. On the same probe it took 0.97 s, 14.57 s and 123.96 s for 10,000, 40,000 and 120,000 files with 400, 2,000 and 6,000 entries. Main took 0.47 s, 7.89 s and 68.19 s.
• At each of those sizes, _coverage_findings, which check and execute already run over the same paths, took longer: 1.31 s, 25.71 s and 212.70 s.
• Both modes at the head 0e37d02 (dirty 0), with a stub gh first on PATH, a bogus GH_TOKEN and an empty GH_CONFIG_DIR. The files: tests/test_doctor_plan_overrides.py, tests/test_doctor_recorded_answers.py, tests/test_manifest_validator.py and tests/test_agents_md_codes.py as one group, and tests/test_shape_doctor.py as another.

group	git-filter-repo on PATH	git-filter-repo hidden
the four files	198 passed, EXIT=0	192 passed, 6 skipped, EXIT=0
tests/test_shape_doctor.py	405 passed, 3 skipped, EXIT=0	401 passed, 7 skipped, EXIT=0

The rest is covered by CI's full Linux suite on this PR and by the earlier runs at fc00656, which differs from 0e37d02 only in answered_path_record and the new test. Those runs covered 137 test files in six groups per mode: every file matching adopt-project|adopt_project|\bADOPT\b|shape-doctor|shape_doctor|bump-leg|bump_leg|validate-manifest|validate_manifest, plus tests/test_shape_doctor.py, tests/test_scaffold_leg_path_collisions.py, tests/test_repo_hygiene.py, tests/test_agents_md_codes.py, tests/test_cli_reference.py and the three files above. With git-filter-repo on PATH, 8,842 passed and 17 skipped; with it hidden, 8,496 passed and 363 skipped. All twelve groups exited 0.

Windows

CI is Linux only since #597, so the Windows newline emulation was not run. The new tests write .gitmodules with write_text (an empty string stays 0 bytes on any platform) and read every file a helper wrote back with read_text(); the one read_bytes() compare is of the empty .gitmodules that git rm itself leaves. Every path they create is legal on NTFS.

Not in this PR

• adopt-project.py plan and check are unchanged. An answer against a rule still passes check with no finding: that was option (B), which the triage did not take.
• _answered_legs is unchanged (see Known limits).
• README.md and docs/handbook.html are not touched.

Known limits

• README.md's verdict table says MISPLACED is "a tracked path sits in a leg the path policy puts elsewhere". It does not mention the exception the record now makes, and README.md does not mention answered_paths: at all. Changing README.md would put this PR in the handbook's serialised README turn.
• A row ending with / that names a file (notes.txt/) also answers nothing and is not said. Only the directory case, which is the likely hand edit, is said here. This is noted for Tracking: adopt-project.py, converting an existing repository in place #457.
• The cost of answered_path_record. It reads every source file through _answered_legs, which is O(files x entries), the same cost class as _coverage_findings. A faster exact _answered_legs is possible with _covering_names, but it is shared with check's submodule findings, so it is left alone here.
• Rows written before this PR. A row adopt-project.py, shape-doctor.py, validate-manifest.py: execute records the plan's answers under answered_paths: and a moved submodule names its row, so an answered examples/ is not asked again and a deliberate move names no plan (#525) #617's execute wrote for a question now also answers a path in that leg that a rule puts elsewhere, a file added after the adoption included, though the comment above that project's block says only that it answers the question. A project whose answered data/ later gains data/schema.md in the code leg read MISPLACED and now reads COMPLIANT.
• Older projects. A project adopted before this PR has no row for an answer against a rule, and its doctor still says MISPLACED for that path. No command writes the row afterwards, and nothing documents how to record one by hand. adopt-project.py, shape-doctor.py, validate-manifest.py: execute records the plan's answers under answered_paths: and a moved submodule names its row, so an answered examples/ is not asked again and a deliberate move names no plan (#525) #617 noted the same limit for a question's answer.

This PR answers four lines of #457, L1-56, L1-57, L1-58 and L1-59, and #457 itself stays open.

Closes #655

🤖 Generated with Claude Code

Summary by Sourcery

Preserve intentional adoption-plan overrides in recorded answers and make the shape doctor accurately handle empty .gitmodules files and directory answer rows.

New Features:

• Record adoption plan answers that place paths against their policy rules so the doctor treats those paths as intentional rather than misplaced.
• Report empty .gitmodules files, slashless directory answer rows, and directory-row coverage explicitly in doctor output.

Bug Fixes:

• Prevent empty tracked .gitmodules files left after submodule removal from being repeatedly reported as unresolved questions.
• Prevent paths intentionally answered into another leg from being reported as misplaced during shape audits.

Enhancements:

• Optimize recorded-answer coverage checks by looking up only path names that can cover each file.
• Clarify manifest, validator, project guidance, and generated record comments for rule overrides and directory answer semantics.

Documentation:

• Update user-facing manifest guidance to document answers against rules, directory coverage, and slashless directory rows.

Tests:

• Add comprehensive coverage for rule overrides, empty .gitmodules, malformed directory rows, later files under answered directories, and end-to-end adoption behavior.

[sourcery-ai]

Reviewer's Guide

The PR expands answered_paths from policy questions to all plan decisions that place a path outside its policy leg, teaches the doctor to honor those records and ignore truly empty .gitmodules files, adds diagnostics and documentation for malformed directory rows and directory coverage, and supplies broad regression and end-to-end tests.

Sequence diagram for recorded plan overrides during adoption and audit

sequenceDiagram
    participant Execute as adopt-project.py execute
    participant Policy as PathPolicy.walk
    participant Record as project.yaml answered_paths
    participant Doctor as shape-doctor.py

    Execute->>Policy: walk(files)
    Policy-->>Execute: called path legs
    Execute->>Execute: answered_path_record
    Execute->>Record: write paths answered into a different leg
    Doctor->>Policy: walk(leg files)
    Policy-->>Doctor: questions and misplaced paths
    Doctor->>Record: answered_questions
    Record-->>Doctor: recorded leg and classified_as
    Doctor-->>Doctor: settle recorded overrides
    Doctor-->>Execute: COMPLIANT when no findings remain

Loading

Flow diagram for empty gitmodules and recorded path handling

flowchart TD
    Start[Doctor audits leg] --> Gitmodules{.gitmodules is a regular zero-byte file?}
    Gitmodules -->|Yes| Ignore[Remove from questions]
    Gitmodules -->|No| Questions[Keep normal submodule question]
    Ignore --> Paths[Evaluate policy questions and misplaced paths]
    Questions --> Paths
    Paths --> Record{answered_paths row covers every file?}
    Record -->|Yes, row names this leg| Settled[Report answered path]
    Record -->|No| Finding[Keep question or MISPLACED finding]
    Settled --> Diagnostics[Emit answer clauses]
    Finding --> Diagnostics
    Diagnostics --> Report[Include empty_gitmodules, answered_paths, or unslashed details when applicable]

Loading

File-Level Changes

Change	Details	Files
Record and honor plan decisions that place paths contrary to policy rules.	Extend adoption records to include entries kept in a leg different from the policy walk, while excluding rule-consistent, root, drop, and .gitmodules cases. Have the doctor apply recorded answers only within the named leg and file-by-file, preserving the original rule classification and resolution in reporting. Add coverage-based lookup for recorded answers to avoid the previous path-by-file quadratic scan.	adopt-project.py shape-doctor.py tests/test_doctor_recorded_answers.py tests/test_doctor_plan_overrides.py
Treat empty .gitmodules files as having no submodule question.	Ignore regular zero-byte .gitmodules files when collecting review questions, while continuing to ask about blank-line, comment, symlink, and non-regular cases. Report omitted empty files explicitly in the leg detail and doctor reason.	shape-doctor.py tests/test_doctor_plan_overrides.py
Diagnose malformed or broad recorded-path entries without changing their effective coverage semantics.	Report slashless rows that name directories as non-answering entries without making them findings. Document that directory rows cover files added after adoption and do not constrain source_commit. Add corresponding manifest comments and validator documentation.	shape-doctor.py adopt-project.py templates/assembly-root/scripts/validate-manifest.py tests/test_doctor_plan_overrides.py
Update project guidance and regression coverage for the expanded record behavior.	Clarify answered_paths semantics in AGENTS.md. Add unit and end-to-end tests covering rule overrides, empty .gitmodules, slashless rows, later-added files, and documentation.	AGENTS.md tests/test_doctor_plan_overrides.py tests/test_doctor_recorded_answers.py

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#655	Record plan answers that place a path in a leg different from the policy’s rule, and have the doctor honor that record so the path is not reported as misplaced.	✅
#655	Stop the doctor from repeatedly asking about a tracked, zero-byte .gitmodules file left after removing a leg’s only submodule.	✅
#655	Make answered_paths: directory-row behavior clear: identify slashless rows that name an existing directory but answer nothing beneath it, and explain that a directory row also answers files added later.	✅

Possibly linked issues

• Tracking: adopt-project.py, converting an existing repository in place #457: The PR implements the requested fixes for L1-56 and L1-57 and documents riders L1-58 and L1-59, so it directly resolves this issue’s specified work.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 4 hours and 25 minutes by commenting @sourcery-ai review. Upgrade to get a review now.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud