adopt-project.py, shape-doctor.py: a plan's answer against a rule is recorded and an empty .gitmodules asks nothing, so docs/ answered code is not MISPLACED after adoption verified, nor a .gitmodules emptied by git rm asked on every run (#655)

AGENTS.md

@@ -159,7 +159,7 @@ install command). Seven steps, in order, and step 4 is the one you must not skip
 
 `execute` also records the plan's answers in the split's `project.yaml`, under
 `submodules:` (per submodule a leg keeps) and `answered_paths:` (per other path
-the policy asked about). They confer nothing, and no command rewrites them.
+the policy asked about or put elsewhere). They confer nothing; no command rewrites them.
 
 ### Adopting a repository that has NO code yet (or no spec)
 

adopt-project.py

@@ -99,7 +99,8 @@
            in a leg was answered into, under `submodules:`, for
            `shape-doctor.py` to check it against, and under
            `answered_paths:` the leg each entry covering a path the policy
-           asked about was answered into, so the doctor asks it no more.
+           asked about, or put in another leg, was answered into, so the
+           doctor neither asks it again nor calls it misplaced.
 
 THE PLAN IS AN ARTIFACT A HUMAN OR AN AI EDITS. That is why it is YAML with
 reasons in it rather than a pipe between two processes: the classifier is
@@ -4420,6 +4421,16 @@ def _covering(entry_paths: list[str], path: str) -> list[str]:
             if p == path or (p.endswith("/") and path.startswith(p))]
 
 
+def _covering_names(path: str) -> list[str]:
+    """Every name `_covering` can find covering `path`: each directory above
+    it, with its `/`, then `path` itself (#655).
+
+    For a caller holding many entries, who looks each of these up rather
+    than holding every entry against every path."""
+    return [path[:index + 1] for index, char in enumerate(path)
+            if char == "/"] + [path]
+
+
 def _coverage_findings(entry_paths: list[str], tree_paths: list[str]) -> list[str]:
     """Every source path covered EXACTLY once, and every entry covering something.
 
@@ -10180,23 +10191,27 @@ def _where_it_landed(assembly: Path, written: str) -> str | None:
 )
 
 #: The key `execute` appends after `submodules:` (#525): the plan's answer
-#: for each path the policy itself asked about. Read by the doctor by this
-#: name through the module it loads, and spelt once more by the project's
-#: own `validate-manifest.py`, as `SUBMODULE_RECORD` is.
+#: for each path the policy itself asked about, or put in another leg than
+#: the plan's (#655). Read by the doctor by this name through the module it
+#: loads, and spelt once more by the project's own `validate-manifest.py`,
+#: as `SUBMODULE_RECORD` is.
 ANSWER_RECORD = "answered_paths"
 
 #: The comment above that block, for the same reader.
 ANSWER_RECORD_PREAMBLE = (
     "",
-    "# THE PATHS THE ADOPTION'S POLICY ASKED ABOUT, and the leg each was",
-    "# ANSWERED into (#525). Written once, by `adopt-project.py execute`,",
-    "# from the adoption plan: one row per entry kept in a leg that covers a",
-    "# path `contracts/path-classification.yaml` could not call, its path",
-    "# inside that leg (a directory's with its `/`), the `resolution:` the",
-    "# plan gave it, and the source commit the plan was answered at.",
-    "# `shape-doctor.py` takes each row as the answer to the question it",
-    "# would ask again of that path in that leg. Like every field here, it",
-    "# confers nothing.",
+    "# THE PATHS THE ADOPTION'S POLICY ASKED ABOUT OR PUT IN ANOTHER LEG,",
+    "# and the leg each was ANSWERED into (#525, #655). Written once, by",
+    "# `adopt-project.py execute`, from the adoption plan: one row per entry",
+    "# kept in a leg that covers a path `contracts/path-classification.yaml`",
+    "# could not call, or called for the other leg or the root, its path",
+    "# inside that leg (a directory's with its `/`: without it, a row names",
+    "# a file), the `resolution:` the plan gave it, and the source commit the",
+    "# plan was answered at. `shape-doctor.py` takes each row as the answer",
+    "# for that path in that leg: not asked again, and not misplaced. A",
+    "# directory's row answers for every file under it, one added after the",
+    "# adoption too: the doctor does not hold a file to `source_commit:`.",
+    "# Like every field here, it confers nothing.",
     f"{ANSWER_RECORD}:",
 )
 
@@ -10228,8 +10243,8 @@ def _record_block(preamble: tuple, rows: list, entries: list,
 def answered_path_record(policy: PathPolicy, entries: list, tree: list,
                          commit: str) -> str:
     """The `answered_paths:` block `execute` appends to `project.yaml` after
-    `submodules:`, or "" when the plan answered no question of the policy's
-    into a leg (#525).
+    `submodules:`, or "" when the plan kept no path in a leg the policy did
+    not put it in (#525, #655).
 
     WHY THE ANSWER IS WRITTEN DOWN. A path the policy cannot call --
     `examples/` under `ambiguous-examples`, a directory whose extensions do
@@ -10238,37 +10253,73 @@ def answered_path_record(policy: PathPolicy, entries: list, tree: list,
     each leg after the split and, with no record, asks the same question
     again on every run, of a path somebody already answered.
 
-    ONLY THE POLICY'S OWN QUESTIONS, RE-ASKED HERE. The policy's `walk` is
-    run again over the source's paths at the plan's commit, gitlinks left
-    out as the doctor leaves them out, and a row is written for each entry
-    the plan answers `spec` or `code` (read as `_answered_legs` reads it)
-    that covers a path the walk leaves `leg: null`. Not from the plan's own
-    `review_required:`, which is a file a person edits: an entry they split
-    by hand to answer one question two ways is still recorded, each half in
-    its own leg. An entry answered `root` or `drop` is in no leg, and the
-    doctor never asks of it. `.gitmodules` is left out: its question is
-    which leg owns the submodules it registers, and `submodules:` answers
-    that per submodule, which is the reading the doctor holds a leg's
-    `.gitmodules` to. And a question `plan` asks for a reason of its own --
-    a gitlink under the entry, a name `execute` will not extract -- is not
-    the policy's, and is recorded only where the policy asks one under it.
+    AND AN ANSWER AGAINST A RULE, FOR THE SAME REASON (#655). A path a rule
+    calls -- `docs/` under `spec-governance` -- may be answered into the
+    other leg in the plan, with a `resolution:` saying why: `check` passes
+    it and `execute` carries it out, because the plan is the person's word.
+    With no record the doctor held the leg to the rule, and its first run
+    after `adoption verified` called that path MISPLACED.
+
+    THE POLICY'S OWN VERDICTS, READ AGAIN HERE. The policy's `walk` is run
+    again over the source's paths at the plan's commit, gitlinks left out
+    as the doctor leaves them out, and a row is written for each entry the
+    plan answers `spec` or `code` (read as `_answered_legs` reads it) that
+    covers a path the walk does not put in that leg: one it leaves `leg:
+    null`, or one it calls for the other leg or the root. An entry
+    answered as its rule says needs no record: the doctor's walk agrees
+    with it. Not from the plan's own `review_required:`, which is a file a
+    person edits: an entry they split by hand to answer one question two
+    ways is still recorded, each half in its own leg, and one split to
+    answer half of it against a rule is recorded for that half alone. An
+    entry answered `root` or `drop` is in no leg, and the doctor never
+    reads it. `.gitmodules` is left out: its question is which leg owns
+    the submodules it registers, and `submodules:` answers that per
+    submodule, which is the reading the doctor holds a leg's `.gitmodules`
+    to. And a question `plan` asks for a reason of its own -- a gitlink
+    under the entry, a name `execute` will not extract -- is not the
+    policy's, and is recorded only where the policy's walk disagrees with
+    the answer under it.
+
+    AGAINST A RULE MEANS AGAINST WHAT `plan` SHOWED AS WELL (#655). `plan`
+    walks the tree WITH its gitlinks, and a gitlink is one more verdict in
+    its directory's extension majority, so it can keep `plan` from folding
+    a directory the walk above folds: `plan` then proposes each file by its
+    own extension. An entry left as `plan` wrote it was answered against
+    nothing, so an answer against a rule is recorded only where it also
+    differs from the leg `plan` showed: that walk, with its entries asked
+    again by `ask_about_submodules` and `ask_about_refused_paths` as `plan`
+    asks them. A question of the walk above is recorded as before.
 
     A SOURCE WITH NO SUCH ENTRY GETS "" and so the `project.yaml` it always
     got, byte for byte.
     """
     gitlinks = {path for path, mode, _, _ in tree if mode == GITLINK_MODE}
     files = [(path, size) for path, mode, _, size in tree
              if path not in gitlinks]
-    asked = [entry.path for entry in walk(policy, files) if entry.leg is None]
-    questions = [path for path, _ in files if _covering(asked, path)]
-    answered = _answered_legs(entries, questions)
-    kept = sorted({entry: leg for leg, entry in answered.values()
-                   if leg in EXTRACTED_LEGS and entry != GITMODULES}.items())
+    called = {entry.path: entry.leg for entry in walk(policy, files)}
+    shown = {entry.path: entry.leg for entry in ask_about_refused_paths(
+        ask_about_submodules(policy, walk(
+            policy, [(path, size) for path, _, _, size in tree]), tree),
+        tree)}
+    answered = _answered_legs(entries, [path for path, _ in files])
+    kept = sorted({entry: leg for path, (leg, entry) in answered.items()
+                   if leg in EXTRACTED_LEGS and entry != GITMODULES
+                   and leg != _walked_leg(called, path)
+                   and (_walked_leg(called, path) is None
+                        or leg != _walked_leg(shown, path))}.items())
     return _record_block(ANSWER_RECORD_PREAMBLE,
                          [(entry, leg, entry) for entry, leg in kept],
                          entries, commit)
 
 
+def _walked_leg(called: dict, path: str) -> str | None:
+    """The leg `walk` gave `path`, from `called` (`{entry path: leg}`): its
+    own entry's, or the folded directory's above it; None for a question.
+    The walk's entries cover each path it was given exactly once (#655)."""
+    return next((called[name] for name in _covering_names(path)
+                 if name in called), None)
+
+
 def submodule_record(entries: list, tree: list, commit: str) -> str:
     """The `submodules:` block `execute` appends to `project.yaml`, or ""
     when the split keeps no submodule of the source's in a leg (#189).
@@ -10401,8 +10452,9 @@ def _commit_the_split(plan: Plan, source: Source, assembly: Path,
                                   source.commit)
         if record:
             append["project.yaml"] = record
-        # And the plan's answers to the policy's own questions, after it
-        # (#525): `answered_path_record`, read before any leg existed.
+        # And the plan's answers to the policy's own questions, and against
+        # its rules, after it (#525, #655): `answered_path_record`, read
+        # before any leg existed.
         if answers:
             append[SHAPE_MANIFEST] = append.get(SHAPE_MANIFEST, "") + answers
         # Each declared pin's file, `contracts/<product>-pin.yaml`, is
@@ -10510,10 +10562,11 @@ def cmd_execute(args) -> int:
                               code_path, tracking, {}, {},
                               shape_commit, pins, naming)
 
-    # The plan's answers to the policy's own questions (#525), worked out
-    # here, before anything exists: the policy is this standard's file, the
-    # one the doctor will ask, and one that does not load refuses now and
-    # not after both legs are pushed. "" for a plan that answered none.
+    # The plan's answers to the policy's own questions (#525), and against
+    # its rules (#655), worked out here, before anything exists: the policy
+    # is this standard's file, the one the doctor will ask, and one that
+    # does not load refuses now and not after both legs are pushed. "" for
+    # a plan that answered none.
     answers = answered_path_record(PathPolicy.load(PATH_POLICY),
                                    plan.entries, source.tree(), source.commit)