Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
159 changes: 153 additions & 6 deletions scripts/shape_materialize.py
Original file line number Diff line number Diff line change
Expand Up @@ -1749,7 +1749,7 @@
def value_of(placeholder: str) -> str:
return str(values[placeholder[2:-2]])

if not source.endswith((".yaml", ".yml")):

Check failure on line 1752 in scripts/shape_materialize.py

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Define a constant instead of duplicating this literal ".yaml" 3 times.

See more on https://sonarcloud.io/project/issues?id=opensoft_openRepoShape&issues=AaEyLydPZ1HED5Tt1S4V&open=AaEyLydPZ1HED5Tt1S4V&pullRequest=266
return PLACEHOLDER_RE.sub(lambda match: value_of(match[0]), text)
return YAML_PLACEHOLDER_RE.sub(
lambda match: (double_quoted(value_of(match[1])) if match[1]
Expand Down Expand Up @@ -2715,6 +2715,33 @@
# `check` say what `execute` was always going to say instead of `check`
# passing a plan `execute` then refuses. The alphabet is `checked_value`'s
# own (`SAFE_ARG_RE`): it is asked here, not copied.
#
# LAST, A PATH THE PROJECT MUST READ BACK AS WRITTEN (#225). The assembly
# root's `project.yaml` holds each leg path on a `path:` line, and the leg's
# pin, `contracts/<role>-pin.yaml`, on a `submodule_path:` line. The project
# reads both with its own copy of `repo_shape`'s YAML reader. Written bare
# there, as the templates wrote them when #225 was found, `null`, `~`,
# `true`, `False`, `2026` and `1.5` are read as no value, a boolean or a
# number. Every one of those is in
# the alphabet above, so `--spec-path 2026` scaffolded three repositories
# whose first bootstrap then failed `manifest-leg-path: leg 'spec': path is
# 2026`, and `adopt-project.py execute` said `adoption verified` before the
# same bootstrap failed. Such a value is refused, with no spelling offered:
# there is none that names the same directory.
#
# * THE TEMPLATES ARE READ, AND THE READER IS ASKED. The rule finds every
# YAML template line that holds a leg path, renders the value into it
# with `render`, as the scaffold and `execute` do, and asks the reader
# what it gives back. It keeps no copy of the line and no list of the
# reader's words: a copy goes stale the day a template quotes the
# placeholder (#259), and a list the day the reader learns a word. Where
# every such line is quoted, every value `checked_value` admits reads
# back, and the rule refuses none, with no edit here.
# * `yes`, `0x1F`, `1e3` and `.inf` are text to this reader today (YAML 1.1
# or 1.2 would read some of them otherwise), so they are paths here, and
# a reader that learned one would refuse it on a bare line with no edit.
# * `tests/test_leg_path_yaml_scalars.py` finds the template lines on its
# own and holds the rule's verdict to what each line reads back.

#: `D:\work\Thing` or `D:/work/Thing` — a Windows absolute path, in either
#: spelling. `_spelling_reason` below asks it whether a leg path is a drive
Expand Down Expand Up @@ -2747,9 +2774,11 @@
"names (`.git`, `git~1`, `.gitmodules`, in any case), ends in a dot or a "
"space, or is a Windows device name (`CON`, `PRN`, `AUX`, `NUL`, "
"`COM1`-`COM9`, `LPT1`-`LPT9`, with or without an extension); the whole "
f"value matches {SAFE_ARG_RE.pattern} and does not begin with `-`; and "
"the two legs take different paths, neither inside the other, whatever "
"the case."
f"value matches {SAFE_ARG_RE.pattern} and does not begin with `-`; the "
"project's YAML reader reads it back as the same text from every line "
"the project writes it into (a bare line reads `null`, `~`, `true`, "
"`False`, `2026` and `1.5` as something else); and the two legs take "
"different paths, neither inside the other, whatever the case."
)


Expand Down Expand Up @@ -2806,6 +2835,117 @@
return None


#: The templates the rule below reads its lines from: the shape's own
#: `templates/`, which both tools render a project from (`SHAPE_ROOT /
#: "templates"`, and this file is `SHAPE_ROOT/scripts/`). A test points it
#: at a copy whose lines it has quoted or left bare.
LEG_PATH_TEMPLATES = Path(__file__).resolve().parents[1] / "templates"

#: The placeholders a leg path is rendered into.
LEG_PATH_PLACEHOLDERS = ("SPEC_PATH", "CODE_PATH")

#: What `_leg_path_lines` found under each templates directory it was asked.
_LEG_PATH_LINES: dict[Path, tuple[tuple[str, str, str], ...]] = {}


def _leg_path_lines(templates: Path) -> tuple[tuple[str, str, str], ...]:
"""Every line of every YAML template under `templates` that holds a leg
path, as `(tree, file, line)`: `tree` is the template tree
(`assembly-root`) and `file` the path the line is written to inside it
(`contracts/spec-pin.yaml`).

FOUND, NOT LISTED, the way `tests/test_leg_path_yaml_scalars.py` finds
them, so a template that writes a leg path another way, or in another
file, changes what this rule asks with no edit here. A Markdown
template's `{{SPEC_PATH}}/` is prose no YAML reader reads, so only `.yaml`
and `.yml` files are asked. The shallowest file comes first, so a
sentence names `project.yaml` before the pins. Read once per process.
"""
if templates in _LEG_PATH_LINES:
return _LEG_PATH_LINES[templates]
found = []
for path in sorted(templates.rglob("*"),
key=lambda p: (len(p.parts), p.as_posix())):
if path.suffix not in (".yaml", ".yml") or not path.is_file():
continue
tree, _, file = path.relative_to(templates).as_posix().partition("/")
for line in path.read_text(encoding="utf-8").splitlines():
if any("{{" + key + "}}" in line for key in LEG_PATH_PLACEHOLDERS):
found.append((tree, file, line))
_LEG_PATH_LINES[templates] = tuple(found)
return _LEG_PATH_LINES[templates]


def _read_back(tree: str, file: str, rendered: str):
"""What the project's reader gives back from one rendered `key: value`
line, read on its own: its scalar reads the same there as inside its
block."""
read = parse_yaml(rendered + "\n")
if not (isinstance(read, dict) and len(read) == 1):
raise Refusal("template-leg-path-line",
f"templates/{tree}/{file}: `{rendered}` is not one "
"`key: value` line, which is the one form the leg-path "
"rule reads a leg path back from",
"Remediation: this is a defect in the shape's "
"templates, not in your invocation.")
return next(iter(read.values()))


def _listed(names: list[str]) -> str:
"""`a`, `a and b`, `a, b and c`."""
return (names[0] if len(names) == 1
else ", ".join(names[:-1]) + " and " + names[-1])


def _read_as(value) -> str:
"""What the YAML reader made of a value, in words: `the number 1.5`."""
if value is None:
return "no value at all"
if isinstance(value, bool):
return f"the boolean {str(value).lower()}"
if isinstance(value, (int, float)):
return f"the number {value!r}"
return NOT_TEXT_KINDS.get(type(value), repr(value))


def _misread_reason(text: str) -> str | None:
"""Why the project would read a leg path written as `text` back as
something else, or None when every line it is written into reads back
`text`.

Each template line that holds a leg path is RENDERED with `text` by
`render`, under its file's name, as the scaffold and `execute` render it,
and read back by the reader the project ships (`parse_yaml`, which every
assembly root carries a copy of). A bare line reads `2026` as a number; a
quoted one (#259) reads it back. The sentence quotes each line that
misreads, as rendered. Asked after `checked_value`, whose alphabet holds
no space, quote, backslash, bracket, brace, `#`, `:`, `&`, `*` or `!`, so
each line parses, bare or quoted, and a bare value is a plain scalar.
"""
values = dict.fromkeys(LEG_PATH_PLACEHOLDERS, text)
misread: dict[tuple[str, str], list[str]] = {}
reads = []
for tree, file, line in _leg_path_lines(LEG_PATH_TEMPLATES):
rendered = render(line, values, file).strip()
read = _read_back(tree, file, rendered)
if read != text:
files = misread.setdefault((tree, rendered), [])
if f"`{file}`" not in files:
files.append(f"`{file}`")
reads.append(read)
if not misread:
return None
trees: dict[str, list[str]] = {}
for (tree, rendered), files in misread.items():
trees.setdefault(tree, []).append(
f"as `{rendered}` in {_listed(files)}")
where = "; ".join(
f"the {tree.replace('-', ' ')} writes a leg path {' and '.join(lines)}"
for tree, lines in trees.items())
return (f"{where}, and the project's own YAML reader reads that as "
f"{_read_as(reads[0])}, so its first bootstrap would fail")


def _canonical_spelling(text: str) -> str | None:
"""The canonical spelling of `text`, where one exists, else None.

Expand All @@ -2818,7 +2958,8 @@
path and one with a `..` have no such spelling -- they name another place,
and picking it for the person is the rewrite this refuses -- and neither
has a result this check, or `checked_value`, would itself refuse
(`./.git` is not answered `.git`, nor `-x/` `-x`).
(`./.git` is not answered `.git`, nor `-x/` `-x`, nor `2026/` `2026`
while a template line would read `2026` back as a number).
"""
if text.startswith(("/", "\\")):
return None
Expand All @@ -2830,9 +2971,10 @@
if _unmountable_reason(candidate) is not None:
return None
try:
return checked_value("leg path", candidate)
checked = checked_value("leg path", candidate)
except Refusal:
return None
return None if _misread_reason(checked) else checked


def leg_path_problem(what: str, value) -> str | None:
Expand All @@ -2842,7 +2984,8 @@

Asked in this order: the SPELLING (canonical or not), then the NAMES Git
and Windows keep, then the alphabet `checked_value` allows on a command
line, so that `plan`, `check` and `execute` refuse the same values.
line, then whether the project reads it back as the text it is (#225),
so that `plan`, `check` and `execute` refuse the same values.

`what` is how the caller names the value -- `--spec-path`, or
`legs.spec_path` for one read out of a plan -- so the sentence points at
Expand All @@ -2865,6 +3008,10 @@
checked_value(what, value)
except Refusal as exc:
return f"{exc.detail}."
misread = _misread_reason(value)
if misread:
return (f"{what} is {value!r}, which the project would not read back "
f"as text: {misread}.")
Comment on lines +3011 to +3014

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Badge Replace the stale advice to quote non-text leg paths

When adopt-project.py check reads a hand-edited plan containing bare spec_path: 2026 (similarly null, booleans, or floats), the earlier non-string branch tells the operator to quote the value. With this new check, changing it to "2026" immediately produces another plan-bad-leg-path, because the generated templates write it bare and read it back as a number. The prescribed remediation therefore cannot clear the finding; tell the operator to choose a different path, or quote the generated template fields.

Useful? React with 👍 / 👎.

return None


Expand Down
Loading
Loading