bootstrap.py (the assembly root's) and repo_shape.py: a .shape-upstream-tip or a YAML file saved with a byte-order mark reads as no commit, or as a first key spelled with U+FEFF, so make bootstrap's upstream notice stays silent #855

Goal: a file the tools read as UTF-8 text, which a Windows editor saved with a byte-order mark, is read as the text it holds, so the upstream notice names the commit in .shape-upstream-tip and the one in contracts/shape-pin.yaml; or the tools say that they read such a file as no answer.

Filed by lane openRepoShape-5, from rv815's NIT 8 on #764's PR (#815, "Checklist candidates (not filed)": _cached_tip reads a cache that starts with a byte-order mark as no answer; measured by the reviewer), and, for the second line, from a check made while placing that candidate (it is not one of the review candidates; drop it if it is better filed on its own). Both lines are measured on main fc867207, by loading the shipped scripts and calling them, as the lines say.

Pinned copies. templates/assembly-root/scripts/bootstrap.py is pinned in every project, and scripts/repo_shape.py in every project and holder (REPO_SHAPE, scripts/shape_materialize.py:417). Each edit reaches a root as an upstream-changed row on its next update-shape.py check. Bundle the first line with #764's (PR #815) and #710's (PR #824) edits to the same bootstrap.py, and the second with the other open edits to repo_shape.py (#821, #823, #827), so each root takes one row per file.

Lands after: lane 5's #764 (PR #815, fix/764-bootstrap-notice-exit-texts, head 05a1ebb3), which moves the cache read into _cached_tip (templates/assembly-root/scripts/bootstrap.py:959-970 at that head) and makes an unreadable or non-UTF-8 cache "no answer". On main the same read is inline in shape_upstream_notice (templates/assembly-root/scripts/bootstrap.py:885) and reads the same.

Lines (path:line at main fc867207 unless a line says 05a1ebb3)

• A .shape-upstream-tip that starts with a byte-order mark is read as no commit. Measured: shape_upstream_notice loaded from templates/assembly-root/scripts/bootstrap.py with scripts/repo_shape.py beside it, in a root whose contracts/shape-pin.yaml says commit: and 40 a, and whose .shape-upstream-tip holds 40 b and a line end. It prints shape: the upstream is at bbbbbbbbbbbb and this project is pinned at aaaaaaaaaaaa. Run openRepoShape's .... With the same cache preceded by the bytes EF BB BF, and again with CRLF as the line end, it prints nothing. cache.read_text(encoding="utf-8").strip() (:885) gives '\ufeff' + 40 b, which COMMIT_40 (:105, ^[0-9a-fA-F]{40}$) does not match, so the function returns at :887 (the test at :886). str.strip() does not take U+FEFF off. rv815 measured the same at 05a1ebb3. A person gets the mark from an older Notepad or from Set-Content -Encoding UTF8 in Windows PowerShell 5.1 (read, not run here), and the file is one that, by its own comment, "a nightly job or a human wrote into the root" (bootstrap.py:98-99; the handbook names it at docs/handbook.html:1545). Direction: encoding="utf-8-sig" in the read (_cached_tip once bootstrap.py: a validator's exit 2 ends bootstrap REFUSED, exit 2, an unreadable upstream path, cache or pin is no answer and a reftable leg's cherry-pick or revert is read from its ref store, so no traceback or ended pick follows (#764) #815 lands), which takes a file with or without the mark. A test beside bootstrap.py: a validator's exit 2 ends bootstrap REFUSED, exit 2, an unreadable upstream path, cache or pin is no answer and a reftable leg's cherry-pick or revert is read from its ref store, so no traceback or ended pick follows (#764) #815's tests/test_bootstrap_upstream_notice.py: the cache with the mark, with the mark and CRLF, and without, each says the upstream's tip.
• repo_shape.load_yaml keeps the mark in the first key, or refuses at the first line. Found while placing, and measured on main by calling repo_shape.load_yaml: a file of EF BB BF, commit: and 40 a, a line end and other: 1 gives the keys '\ufeffcommit' and 'other', where the same file without the mark gives 'commit' and 'other'; the notice above, run with such a contracts/shape-pin.yaml and a good cache, prints nothing too (pinned is '' at :873-874). A file that opens with a comment line gives REFUSED yaml-unparsable: <file>: line 1: expected key: value, got '\ufeff# a comment'. load_yaml reads with read_text(encoding="utf-8") (scripts/repo_shape.py:1060), and every file the tools read through it (the manifest, family.yaml, contracts/shape-pin.yaml) meets the mark there. What a validator then says of a first key that reads as another name was not run. Direction: encoding="utf-8-sig" at :1060, which takes the file with or without the mark; a test that reads each of the three shapes above both ways. The tools write their own files without a mark, so no round trip changes. Decide first whether the reader should accept a mark at all, or refuse it by name (yaml-unreadable, saying the file starts with a byte-order mark and how to save it without one): a hand-saved file the notice reads silently wrong is the worse of the two answers.

Verify: tests/test_bootstrap_upstream_notice.py (new in #815), tests/test_bootstrap_without_git.py, tests/test_update_shape.py and tests/test_repo_hygiene.py pass on Linux. The two measurements above, as tests, each failing on main and passing with the change; a mutant for each utf-8-sig put back to utf-8, each caught. Run the test files that import repo_shape.load_yaml for the second line (git grep -l load_yaml tests/). No Windows run: CI is Linux only (#597), and the bytes are written by the test.

Overlaps. #815 and #824 edit templates/assembly-root/scripts/bootstrap.py (#815 the notice, #824 _head_of and the branch reads); #821, #823 and #827 edit scripts/repo_shape.py (#823 the YAML reader's spaces, which is the nearest to the second line: read its head first). Both files are pinned copies, as said above. No README or handbook sentence is made false.

Parent: #458.

Milestone: v1.1.1. Nothing is lost: the notice is optional and silent where it has no answer, and a person who sees no notice is where they were. It is not a regression and blocks nothing in v1.1.0.

🤖 Generated with Claude Code