update-shape.py and bump-leg.py: the byte-order-mark rows (#1050) hold what each rewriter returns but not the tools' own read and write, nor that only U+FEFF is cut, so a utf-8-sig read that drops the mark and a cut that also takes spaces both survive #1176

Goal: the byte-order-mark rows #1050 added hold the bytes each tool writes, through its own read and write, and hold that the cut in front of line 1 takes U+FEFF and nothing else, so each of rv1050's five surviving mutants fails a test.

Filed by lane openRepoShape-2, from rv1050's independent review of #1050 (#933) at 434a07cb (#1050 issuecomment-6289274812), its NITs 1 and 2. Neither held the landing; #1050 landed as cf0bac5a7. Line numbers are main 038dafba9's: update-shape.py and tests/test_bom_rewrites.py are as #1050 left them, and scripts/bump-leg.py's lines have moved by seven since, so the review's :614 and :387 are :621 and :394 here. The mutants were not re-run for this filing.

The tools' own read and write are not held

State. test_a_first_key_after_the_mark_is_rewritten_and_the_mark_kept (tests/test_bom_rewrites.py:117) checks what each rewriter returns (:129-133), for one mark and two. No test goes through _repin (update-shape.py:4954), which reads the shape pin at :4970 and the manifest at :4973, or through _leg_pin_text (scripts/bump-leg.py:599, its read at :621) and then write_lf. By grep, no test calls _repin or _leg_pin_text.

What happens. rv1050's three mutants, read_text(encoding="utf-8-sig") in place of read_text(encoding="utf-8") at update-shape.py:4970, at :4973 and at scripts/bump-leg.py:621, each pass tests/test_bom_rewrites.py and tests/test_bom_reads.py (37 passed). Under each, a file with one mark is written with none, and a file with two marks is written with one. Every reader reads both forms, so nothing reads wrong. But #933's Direction says "the marked first-key file is written, its first bytes are the mark", and today only the rewriters' return values hold that, not the files the tools write. rv1050's probe shows that the tools keep the mark today.

Direction. One row per tool on its real path, for one mark and two: write the marked file as bytes under tmp_path; run _repin(root, PROJECT_KIND, Rollback(root), rows, ...) for update-shape.py, and write_lf(path, rewrite_pin(_leg_pin_text(root, path), ...)) for bump-leg.py; then assert that path.read_bytes() starts with MARK * marks + b"commit:", or b"shape:" for the manifest.

Nothing holds that only U+FEFF is cut

State. _marks_off (update-shape.py:1714) cuts the marks with text.lstrip(BYTE_ORDER_MARK) (:1728), and rewrite_pin (scripts/bump-leg.py:349) cuts them the same way (:394).

What happens. rv1050's mutant at either line, an lstrip that also cuts spaces and line feeds, survives tests/test_bom_rewrites.py, and also tests/test_pin_rewrite_layout.py, tests/test_update_shape_apply_lines_and_headers.py, tests/test_update_shape_header_comments.py and tests/test_bump_leg_dry_run_truth.py (233 passed). Measured under it:

• a leg pin or a shape pin whose line 1 is " commit: ..." is written, exit 0. Without the mutant it is refused as "0 commit line(s)", as it was before update-shape.py and scripts/bump-leg.py: the three line rewriters find a first-line key after a byte-order mark, as every reader does, and write the mark back as it was (#933) #1050, and the reader refuses it too;
• a leg pin indented two spaces on every line is written instead of refused;
• a leading blank line gives the same output either way, so that case is equivalent.

Direction. A row in test_a_mark_after_line_1_keeps_todays_answer (tests/test_bom_rewrites.py:150) with " " + the_file(...), expecting the code and words it gets without the mutant.

• Run each of the five mutants once: it fails a new row, and every row passes without it.

Related: #933, #1050, #458 (the tracker for these tools), #461 (the tracker for tests).

Size: S.

Lane: openreposhape-2 (openRepoShape-2)