templates/assembly-root/scripts/validate-pins.py: a workflow file under .github/workflows/ that is not UTF-8 ends the run in a UnicodeDecodeError traceback, exit 1, with no refusal naming the file, so the project's pins gate (make validate, the validate workflow) dies on a Latin-1 comment where a non-UTF-8 pin is a clean REFUSED yaml-unreadable, exit 2 #359

(lane openRepoShape-2, 2026-10-12T13:01:30Z)

Found by the writer of lane-2 PR #352 (the fix for #289), which left it out on purpose: the file is a pinned template copy, and #289's "Not in this issue" names it. Reproduced again on origin/main a1b3a4b, where #352 is not merged. Not claimed.

What is wrong. _workflow_refs (templates/assembly-root/scripts/validate-pins.py:102-113) reads every .yml and .yaml file under .github/workflows/ with path.read_text(encoding="utf-8") (:111), and nothing on the path catches a UnicodeDecodeError. _run_all_checks calls it (:609) inside main()'s one try (:652-657), whose only handler is except Refusal (:656). UnicodeDecodeError is a ValueError, so a committed workflow file that is not UTF-8 (a Latin-1 # café comment, a UTF-16 file with a byte-order mark) ends the run in a Python traceback, exit 1, which is the code this validator uses for a finding, and no line names the file. The same script refuses a non-UTF-8 pin by name, because the pin goes through load_yaml (scripts/repo_shape.py:798-823), which turns the decode error into Refusal("yaml-unreadable", "<path>: not UTF-8 at byte N (...)", remediation) (#192). The workflow read goes around load_yaml.

This is the project's own gate: make validate runs the script (templates/assembly-root/Makefile:49,52), and so does the pins step of the validate workflow (templates/assembly-root/.github/workflows/validate.yml:200). A project whose workflow file gains one Latin-1 byte gets a red check and a traceback to read in place of a finding.

# $U: a checkout of openRepoShape at a1b3a4b. Every repository is local (--local-remote-dir); no network.
python3 "$U/scaffold-project.py" --org testorg --project Northwind --elected-by 'T H' --elected-on 2026-09-02 \
        --local-remote-dir "$R/remotes" --work-dir "$R/work"
git -c protocol.file.allow=always clone -q --recurse-submodules "$R/remotes/Northwind.git" "$C"
cd "$C"; SPEC=$(git rev-parse HEAD:spec)
printf 'on: [pull_request]\njobs:\n  s:\n    uses: testorg/Northwind-spec/.github/workflows/x.yml@%s\n' "$SPEC" > .github/workflows/legs.yml
git add .github/workflows/legs.yml && git commit -qm workflow
printf '# caf\xe9\n' >> .github/workflows/legs.yml && git commit -qam 'latin-1 comment'
python3 scripts/validate-pins.py --root "$C"
#   exit 1
#   Traceback: validate-pins.py line 655, in main / line 609, in _run_all_checks / line 111, in _workflow_refs
#   UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 133: invalid continuation byte

The scaffolded scripts/validate-pins.py is byte for byte the template (cmp against templates/assembly-root/scripts/validate-pins.py: identical), so the traceback is the project's own.

Why. _workflow_refs reads the file before it knows whether it names a leg (the UTF-16 row names none and still stops the run), and its read is the only one in the script that does not go through load_yaml. Nothing between it and main() converts the exception, and main() handles Refusal only.

What it should do.

1. Refuse by name, exit 2, in the words load_yaml uses: a refusal (for example workflow-unreadable) naming the file by its path relative to the root, the byte offset and the reason (.github/workflows/legs.yml: not UTF-8 at byte 133 (invalid continuation byte)), with a remediation that says to re-save the file as UTF-8 (a UTF-16 file needs converting), commit it, and run the same command again. Naming every such file in one refusal is better than naming the first (it is what bump-leg.py: a workflow file or a leg pin that is not UTF-8 is refused by name before anything moves, so a Latin-1 comment no longer ends the run in a UnicodeDecodeError traceback (#289) #352 does in bump-leg.py). The same refusal covers a file this user cannot open (the last row), which is the same line.
2. Refuse, do not skip. A skipped workflow could hide a @<sha> reference that is out of lockstep, which is what this gate exists to find. A lossy read (errors="replace") writes nothing, so it would be harmless to bytes, but it would call green a tree that the project's other YAML readers refuse and that bump-leg.py refuses once bump-leg.py: a workflow file or a leg pin that is not UTF-8 is refused by name before anything moves, so a Latin-1 comment no longer ends the run in a UnicodeDecodeError traceback (#289) #352 lands; refusing keeps the three readers saying one thing.
3. Mind where the fix goes. The file is a pinned template copy: contracts/shape-pin.yaml records its sha256 (the files: row scripts/validate-pins.py, and verify_pin: scripts/validate-pins.py), so a fix moves that digest and every project already cut sees the copy as upstream-changed until it runs update-shape.py apply. That is the ordinary cost of changing a pinned copy (bump-leg.py: a committed workflow file under .github/workflows/ or a contracts/<role>-pin.yaml that is not UTF-8 ends in a UnicodeDecodeError traceback, exit 1, with no refusal naming the file, so the operator reads Python's decode error and --dry-run dies the same way #289 records it), and the reason this is its own change.

How to test. Beside the tests that run a scaffolded scripts/validate-pins.py (tests/test_neutral_pin_validator.py:31 runs it in a scaffolded root; a new tests/test_validate_pins_unreadable_workflow.py is as good), with local bare remotes and the workflow written with write_bytes, so the test is portable to Windows:

• (a) a committed workflow holding b"# caf\xe9\n": the run exits 2, stderr holds the refusal id and the file's relative path and the byte offset, no Traceback;
• (b) a UTF-16 workflow with a byte-order mark that names no leg: the same;
• (c) two non-UTF-8 workflow files: both are named;
• (d) a committed contracts/spec-pin.yaml with a Latin-1 byte still ends REFUSED yaml-unreadable (control);
• (e) a UTF-8 workflow with a non-ASCII comment still gives pins ok, exit 0 (control).
  For the unreadable-file row, chmod 000 the file and skip the test on Windows and when os.geteuid() == 0 (tests/test_family_add_refusals.py:677 is the precedent), restoring the mode in a finally. Reverting the guard fails (a) to (c).

Related: #289 and PR #352 (the same non-UTF-8 workflow file in bump-leg.py); #192 (closed: the same crash in load_yaml, fixed there); #295 and PR #312 (other defects in this script, not about encoding).

Lane: openreposhape-2 (openRepoShape-2)