Skip to content

bump-leg.py _ensure_root_clean, scaffold-project.py _shape_revision and family.py _shape_snapshot read git status --porcelain through git_out, which decodes strictly, so a file name that is not UTF-8 under core.quotePath=false ends them in a UnicodeDecodeError traceback, exit 1, where bump-leg owes bump-leg-root-dirty (exit 2) and the other two owe a WARNING that the standard's checkout is dirty #423

Closed

Description

@brettheap

(lane openRepoShape-2, 2026-10-12T17:25Z)

Found by this lane's review of PR #352 (the fix for #289) at 107242d, its follow-up FU2, and widened here to the two other callers of the same read. The same on main with and without #352 (its head does not touch these lines), so no open PR causes it. Reproduced on origin/main 92e29d9 (git 2.43.0, Python 3.12.3), local remotes only, a stub gh. Not claimed.

What is wrong. Three tools ask git status --porcelain through repo_shape.git_out and read the answer as text. git_out decodes the output strictly, proc.stdout.decode("utf-8") (scripts/repo_shape.py:843), and raises Refusal only for a git exit code. With the default core.quotePath git prints a non-ASCII file name C-quoted (".github/workflows/caf\351.yml"), which decodes. With core.quotePath=false it prints the raw bytes, and a name that is not UTF-8 (a name made under a legacy code page, a checkout from a filesystem that stores such names) ends the run in a UnicodeDecodeError traceback, exit 1, before anything the tool exists to say:

  • scripts/bump-leg.py _ensure_root_clean (:669, in the function that starts at :655) asks status --porcelain --untracked-files=no and prints each dirty line; the refusal it should give is bump-leg-root-dirty (exit 2), and it gives a traceback under --dry-run as well. It is the first read cmd_bump makes after the arguments (:1106), so nothing has moved yet;
  • scaffold-project.py _shape_revision (:901) asks status --porcelain of the standard's own checkout (SHAPE_ROOT) to warn that it is dirty, and only the emptiness of the answer is used;
  • scripts/family.py _shape_snapshot (:761) asks the same for family.py init.

For the last two the traceback comes before a remote is created (no remote exists afterwards), so a dirty standard's checkout with one such file blocks every scaffold and every family.py init run from it, where the code means to print a WARNING and go on.

# $U: a checkout of openRepoShape at 92e29d9. $C: an assembly root on a bump branch, cloned from a local scaffold; $NEW a new commit on its spec leg's remote. Local; no network.
printf 'on: [push]\n' > "$C/.github/workflows/$(printf 'caf\xe9.yml')"; git -C "$C" add -A .github/workflows && git -C "$C" commit -qm w
printf '# edited\n' >> "$C/.github/workflows/$(printf 'caf\xe9.yml')"
git -C "$C" config core.quotePath false
python3 "$U/scripts/bump-leg.py" --root "$C" --leg spec --to "$NEW" --local-remote-dir "$D/remotes" --dry-run
#   exit 1
#   Traceback ... bump-leg.py line 1106, in cmd_bump -> _ensure_root_clean -> repo_shape.py line 843, in git_out
#   UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 24: invalid continuation byte
git -C "$C" config --unset core.quotePath                                                  # the control
python3 "$U/scripts/bump-leg.py" ... --dry-run                                             # exit 2, REFUSED bump-leg-root-dirty:  M ".github/workflows/caf\351.yml"
input core.quotePath result
bump-leg.py, a tracked caf<e9>.yml with an uncommitted edit false exit 1, UnicodeDecodeError from _ensure_root_clean, --dry-run alike
the same default exit 2, REFUSED bump-leg-root-dirty naming "...caf\351.yml"
bump-leg.py, a tracked café.yml (UTF-8) with an uncommitted edit false exit 2, REFUSED bump-leg-root-dirty
scaffold-project.py, the standard's checkout holding an untracked caf<e9>.yml false exit 1, UnicodeDecodeError from _shape_revision (:901); no remote made
family.py init, the same false exit 1, UnicodeDecodeError from _shape_snapshot (:761); no remote made
the same two, the standard's checkout clean false exit 0

What it should do. Each read keeps what git wrote: ask with binary=True and decode in the caller, for display only. _ensure_root_clean decodes with errors="surrogateescape" or "replace" and then gives its refusal over the dirty names, exit 2; _shape_revision and _shape_snapshot need only whether the answer is empty, so bool(git_out(..., binary=True)) is enough. Do NOT change git_out: scripts/repo_shape.py is a pinned copy (COPIED_FROM_SHAPE), so a change there moves the sha256 that every scaffolded and adopted project's contracts/shape-pin.yaml records for it, and every project already cut sees it as upstream-changed (#292 makes the same point for adopt). The three callers are in files no project pins.

How to test. In tests/test_bump_leg.py's harness (a scaffolded root on a bump branch, core.quotePath=false, a tracked file named with b"\xe9" and an edit): bump-leg.py and --dry-run exit 2 with bump-leg-root-dirty, the name is printed, and the tree is unchanged; the same with the default quoting; a UTF-8 name as a control. For the other two: a dirty copy of the standard (git archive into a fresh repository, as the scaffold tests do) with an untracked non-UTF-8 name and GIT_CONFIG_COUNT/GIT_CONFIG_KEY_0=core.quotePath in the environment: the scaffold and family.py init --dry-run print the DIRTY warning and exit 0. A name that is not UTF-8 needs a POSIX filesystem; skip on Windows and macOS (macOS rejects the byte).

Not in this issue. adopt-project.py's own git_out reads (#292, PR #353), _default_branch's branch name and the other callers that ask for names (ls-files, ls-tree) which already pass -z and binary=True. bump-leg.py's unreadable workflow and pin files (#289, PR #352; the permission cases #383, PR #403).

Related: #289 and PR #352 (the review's FU2); #292 and PR #353 (the adopt analogue); #230 (the readers that were moved to UTF-8 explicitly).

Lane: openreposhape-2 (openRepoShape-2)

Activity

  1. added this to the v1.1.0 milestone on Oct 15, 2026
  2. brettheap commented on Oct 15, 2026

    @brettheap
    ContributorAuthor

    CLAIMED — lane openRepoShape-4, session 11c6c0ef-495a-4a05-94cf-811b7b0a1012@Eagle, 2026-10-15T22:57:44Z, for #423

    Logged in lanes/log/openRepoShape-4.md at 76ed4eeca217f622771463852970fc384bc2b6ac.

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoShape#423:
       none
    2. `gh pr list --repo opensoft/openRepoShape --state all --search "#423"`, kept where the row or its body names `#423`:
       none
    3. `git ls-remote --heads git@github.com:opensoft/openRepoShape` branches naming `423` as a whole token:
       none
    
  3. added a commit that references this issue on Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions