Skip to content

shape-doctor.py: every non-ASCII character in a path it prints becomes ?, so a NEXT or keep line naming such a path (a member café, a holder under Projets/Équipe) cannot be pasted as printed #785

Closed

Description

@brettheap

Goal: every command the doctor prints, as a NEXT, a keep line or a WHY, runs as printed when a path it names holds a non-ASCII character. This applies to the table and to --json alike, under any locale.

Filed by lane openRepoShape-3 under v1.1.1, on Brett Heap's word of 2026-10-16T13:35:28Z ("yes create v1.1.1"). Lane 3's reviewer RV-c measured it while reviewing #774 (#732), as SHOULD 1 (#774 (comment)). Main behaves the same for UTF-8 names, so it is not a regression of #774.

What happens (measured on #774's head 311ad7d9 and on main):

  • The holder: a member named café (UTF-8, caf\xc3\xa9) or caf\xe9 (Latin-1), with a commit on its detached HEAD, then deinit -f.
  • The doctor's keep line, in all six runs (each name under a UTF-8 locale, under LC_ALL=C, and under LC_ALL=C PYTHONUTF8=0): git --git-dir='…/.git/modules/members/caf?' branch <name> <commit>, or caf??.
  • Where it shows: the table and --json alike.
  • What it does: that path does not exist, so the pasted line fails.
  • Why: the doctor flattens every non-ASCII character to ?, even é under a UTF-8 locale where the terminal could show it.

What to find and fix:

  • Find where the doctor replaces non-ASCII characters (an ascii/replace encode, or a quoting helper). It may have been put in on purpose, for terminals that cannot print them, or for Windows consoles.
  • Keep a printed command runnable:
    • quote the path with the platform's quoting (quote_arg and the POSIX quoting the doctor already uses for paste lines);
    • give --json the real characters, since JSON can carry them, or \u escapes that decode to them;
    • let the table print them where the output encoding can, falling back to a quoted form a shell turns back into the same bytes (for example POSIX $'\303\251').
  • A path that is not UTF-8 needs bytes-faithful quoting: surrogateescape back to bytes, then $'\xe9'.

Tests:

  • a member and a holder path with a UTF-8 non-ASCII name;
  • the same with a Latin-1 name;
  • each pasted line run as printed, under a UTF-8 locale and under LC_ALL=C;
  • the --json values checked against the real path.

Note: #774 (#732) and #775 (#733) touch the same .gitmodules reading. This issue is about printing, so it can land after them.

Activity

  1. added this to the v1.1.1 milestone on Oct 17, 2026
  2. brettheap commented on Oct 17, 2026

    @brettheap
    ContributorAuthor

    CLAIMED — lane openRepoShape-3, session b30e8b49-f249-4668-adb1-1dbfa93c09e1@Eagle, 2026-10-17T04:17:09Z, for #785

    Logged in lanes/log/openRepoShape-3.md at eb6c35230aba8599e6f26b9ad3ce31afef23106c.

    The three reads (lane-collision-protocol Rule 1):

    1. existing `CLAIMED —` comments on opensoft/openRepoShape#785:
       none
    2. `gh pr list --repo opensoft/openRepoShape --state all --search "#785"`, kept where the row or its body names `#785`:
       none
    3. `git ls-remote --heads git@github.com:opensoft/openRepoShape` branches naming `785` as a whole token:
       none
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions