validate-manifest.py accepts a display name holding a lone surrogate (name: "a\ud83db") and then raises UnicodeEncodeError printing it, so make validate and shape-doctor.py fail on a traceback instead of a finding while bump-leg.py and update-shape.py apply pass the same project once #254 lands #280

Lane: openreposhape-5 (openRepoShape-5)

Found by this lane's adversarial review of #254's fix (fix/validator-child-encoding at 5757e29) as S1. It was scoped out of that PR because the defect is in the project's own validator, not in the two callers that PR changes. It is pre-existing: the run below is on main 60588d5. #254's change does not touch shape-doctor.py or the validators.

What is wrong. The double-quoted YAML reader turns the escape "a\ud83db" into a string holding a lone surrogate (U+D83D). The project's own scripts/validate-manifest.py accepts that string as the display name. It then raises at its own print(f"manifest ok: {manifest.get('name')} "... (templates/assembly-root/scripts/validate-manifest.py:707), because no UTF-8 stream can hold a lone surrogate under strict or surrogateescape. As a result:

• scripts/validate-manifest.py exits 1 with a traceback instead of a finding, so make validate stops at Makefile:48 (make exits 2). This happens whether stdout is a file or a UTF-8 terminal, under LANG=C.UTF-8 or no locale.
• shape-doctor.py runs the validator with PYTHONIOENCODING=utf-8 (shape-doctor.py:821). It reports the traceback's last line as the manifest FINDING and says INVALID (manifest).
• After bump-leg.py (a real bump) and update-shape.py apply run the project's own validators as children that cannot print its display name on a cp1252 stdout, so a project named 日本 is refused and rolled back as bump-leg-validators-red or update-validators-red under PYTHONIOENCODING=cp1252:surrogateescape, and the refusal blames the validators #254, a real bump-leg.py and update-shape.py apply give the same child PYTHONIOENCODING=utf-8:backslashreplace. On the same project they print manifest ok: a\ud83db (atlas), 3 legs and exit 0. That is the review's run on 5757e29 with Python 3.12.

So three launchers disagree about one project, and none of them names the problem.

cd <an assembly root that passes `make validate`, e.g. one scaffold-project.py made with --local-remote-dir>
python3 - <<'EOF'
import pathlib, re
p = pathlib.Path("project.yaml")
t = p.read_text(encoding="utf-8")
p.write_text(re.sub(r'(?m)^name: .*$', r'name: "a\\ud83db"', t, count=1), encoding="utf-8")
EOF
python3 scripts/validate-manifest.py > out.txt; echo "rc=$?"; tail -1 out.txt
git -c user.name=T -c user.email=t@example.invalid commit -qam rename
python3 <openRepoShape checkout>/shape-doctor.py --root .

Observed on main 60588d5, Python 3.12.3, Linux:

rc=1
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud83d' in position 14: surrogates not allowed

  FINDING  manifest         the project's own scripts/validate-manifest.py reports a finding (exit 1): UnicodeEncodeError: 'utf-8' codec can't encode character '\ud83d' in position 14: surrogates not allowed
INVALID (manifest)   (exit 1)

The same project named Atlas prints manifest ok: Atlas (atlas), 3 legs, and the doctor reports no manifest finding.

Expected. Decide whether a display name may hold a lone surrogate.

• If it may not. Such a name cannot be written to any UTF-8 file, commit message or console except as an escape. validate-manifest.py would refuse it with a named finding, and every launcher would then agree. That is a shape change: the validator is a digest-pinned copy, and the change reaches projects through update-shape.py.
• If it may. validate-manifest.py would print the name with a handler that does not raise. shape-doctor.py would give its child utf-8:backslashreplace, as bump-leg.py (a real bump) and update-shape.py apply run the project's own validators as children that cannot print its display name on a cp1252 stdout, so a project named 日本 is refused and rolled back as bump-leg-validators-red or update-validators-red under PYTHONIOENCODING=cp1252:surrogateescape, and the refusal blames the validators #254 does for bump-leg.py.

Either way, add a test that runs name: "a\ud83db" through make validate, shape-doctor.py, a real bump-leg.py and update-shape.py apply.

Found by. handoffs/openRepoShape/attachments/lane-openRepoShape-5/reviewE-report.md (S1) in opensoft/brett-wip. The run above is new for this issue.

Not claimed.

🤖 Generated with Claude Code