scripts/repo_shape.py treats a quote character in the MIDDLE of a plain scalar as opening a quote, so k: [it's, a] reads ["it's, a"] where its block twin reads ["it's", "a"], k: it's fine # note keeps the comment in the value, and #244 turns k: [a"b\", c] from right (by accident) into one item #249

Lane: openreposhape-5 (openRepoShape-5)

Found by this lane while fixing #213: the writer listed it as follow-up 1 and the adversarial review as N2, and PR #244's body leaves it out ("Left out") for a follow-up issue. Pre-existing on main 54aa69a. Latent: nothing this repository writes holds a quote character in a plain scalar (see Reach), so this is a reader that disagrees with itself, not a live failure. One input in the class is NOT unchanged by #244 (fix/yaml-flow-escaped-quote, head 86ffe66): k: [a"b\", c] reads right on main, by accident, and is one item on that branch.

What is wrong. The YAML subset reader opens a quote at ANY " or ' it meets, where YAML opens one only at the START of a scalar. So a quote character in the middle of a PLAIN scalar (it's, a"b) swallows what follows it to the end of the text, and the next real separator, comma or # is read as text inside a quote that was never opened. Four sites do it, each its own toggle: _strip_comment (scripts/repo_shape.py:206), _split_key (:229), _split_key_flow (:330) and _split_flow (:347). A flow sequence, a flow mapping, a block scalar's trailing comment and a key are all affected, and the block form of the same list reads right, so the same text reads two ways.

SHAPE=<openRepoShape checkout>
SHAPE=$SHAPE python3 - <<'PY'
import os, sys
sys.path.insert(0, os.environ["SHAPE"] + "/scripts")
from repo_shape import parse_yaml, YamlError
def read(text):
    try:
        return parse_yaml(text)
    except YamlError as exc:
        return f"YamlError: {exc}"
# (document, what PyYAML 6.0.3 and the block form read)
rows = [
    ("k: [it's, a]\n",         {"k": ["it's", "a"]}),
    ("k: {a: it's, b: 1}\n",   {"k": {"a": "it's", "b": 1}}),
    ("k: it's fine # note\n",  {"k": "it's fine"}),
    ("k: [it's, a] # note\n",  {"k": ["it's", "a"]}),
    ("it's: 1\n",              {"it's": 1}),
    ('k: [a"b, c]\n',          {"k": ['a"b', "c"]}),
    ('k: [a"b\\", c]\n',       {"k": ['a"b\\"', "c"]}),
]
for text, right in rows:
    got = read(text)
    print(f"{text!r:26} -> {got!r}" + ("" if got == right else f"\n{'right':26} -> {right!r}"))
print("block twin of the first :", read("k:\n  - it's\n  - a\n"))
PY

Observed on main 54aa69a (a row with no right line under it is read right):

"k: [it's, a]\n"           -> {'k': ["it's, a"]}
right                      -> {'k': ["it's", 'a']}
"k: {a: it's, b: 1}\n"     -> {'k': {'a': "it's, b: 1"}}
right                      -> {'k': {'a': "it's", 'b': 1}}
"k: it's fine # note\n"    -> {'k': "it's fine # note"}
right                      -> {'k': "it's fine"}
"k: [it's, a] # note\n"    -> 'YamlError: not a flow collection: "[it\'s, a] # note"'
right                      -> {'k': ["it's", 'a']}
"it's: 1\n"                -> 'YamlError: line 1: expected `key: value`, got "it\'s: 1"'
right                      -> {"it's": 1}
'k: [a"b, c]\n'            -> {'k': ['a"b, c']}
right                      -> {'k': ['a"b', 'c']}
'k: [a"b\\", c]\n'         -> {'k': ['a"b\\"', 'c']}
block twin of the first : {'k': ["it's", 'a']}

Observed on fix/yaml-flow-escaped-quote 86ffe66 (PR #244): every row above reads as on main except the last, which is now wrong:

'k: [a"b\\", c]\n'         -> {'k': ['a"b\\", c']}
right                      -> {'k': ['a"b\\"', 'c']}

Row 1 is the report: two items it's and a read as one, it's, a, and nothing refuses it. Row 3 is the quietest: a trailing comment after a value that holds an apostrophe becomes part of the value, and the validator carries on with a wrong string. Rows 4 and 5 are refusals of a valid document. The last row is the regression: on main the \" closed the quote the middle " had opened, so ['a"b\\"', 'c'] was right; #244 takes \" as an escape inside that quote, so the item runs on to the end. The reader's own header says it "raises rather than guessing", and these guess.

How often (from the review of #244, at 8c656f4; #244's 86ffe66 changed only tests): over a raw corpus of 327,156 inputs, 47,239 of them valid YAML, #244 regresses 5 valid documents, all in this class, and fixes 6. Over a corpus built to be dense in this class (50,000 valid documents), 88 are right on main and wrong on #244, 4,574 are wrong on main and right on #244, and 19,934 are wrong on both.

Expected. A quote character opens a quote only at the start of a scalar (after the separator and its whitespace: : , - , [, {, ,), as YAML does; anywhere else it is plain text. Then k: [it's, a] is ["it's", 'a'], k: it's fine # note is it's fine, it's: 1 is {"it's": 1}, and k: [a"b\", c] is ['a"b\\"', 'c'] again, on main and with #244. _strip_comment, _split_key, _split_key_flow (or _split_key, which #244 makes it call) and _split_flow ask the same question, so they take one definition of "at the start of a scalar". A single-quoted item keeps its '' rule. This is a reader change and needs its own differential check: the 14 tracked YAML files read as before, the plan round-trip tests, and a flow-against-block fuzz like the review's (seed 2130213, 100,000 documents, 0 differences on #244). A test pins the rows above, and the last one fails on #244 until this lands.

Reach. Nothing this repository writes holds a quote character in a plain scalar. y() (adopt-project.py:220-233) writes a value bare only when it matches _PLAIN_RE, which has no quote; the manifest template quotes the free-text fields (name, reference, elected_by, templates/assembly-root/project.yaml:32,42,46); --tracking-branch and the other bare fields pass SAFE_ARG_RE, which has none; and the four flow writers write bare words (adopt-project.py pins and allow_empty_legs, scripts/shape_materialize.py also_matches and referent_chain). So it shows only on a file a person edits or another tool writes, an apostrophe in a plain value, most often with a trailing # comment. PyYAML 6.0.3 reads every row above right.

Found by. handoffs/openRepoShape/attachments/lane-openRepoShape-5/ in opensoft/brett-wip: impl213-report.md (candidate follow-up 1) and review213-report.md (N2, with the corpus counts and the input k: [a"b\", c]). The first five rows and the comment rows were run again on main 54aa69a and on 86ffe66 for this issue.

Not claimed.

🤖 Generated with Claude Code