Skip to content

Support multiple space-separated values in display statements - #636

Merged
logbie merged 11 commits into
mainfrom
claude/display-concatenation-bug-4puwvh
Jul 19, 2026
Merged

Support multiple space-separated values in display statements#636
logbie merged 11 commits into
mainfrom
claude/display-concatenation-bug-4puwvh

Conversation

@logbie

@logbie logbie commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

The display statement now accepts multiple space-separated values, folding them into a single left-associative concatenation. This resolves a usability issue where trailing values were silently dropped, and aligns with the natural-language principle that users expect display "label" value to work intuitively.

Changes

Parser (src/parser/stmt/io.rs)

  • Refactored parse_display_statement to:
    • Anchor the statement at the display keyword itself (preserving line/column info)
    • Parse the first value with the full expression parser
    • Loop while the next token is a value-start (string/int/float/bool/nothing literals, identifiers, or (), folding each additional value into a left-associative Concatenation
    • Removed 163 lines of repetitive pattern-matching boilerplate that extracted line/column from every expression variant
  • Added Parser::is_value_start helper (src/parser/helpers.rs) to identify tokens that begin a fresh value, ensuring:
    • Direct index access (display numbers 0) remains a single IndexAccess (the 0 is absorbed by the first expression)
    • Line breaks terminate the statement (Eol is not a value-start)
    • Single values are not wrapped in a Concatenation

Backward Compatibility

  • No breaking changes: Programs that previously worked continue to work identically
  • Silent bugs fixed: Only programs that were already broken (silently dropping trailing values) have their behavior corrected
  • Index access, line breaks, and single-value displays all remain unchanged

Documentation & Examples

  • User guide (Docs/02-getting-started/hello-world.md): Added "Display Several Values at Once" section with examples and a tip comparing space-separated and with forms
  • Test program (TestPrograms/display_multiple_values.wfl): Comprehensive example covering the original report, with equivalence, mixed values, expressions, index access, and conditional blocks
  • Manifest-tracked example (TestPrograms/docs_examples/basic_syntax/display_multiple_01.wfl): 5-layer validated backing the new docs section

Tests

  • Parser unit tests (src/parser/tests.rs): 5 new tests covering:
    • String then variable
    • Variable then string
    • Three-value left-associative fold
    • Index-access regression (ensures display numbers 0 stays a single value)
    • Single-value regression (ensures no unnecessary Concatenation wrapper)
    • Statement boundary test (ensures the following statement parses correctly)

Dev Diary

  • Added Dev diary/2026-07-18-display-multiple-values.md documenting the bug report, root cause, implementation rationale, backward compatibility, spacing semantics, and test coverage

Implementation Details

The fold uses the same semantics as with — values are joined directly with no separator inserted. Spaces come from the quotes: display "I am " age " years old" produces I am 25 years old. This upholds the No-Unlearning Invariant: the space-separated form and the with form are the same form, with nothing to unlearn.

The implementation is conservative: only tokens that clearly begin a value trigger the fold. Operators, keywords, and statement boundaries do not, preserving existing behavior for edge cases and keeping the language predictable.

https://claude.ai/code/session_01KUmoGX1Eyt6H1BcXKg34Vu


Open in Devin Review

Summary by CodeRabbit

  • New Features
    • display now accepts multiple space-separated values in one statement, desugaring like chained with (including keyword-led value forms such as call, count from, not, and file/directory/process/header/list/read constructs) with right-associative folding.
  • Bug Fixes
    • Fixed same-line parsing so trailing values aren’t dropped; statement/line boundaries are respected, and direct index access (e.g., scores 0) remains a single value.
    • Prevents display ... count/read ... patterns from being incorrectly folded.
  • Documentation
    • Added getting-started guidance and executable examples for multi-value display and spacing rules (no added spaces; spaces come from quoted text).
  • Tests
    • Added parser and exact-stdout regression coverage (including display vs with equivalence) plus a new test program.

`display` parsed only the first value and silently dropped any tokens that
followed on the same line, so `display "user age is " user age` printed just
"user age is " with the variable thrown away — silent, partial output with no
error.

`display` now folds additional space-separated values into a single
left-associative concatenation, matching `display a with b with c`. Quoted text
is a string literal; anything else is a variable/expression. Values are joined
directly (no separator), so spaces come from the quotes.

Backward compatible: direct index access (`display numbers 0`) is absorbed by
the first expression before the fold, a line break ends the statement because
`Eol` is not a value start, and single values are unchanged. Only programs that
previously dropped trailing values change behavior.

- Add Parser::is_value_start and rewrite parse_display_statement to fold
  trailing value-start tokens into a Concatenation.
- Tests: parser unit tests (string/var interleavings, three-value fold,
  index-access and single-value regressions, no-leak-into-next-statement) and
  TestPrograms/display_multiple_values.wfl.
- Docs: "Display Several Values at Once" in hello-world.md plus a
  manifest-tracked, 5-layer-validated example; Dev Diary entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KUmoGX1Eyt6H1BcXKg34Vu
Copilot AI review requested due to automatic review settings July 18, 2026 18:25
@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The display parser now accepts multiple space-separated values and folds them into right-associative concatenations. Parser tests, exact-output integration tests, executable examples, manifest metadata, developer notes, and getting-started documentation cover the new behavior and compatibility cases.

Changes

Multi-value display

Layer / File(s) Summary
Display parsing and value classification
src/parser/helpers.rs, src/parser/stmt/io.rs, src/parser/expr/primary.rs, Dev diary/...
Value-start tokens are classified centrally, additional display expressions are folded right-associatively, statement boundaries are guarded, and parser dispatch consistency is debug-asserted.
Parser and runtime regression coverage
src/parser/tests.rs, tests/display_multiple_values_stdout_test.rs, TestPrograms/display_multiple_values.wfl
Tests cover AST structure, keyword-led values, exact output, evaluation order, mutation behavior, compatibility cases, and separation from following statements.
Documentation and executable examples
Docs/02-getting-started/hello-world.md, TestPrograms/docs_examples/..., Dev diary/...
Documentation and examples describe multi-value syntax, grammar boundaries, spacing, and equivalence with with; the example is registered in the manifest.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Suggested reviewers: copilot

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and accurately summarizes the main change: display now supports multiple space-separated values.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/display-concatenation-bug-4puwvh

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aa599932f8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/parser/stmt/io.rs Outdated
Comment on lines +415 to +416
let (cat_line, cat_column) = match self.cursor.peek() {
Some(token) if Self::is_value_start(&token.token) => (token.line, token.column),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Consume keyword-starting display values

When the next display item starts with a valid expression keyword rather than an identifier/literal, this guard stops folding and leaves the tokens as a separate statement. For example, display "number: " call get number prints only number: and then evaluates the action result separately, and display "count is " count inside a count loop fails because count is parsed as a new count statement. Since the new docs describe subsequent items as variables or expressions, include the keyword expression starters that parse_primary_expression accepts (e.g. call, current, contextual count, etc.) here instead of only is_value_start's current literals/identifiers.

Useful? React with 👍 / 👎.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds parser support for display statements to accept multiple space-separated values on the same line by folding them into a left-associative Concatenation, fixing prior behavior where trailing values could be silently ignored. This change sits in the WFL parser and is backed by new parser tests, docs, and example programs.

Changes:

  • Refactors parse_display_statement to parse the first expression and then fold subsequent value-start expressions into Expression::Concatenation.
  • Introduces Parser::is_value_start to conservatively detect when another space-separated value begins.
  • Adds parser unit tests plus docs + manifest-tracked examples and an end-to-end TestPrograms example.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/parser/stmt/io.rs Implements the multi-value display fold and anchors statement position at display.
src/parser/helpers.rs Adds Parser::is_value_start token predicate used by display folding.
src/parser/tests.rs Adds unit tests for concatenation folding and regressions (index access, single value, statement boundary).
Docs/02-getting-started/hello-world.md Documents the new space-separated display form with examples and guidance.
TestPrograms/display_multiple_values.wfl Adds an end-to-end example program covering common and regression scenarios.
TestPrograms/docs_examples/basic_syntax/display_multiple_01.wfl Adds a manifest-tracked docs example for the new docs section.
TestPrograms/docs_examples/_meta/manifest.json Registers the new docs example for validation.
Dev diary/2026-07-18-display-multiple-values.md Records the motivation, root cause, design constraints, and tests/docs added.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread Docs/02-getting-started/hello-world.md Outdated
Comment on lines +140 to +142
You don't have to write `with` between every piece. A `display` can list
several values separated by spaces — quoted text is shown as-is, and anything
else (a variable or an expression) is evaluated first:
Comment thread src/parser/stmt/io.rs Outdated
Comment on lines +395 to +398
// Anchor the statement at the `display` keyword itself.
let (line, column) = self
.bump_sync() // Consume "display"
.map_or((0, 0), |token| (token.line, token.column));
Addresses automated review on #636: the first cut of multi-value `display`
only folded literals, identifiers and `(`, so a value that *starts* with a
keyword was still dropped — or worse. `display "count is " count` in a count
loop failed to parse (`count` reached the statement parser and was treated as a
new count loop), and `display "number: " call get_number` dropped the call.

- Extend `Parser::is_value_start` to also accept `call`, `count`, and `current`
  — the keyword-led primary expressions that are not also binary operators (so
  they reach the fold instead of dangling). Keywords the binary parser already
  consumes after a value (`with`, `find`, `replace`, `split`, `matches`,
  arithmetic) never reach the check; a leading `-` parses as subtraction.
- `parse_display_statement` now `expect`s the `display` token instead of
  defaulting the statement position to a misleading (0, 0); it is only
  dispatched on `display`.
- Docs: describe display values honestly (variable, number, action call, or
  arithmetic expression) instead of promising arbitrary expression forms.
- Tests: parser folds for the `count` variable and a `call` action; E2E example
  gains a folded `call` and a `count`-loop display.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KUmoGX1Eyt6H1BcXKg34Vu
Copilot AI review requested due to automatic review settings July 18, 2026 18:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Comment thread src/parser/helpers.rs
Comment on lines +259 to +273
pub(crate) fn is_value_start(token: &Token) -> bool {
matches!(
token,
Token::StringLiteral(_)
| Token::IntLiteral(_)
| Token::FloatLiteral(_)
| Token::BooleanLiteral(_)
| Token::NothingLiteral
| Token::Identifier(_)
| Token::LeftParen
| Token::KeywordCall
| Token::KeywordCount
| Token::KeywordCurrent
)
}

@logbie logbie left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep-review verdict: do not merge yet. Reviewed at 498a2c3e. The focused parser tests and full CI are green, but the feature still has observable correctness and grammar problems that the current tests do not exercise.

Merge-blocking findings

  1. The space-separated form is not semantically equivalent to with.

    parse_display_statement builds a left-deep concatenation at src/parser/stmt/io.rs:423-429, while explicit with parses its right-hand side recursively at src/parser/expr/binary.rs:346-355. Because concatenation evaluates the left value, then the right value, and only then stringifies them, mutation makes this observable.

    With two identical lists initially containing before and after:

    display left_items "" pop of left_items
    display right_items with "" with pop of right_items
    

    produces:

    [before, after]after
    [before]after
    

    This contradicts the docs' “exactly the same” / No-Unlearning claim. The left fold also repeatedly copies a large leading text value, adding roughly k * L copying for k suffixes and an L-byte prefix. Please either build the same association as with or introduce a flat concatenation representation with deliberately specified evaluation/stringification semantics.

  2. is_value_start is still incomplete.

    The second commit correctly added call, count, and current, but src/parser/helpers.rs:259-272 still omits valid starters accepted by parse_primary_expression, including not, pattern, output, file, directory, process, header, list, read, and contextual-keyword variables.

    Confirmed current-head behavior:

    display "not: " not yes                 // prints only "not: "
    display "files: " list files in "."     // prints only the label
    display "content: " read content from source // parse failure
    

    Please centralize the “can start a primary expression” decision instead of maintaining a display-specific subset.

  3. find / replace / split being consumed by the binary parser does not make them valid subsequent display values.

    display "parts: " split "a,b" by ","
    

    prints [a, b], not parts: [a, b]. The split branch at src/parser/expr/binary.rs:524-549 replaces/discards the preceding left expression. find similarly loses the label; replace can hard-fail. This must be resolved at the operand-boundary/grammar level; adding tokens to is_value_start after the first parse_expression has already consumed them is insufficient.

  4. The advertised whitespace-separator contract conflicts with existing lexical grammar.

    display a b c lexes as one Identifier("a b c") at src/lexer/mod.rs:259-273, not three values, despite the implementation comments, test program, and diary claiming equivalence with display a with b with c. Other ambiguous cases include age 0 (direct index), "list: " [1,2] (postfix index), and "n: " -5 (subtraction).

    Please either choose an unambiguous separator, or narrow the public contract and explicitly require with for ambiguous pairs.

  5. The tests do not verify runtime output.

    The new unit tests inspect AST shape, and the executable examples place expected output only in comments. The CI program runner redirects stdout/stderr to /dev/null and manifest validation checks only exit status. Add interpreter/CLI tests asserting exact stdout, including the mutation regression and prefix-expression cases above.

The second commit did fix the named call/count/current cases and replaced the misleading (0, 0) fallback with an invariant expect; those changes look good.

@claude Please independently validate the current head and report back on all five findings. In particular, exercise multi-value display with WFL user-defined action calls and container values/properties/methods, including mutable lists/containers and side effects. Compare exact stdout between the space-separated and with forms; an exit code of zero is not sufficient. Please run targeted parser plus interpreter/CLI tests, inspect the completed GitHub Actions results, and identify the minimal parser/AST approach that preserves existing action/container behavior and with semantics.

@claude

claude Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Claude finished @logbie's task in 4m 54s —— View job


Independent validation of @logbie's deep-review findings (head 498a2c3e)

Todo list:

  • Read current code: src/parser/stmt/io.rs, src/parser/helpers.rs, src/parser/expr/binary.rs, src/lexer/mod.rs
  • Build release binary — blocked, see note below
  • Finding 1: Reproduce mutation-order divergence between space-separated and with forms
  • Finding 2: Verify is_value_start gaps
  • Finding 3: Verify find/replace/split consuming the preceding label
  • Finding 4: Verify lexer/grammar ambiguity claims
  • Test reasoning for user-defined action calls and container values/mutation (traced through Value representation)
  • Compare exact stdout between space-separated and with forms (via source trace, see caveat)
  • Check GitHub Actions CI results for this PR
  • Post final report with minimal-fix recommendation

⚠️ Tooling limitation (please read first)

cargo build / cargo check were blocked by this session's tool-approval policy (This command requires approval, with no human available to approve it headlessly). I could not execute the binary to get literal stdout. Everything below is a source-level trace through the actual parser/interpreter code paths on this branch, not an empirical run. If you want a live-execution confirmation pass, re-invoke @claude with cargo build/cargo run allowed in --allowedTools and I'll produce actual stdout diffs. CI status was checked via the GitHub API (see Finding 5).


Verdict: agree with @logbiedo not merge yet. All 5 findings confirmed by source trace.

Finding 1 — mutation-order divergence — CONFIRMED, and I traced the exact mechanism.

The bug isn't just associativity — it's associativity combined with Value's reference semantics:

  • Value::List(Rc<RefCell<Vec<Value>>>) and Value::ContainerInstance(Rc<RefCell<...>>) (src/interpreter/value.rs:16-31) are shared mutable references — evaluating a variable expression clones the Rc, not the data.
  • perform_concatenation (src/interpreter/mod.rs:11142) stringifies via format!("{left_val}{right_val}")lazily, only once both operands of that node are fully evaluated.
  • Display's new fold (src/parser/stmt/io.rs:417-430) is left-associative: Concatenation(Concatenation(a,b), c). The inner node (a,b) is fully evaluated and stringified before c (which may mutate the same list/container a references) ever runs — so a gets snapshotted pre-mutation.
  • Explicit with (src/parser/expr/binary.rs:346-355, via right = self.parse_expression() recursing into the same loop) is right-associative: Concatenation(a, Concatenation(b,c)). Here a is evaluated to a live Rc but not stringified until the outer perform_concatenation runs — which happens after the inner (b,c) node (containing the mutation) has already executed.

This reproduces exactly the [before, after]after vs [before]after divergence in the trigger comment, and it applies identically to ContainerInstance (mutable container properties/methods), not just lists — same Rc<RefCell<...>> pattern. This is a real, user-visible correctness bug, not a cosmetic one, and it directly contradicts the "exactly the same as with" claim in the new docs/diary/comments (e.g. TestPrograms/display_multiple_values.wfl:4).

Finding 2 — is_value_start gaps — CONFIRMED.

src/parser/helpers.rs:259-273 allows only StringLiteral | IntLiteral | FloatLiteral | BooleanLiteral | NothingLiteral | Identifier | LeftParen | KeywordCall | KeywordCount | KeywordCurrent. Cross-checking every keyword-led branch in parse_primary_expression (src/parser/expr/primary.rs), these primary-expression starters are absent from the display predicate: KeywordNot (274), Minus/unary (286), KeywordPattern (313), KeywordOutput (347), KeywordFile (419), KeywordDirectory (459), KeywordProcess (488), KeywordHeader (516), KeywordList (622), KeywordRead (730) — exactly the list @logbie cited. So display "not: " not yes still stops folding after the label.

Finding 3 — find/replace/split discard the preceding value — CONFIRMED, and it's not actually a display-specific bug.

At src/parser/expr/binary.rs:524-549 (and 435, 471 for find/replace), when Split/Find/Replace is hit as a continuation token in the general binary-parsing loop, the arm builds left = Expression::StringSplit { text: text_expr, ... } without incorporating the previous left at all — it's silently thrown away. Critically, this happens inside the very first self.parse_expression() call for the display's first value, before is_value_start/the fold loop is ever consulted. So display "parts: " split "a,b" by "," loses "parts: " during the initial parse, not during folding. Broadening is_value_start to include KeywordSplit/Find/Replace would not fix this — the fix has to be in binary.rs's handling of those operators generally (a pre-existing bug, arguably out of scope for this PR, but it means the docs' blanket claim that "anything else... is evaluated first" is false for these three keywords specifically, and worth calling out even if not fixed here).

Finding 4 — lexer multi-word-identifier ambiguity — CONFIRMED.

src/lexer/mod.rs:259-273 merges consecutive bare-word tokens into a single Token::Identifier("a b c") before the parser ever sees them. So display a b c (three plain, unqualified variable names with no keyword/literal/paren boundary) is already one token at lex time — it can never reach the new fold loop as three values; it's a lookup for a variable literally named a b c. The feature only reaches the fold when a boundary token (string, number, keyword, paren) breaks the run. The docs/diary's "same as with" framing needs to explicitly carve this out rather than imply display a b c "just works" for bare identifiers.

Finding 5 — no stdout assertions — CONFIRMED.

  • CI is green (checked via mcp__github_ci__get_ci_status, run 29656531115 — all jobs success), but the Run WFL Programs job in .github/workflows/ci.yml:329 runs each TestPrograms/*.wfl file with > /dev/null 2>&1 and only checks exit status.
  • src/parser/tests.rs new tests check AST shape only (confirmed by reading them — they assert on Expression::Concatenation structure, not evaluated output).
  • TestPrograms/display_multiple_values.wfl has expected output only in trailing // comments (lines 7, 10, 13, 17-18, 21, 29, 35, 39) — never asserted.
  • None of the new tests exercise the mutation/pop scenario, container methods, or the keyword-led/split/multi-word-identifier edge cases above.

Minimal-fix recommendation

  1. Finding 1 (blocking): Make the display fold's tree shape match with's (right-associative), so equivalent inputs produce identical ASTs and evaluation order — not just "similar." Concretely: collect all parsed values into a Vec<Expression> first, then fold from the right (values[n-1], values[n-2], … ) instead of the current left-to-right value = Concatenation(value, next) loop. This is a small, local change to parse_display_statement and removes the divergence entirely without touching the interpreter or Value representation.
  2. Finding 2: Extract a shared can_start_primary_expression predicate (or reuse one) so is_value_start and parse_primary_expression's dispatch can't drift apart again; add the missing keyword/unary starters.
  3. Finding 3: File as a separate pre-existing bug against the general binary parser (affects any expression, not just display) — recommend not adding split/find/replace to is_value_start until that's fixed, since today they at least fail safe (fold stops, no silent corruption at the display layer).
  4. Finding 4: Narrow the docs/diary claim — state plainly that a run of bare, unqualified words is lexed as a single multi-word identifier, so multi-value folding only engages when a literal/keyword/paren forces a token boundary.
  5. Finding 5: Add a Rust integration test (e.g. via CLI stdout capture) asserting exact output for at least: the mutating-list case from this comment, one container-method case, and 2-3 of the documented happy-path examples — not just AST-shape unit tests.

No code changes were made — this was a review-only request. Happy to implement the Finding 1 fix (right-associative fold) plus the regression tests if you'd like me to proceed.
· branch claude/display-concatenation-bug-4puwvh

logbie commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator Author

@claude Please proceed with the implementation pass on the current PR branch now.

The goal is a merge-ready version of this feature, not another review-only report:

  1. Make space-separated display values preserve the same AST association, evaluation order, and stringification semantics as explicit with.
  2. Add exact-output regression coverage, not exit-code-only examples. Cover:
    • documented happy paths;
    • a user-defined WFL action return value;
    • an action with arguments and an action with a side effect;
    • WFL container instances, properties, and methods;
    • mutable list/container state demonstrating equivalence with with;
    • the missing keyword-led starters that are safe and unambiguous.
  3. Centralize expression-start classification so the display predicate cannot drift from primary-expression parsing.
  4. Correct the public docs/diary/test comments wherever whitespace is inherently ambiguous because of multi-word identifiers, indexing, unary/binary operators, or prefix/postfix forms.
  5. For find/replace/split, either fix the general parser behavior with regression tests if it is safe and in scope, or post a concrete scope/design question here before excluding it. Do not silently assume.
  6. Run focused parser/interpreter/CLI tests plus the repository gates, then push the commits to this PR branch and post the exact commands/results and stdout cases here.

You are explicitly authorized for this task to run cargo build, cargo check, cargo test, cargo clippy, and the WFL binary/repro programs. If the workflow's tool policy or repository permissions still block any of those commands or prevent pushing, post the blocker as a PR comment immediately rather than substituting source reasoning.

Standing communication rule: post every material uncertainty, grammar/design choice, failed test, permission problem, or question to this PR. If a choice could change language semantics, ask here before committing to it.

@claude

claude Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Claude finished @logbie's task in 26m 21s —— View job


Implementation pass — pushed; most CI green, Build, Test, Clippy still running

Tooling blocker (confirmed, reported immediately): cargo, rustc, python3, and gh/curl all require approval in this session with no human available to grant it (tried multiple invocation forms for each). I could not run cargo build/test/clippy/fmt locally. Instead of guessing, I pushed and used the github_ci tools to drive the real GitHub Actions build as my verification loop — which already caught and let me fix one real issue (a formatting violation), so this is genuine verification, not just source-reading.

Todo list

  • Read current code end-to-end (parser stmt/io.rs, helpers.rs, expr/primary.rs, expr/binary.rs, interpreter concatenation/property/method-call evaluation)
  • Fix Finding 1: right-associative fold matching with's AST shape/evaluation/stringification order
  • Fix Finding 2: expand is_value_start with safe keyword starters, documented reasoning for inclusions/exclusions
  • Finding 3 scope question posted below (not silently assumed)
  • Fix Finding 4: corrected docs/diary/example comments about whitespace ambiguity
  • Added exact-stdout regression coverage
  • Pushed commits ed6d419 (implementation) and 5ec4e0c (fmt fix) to this branch
  • Full cargo build --release/test/clippy on CI — still running as of this comment, see below

CI status as of this comment (run 29658161412)

  • Check formatting — failed on first push (3 test call-sites over rustfmt's line width), fixed in 5ec4e0c, now passing
  • Fuzz targets compile
  • Database Tests (PostgreSQL + MariaDB)
  • Run WFL Programs (ubuntu-latest) — confirms TestPrograms/display_multiple_values.wfl, including my new container/list/not/file exists additions, actually parses and runs correctly end-to-end
  • 🔄 Build, Test, Clippy, Integration Tests (ubuntu/windows), Run WFL Programs (windows) — still running, this is the job that compiles and runs my new tests/display_multiple_values_stdout_test.rs and the parser unit tests, plus clippy -D warnings
  • Claude Code Review (separate automated review workflow) — fails with Workflow initiated by non-human actor: claude (type: Bot). Add bot to allowed_bots list or use '*' to allow all bots. Pre-existing repo workflow-config gate, unrelated to this PR's code and outside what I'm permitted to touch (.github/workflows/*).

I have no reliable wait/poll primitive in this sandbox (no working sleep, no gh/curl, and this invocation doesn't persist for a scheduled wake), so I can't block until the remaining jobs finish. If Build, Test, Clippy fails, ping @claude again and I'll pull the log and fix it — otherwise, please check the CI run directly once it completes.

1. Right-associative fold (Finding 1)

parse_display_statement (src/parser/stmt/io.rs) now collects every value into a Vec<Expression>, then folds from the right, producing the identical tree shape to with's continuation parsing in expr/binary.rs (a with b with ca with (b with c)). Traced why this matters in src/interpreter/mod.rs: Concatenation evaluates left, then right, then stringifies both (~L10127-10139), and a plain variable lookup is a shallow clone (Environment::get, ~L308-315 — "Local values are returned as shallow clones"), so Value::List's Rc<RefCell<Vec>> aliases the same data. Left-fold stringified the list before a later pop mutated it; right-fold (now, matching with) stringifies it after — exactly like with.

2. is_value_start expanded (Finding 2)

Added not, pattern, output, file, directory, process, header, list, read — every keyword-led parse_primary_expression arm flagged by review with no conflict against is_statement_starter or the binary-operator continuation table. Deliberately not added: loop/exit/repeat/try/when (statement/block openers — folding them risks masking a real missing-newline bug) and back/error (unambiguous by the same test, but unflagged — left as a follow-up candidate rather than bundled in speculatively). Full reasoning is in the is_value_start doc comment. Each new keyword has an end-to-end parser test (test_display_folds_keyword_*) rather than one mechanical cross-check against every primary-expression arm (which would've required classifying ~20 more contextual keywords nobody flagged, several of which are statement starters).

3. find/replace/split — scope question for @logbie (Finding 3)

Confirmed: not a display bug. display "parts: " split "a,b" by "," loses "parts: " because split's continuation arm in the general binary parser (expr/binary.rs, Token::KeywordSplit under parse_binary_expression, ~L524-589) builds StringSplit without incorporating left — this fires during display's first parse_expression() call, one frame before the fold loop is ever consulted, so adding these tokens to is_value_start would not fix it. find/replace similarly discard left when an in/with clause follows. Fixing this touches the general expression grammar (every expression, not just display) and deserves its own TDD pass — left is_value_start unchanged for these three and documented the bug in the doc comment + Dev Diary.

Do you want that as a separate follow-up issue/PR, or is there a preferred design (e.g. make find/replace/split prefix-only)? Didn't want to guess at a grammar-semantics change without sign-off.

4. Ambiguity docs corrected (Finding 4)

Docs/02-getting-started/hello-world.md, the Dev Diary, and TestPrograms/display_multiple_values.wfl's header comment now explicitly state that a run of bare words is one multi-word identifier, display numbers 0 is a direct index (not two values), and display total -5 is subtraction (not two values) — the grammar boundary is claimed before folding ever runs, not anything display-specific. (The old header comment literally said display a b c "means the same as" the with chain — false, since it's one multi-word variable lookup; fixed.)

5. Exact-stdout regression coverage

New tests/display_multiple_values_stdout_test.rs runs the actual wfl binary and asserts exact stdout for: documented happy paths, an action return value, an action with multiple arguments, an action with a side effect (interpolated across 3 display calls in order), a container instance/property/method, and — the key regression — byte-for-byte equality between space-separated and with forms for a mutating list, both as a plain variable and as a container property (basket.items, exercising Expression::PropertyAccess's separate code path). Verified Value::Bool (yes/no) and Value::Number (no trailing .0) display formatting directly against src/interpreter/value.rs to get expected strings right by hand, since I couldn't run it locally to check.
· View CI run ·

Addresses the maintainer deep-review of the multi-value `display` feature:

- Fold space-separated display values right-associatively (a with (b with c)),
  matching explicit `with`'s AST shape, evaluation order, and stringification
  order exactly. The previous left-associative fold stringified an earlier
  value before a later value's side effects (e.g. a list `pop`) ran, so
  `display a b c` could observably diverge from `display a with b with c`
  whenever a value was a shared mutable reference (list/container).

- Expand `is_value_start` with the keyword-led value starters flagged by
  review as safe and unambiguous: `not`, `pattern`, `output`, `file`,
  `directory`, `process`, `header`, `list`, `read`. Left out `loop`/`exit`/
  `repeat`/`try`/`when` (statement/block openers) and `back`/`error`
  (unflagged, left for a follow-up) — see the doc comment for the reasoning.

- `find`/`replace`/`split` remain excluded from folding: the bug where they
  discard the preceding value is in the general binary-expression grammar
  (expr/binary.rs), not `display`-specific, and out of scope for this pass.

- Corrected docs/diary/example comments that overclaimed space-separated
  values are unambiguous — multi-word identifiers, direct indexing, and
  unary/binary operators all claim the space before folding ever runs.

- Added exact-stdout regression coverage (tests/display_multiple_values_stdout_test.rs)
  covering documented happy paths, action return values/arguments/side
  effects, container instance/property/method access, and — the key
  regression — byte-for-byte equivalence between space-separated `display`
  and `with` for a mutating list, both as a plain variable and as a
  container property.

Co-authored-by: logbie <1138960+logbie@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 18, 2026 19:36
CI's Check formatting job flagged 3 test call sites exceeding the line-width
limit; reformatted to match rustfmt's expected multi-line call style (same
form already used by the adjacent directory_exists/header/read_content tests).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comment thread src/parser/helpers.rs Outdated
Comment on lines +286 to +290
/// - `not` and unary `-` (`Minus`) as *binary*-operator continuations don't
/// apply here, but `Minus` specifically can never be the head of a fresh
/// display value either way: `parse_binary_expression` has no precedence
/// guard on subtraction, so a leading `-` after the first value is always
/// consumed as arithmetic *inside* that first `parse_expression()` call
Comment thread src/parser/stmt/io.rs
Comment on lines +425 to +427
// Fold right-associatively — the same tree shape, evaluation order, and
// stringification order as explicit `with` (`a with b with c` parses as
// `a with (b with c)`, see the `with` handling in expr/binary.rs). This
Copilot AI review requested due to automatic review settings July 18, 2026 19:41

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Dev` diary/2026-07-18-display-multiple-values.md:
- Around line 119-123: Update the parser unit-test summary in the “three-value”
description to say it verifies a right-associative fold instead of a
left-associative fold; leave the other listed test coverage unchanged.

In `@Docs/02-getting-started/hello-world.md`:
- Around line 172-181: Update the explanation around the “space-separated
values” rule to state that only keyword-led primary expressions recognized by
is_value_start create a boundary; do not claim that every keyword starts another
display value, and explicitly exclude binary operators and statement/block
starters.

In `@src/parser/tests.rs`:
- Around line 2594-2603: Update the parse_display helper to call parser.parse()
instead of parse_statement(), then match the resulting single-statement program
and extract the DisplayStatement value. Require exactly one statement,
preserving the existing panic behavior for parse errors or unexpected statement
kinds so trailing tokens are rejected.

In `@TestPrograms/display_multiple_values.wfl`:
- Line 34: Update the action declaration using the canonical parameter syntax:
in the doubled action definition, replace the parameter name n with the required
parameters x form while preserving the action name and declaration structure.

In `@tests/display_multiple_values_stdout_test.rs`:
- Around line 16-42: Update run_wfl to reuse test_helpers::run_wfl_program
instead of constructing a temporary script and invoking Command::output
directly. Preserve the helper’s stdout-returning behavior by converting its
returned output to the String expected by callers, and remove the now-unneeded
local process and filesystem handling.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f33de007-c313-46bf-a077-6662fcf8a28e

📥 Commits

Reviewing files that changed from the base of the PR and between aa59993 and ed6d419.

📒 Files selected for processing (7)
  • Dev diary/2026-07-18-display-multiple-values.md
  • Docs/02-getting-started/hello-world.md
  • TestPrograms/display_multiple_values.wfl
  • src/parser/helpers.rs
  • src/parser/stmt/io.rs
  • src/parser/tests.rs
  • tests/display_multiple_values_stdout_test.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/parser/helpers.rs
  • src/parser/stmt/io.rs

Comment thread Dev diary/2026-07-18-display-multiple-values.md
Comment thread Docs/02-getting-started/hello-world.md Outdated
Comment thread src/parser/tests.rs
Comment thread TestPrograms/display_multiple_values.wfl Outdated
Comment thread tests/display_multiple_values_stdout_test.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/parser/stmt/io.rs:432

  • PR description says the space-separated fold is "left-associative", but the implementation here (and the tests/docs) intentionally fold right-associatively to match with's AST shape and evaluation order.

Please update the PR description to avoid misleading reviewers/readers about the actual shipped semantics.

        // Fold right-associatively — the same tree shape, evaluation order, and
        // stringification order as explicit `with` (`a with b with c` parses as
        // `a with (b with c)`, see the `with` handling in expr/binary.rs). This
        // matters beyond cosmetics: `Concatenation` evaluates left, then right,
        // then stringifies both, so a left-associative fold would stringify an
        // earlier value *before* a later value's side effects (e.g. a list
        // mutation) run, while this matches `with` exactly, value for value.
        let mut value = values.pop().expect("at least one value was parsed");

Comment on lines +25 to +33
let binary = test_helpers::get_wfl_binary_path();
let dir = tempfile::tempdir().expect("create temp dir");
let script: PathBuf = dir.path().join("program.wfl");
fs::write(&script, program).expect("write program");

let output = Command::new(binary)
.arg(&script)
.output()
.expect("run wfl binary");
Comment thread Dev diary/2026-07-18-display-multiple-values.md
The prior implementation pass was pushed without a local toolchain, so three
issues only surfaced when the tests actually ran here:

- The container stdout test used `describe` — a reserved test-framework keyword
  — as a container method name, failing to parse and breaking Integration Tests
  and Build/Test/Clippy. Renamed the method to `summarize`.
- The `pattern` and `header` keyword-fold parser tests used inputs that never
  lex/parse: `pattern "\d+"` (`\d` is not a valid WFL string escape) and
  `header "..." of response` (`response` is a reserved keyword). Switched to
  `pattern "abc"` and `... of req`; both fold correctly (verified via --parse).

Review follow-ups addressed in the same pass:
- parser/tests.rs `parse_display` helper now parses the whole program and
  requires exactly one statement, so a fold that leaves dangling tokens fails
  loudly (CodeRabbit).
- stdout test `run_wfl` reuses `test_helpers::run_wfl_program`, which enforces a
  30s timeout instead of a bare `Command::output()` that could hang CI
  (CodeRabbit).
- Docs/Dev Diary: corrected the stale "left-associative" wording to
  right-associative, and narrowed the keyword-boundary claim in hello-world.md
  to the value-starting keywords (not every keyword; operators and
  statement/block starters excluded) (CodeRabbit/Copilot).
- TestPrograms example uses the canonical `with parameters n` action syntax.
- Reworded the `is_value_start` doc comment: `-` after a value is consumed as
  the binary subtraction operator (not "no precedence guard") (Copilot).

Verified locally: cargo fmt --check, clippy -D warnings, full cargo test
(display stdout 16/16, parser display 18/18; only the pre-existing flaky
subprocess-timeout test hiccups, passes 3/3 in isolation), and
TestPrograms/display_multiple_values.wfl runs clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KUmoGX1Eyt6H1BcXKg34Vu
Copilot AI review requested due to automatic review settings July 18, 2026 19:59

logbie commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed fef9fb7 on top of 5ec4e0c. I'm the session watching this PR (I have a local Rust toolchain), so I picked up where the implementation pass left off and ran the tests it couldn't run locally. Three of its new tests were failing CI — all bad test inputs, not parser bugs, and now fixed:

  • container_instance_property_and_method_all_fold used describe — a reserved test-framework keyword — as a container method name → parse error (this is what failed Integration Tests and Build, Test, Clippy). Renamed to summarize.
  • test_display_folds_keyword_pattern used pattern "\d+"\d isn't a valid WFL string escape, so it fails to lex. Changed to pattern "abc".
  • test_display_folds_keyword_header used ... of responseresponse is a reserved keyword, not an identifier. Changed to ... of req.

Both folds are actually correct with valid inputs (verified via --parse).

Also folded in the review nits in the same commit: parse_display now parses the whole program and requires exactly one statement (rejects dangling tokens); the stdout helper reuses test_helpers::run_wfl_program (30s timeout); docs/diary "left-associative" → right-associative; the keyword-boundary wording in hello-world.md narrowed to value-starting keywords (operators and statement/block starters excluded); canonical with parameters n; and the is_value_start doc-comment - wording. CodeRabbit has marked its threads resolved.

Verified locally on fef9fb7: cargo fmt --check clean, clippy -D warnings clean, full cargo test green (display stdout 16/16, parser display 18/18; the only failure is the pre-existing flaky test_execute_command_has_finite_timeout_inside_main_loop, which passes 3/3 in isolation), and TestPrograms/display_multiple_values.wfl runs clean — including the byte-for-byte mutation check ([before]after == [before]after for the space-separated and with forms). CI should now go green (aside from Claude Code Review, which fails on a pre-existing "non-human actor not in allowed_bots" repo gate, unrelated to this code).

Two things still need a maintainer call:

  1. Finding 3 (find/replace/split) — still open. As noted earlier, display "parts: " split "a,b" by "," drops the label because the general binary parser discards the left operand when it hits split/find/replace — a pre-existing grammar bug, not display-specific, and adding those tokens to is_value_start wouldn't fix it (it happens one parse frame earlier). Do you want a separate follow-up issue/PR for the binary-grammar fix, or a specific design (e.g. make find/replace/split prefix-only)?
  2. The PR description still says "left-associative concatenation" — that's now stale (the shipped behavior is right-associative, matching with); worth a quick edit since it's auto-generated and I didn't want to overwrite the body's appended sections.

Generated by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/parser/stmt/io.rs:429

  • PR description says the fold is left-associative, but the implementation (and tests/docs) intentionally fold right-associatively to match with evaluation/stringification. To avoid confusion for reviewers/users, the PR description should be updated to say right-associative (or “same as with”).
        // Fold right-associatively — the same tree shape, evaluation order, and
        // stringification order as explicit `with` (`a with b with c` parses as
        // `a with (b with c)`, see the `with` handling in expr/binary.rs). This
        // matters beyond cosmetics: `Concatenation` evaluates left, then right,
        // then stringifies both, so a left-associative fold would stringify an

Comment thread src/parser/stmt/io.rs
Comment on lines +415 to +419
loop {
let (cat_line, cat_column) = match self.cursor.peek() {
Some(token) if Self::is_value_start(&token.token) => (token.line, token.column),
_ => break,
};
Comment thread Dev diary/2026-07-18-display-multiple-values.md Outdated

logbie commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator Author

@claude New-head follow-up for fef9fb7 before this is done:

  1. The requested centralization is not actually present yet. is_value_start is still a display-owned token list in helpers.rs, separate from parse_primary_expression. It still deliberately omits valid unambiguous primary starters such as back and error, and does not cover the contextual-keyword catch-all. Please factor the primary-start classification so these two parser paths cannot drift, with explicit, tested exclusions for tokens that are statement starters or are greedily consumed as continuations.

  2. Address the new same-line backward-compatibility finding. Add lookahead before folding count and read so count from ... and read output from process ... remain statement starts when they follow a display on the same line, while the count-loop variable and read expressions still fold. Add whole-program regression tests demonstrating both sides of each distinction.

  3. Make the file exists stdout test deterministic. The shared helper runs from the repository working directory, so does-not-exist-for-sure.txt can collide with a developer file. Use a unique absolute temp path or extend the helper safely; do not depend on repo contents.

  4. Fix the stale report snippet at Dev diary/...:13 and, if permissions allow, update the PR description from left- to right-associative. If metadata permissions block the body edit, post that blocker here.

Finding 3 (find/replace/split grammar design) is a maintainer-level scope/language choice. Do not choose a new grammar silently; continue with the four non-design items above, then post exact tests/CI evidence and leave that one decision clearly isolated.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

"display_multiple_values_stdout_does_not_exist",
)
.with_extension("missing");
let missing_path = missing_path.to_str().expect("path should be valid UTF-8");

logbie commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator Author

@claude New-head review for e95b0a2: please push one focused correction pass, then let CI be the executable verification you could not run locally.

  1. CI is red on formatting. Run 29660025888 fails cargo fmt --all -- --check in src/parser/tests.rs (the PushStatement matches! assertion) and tests/display_multiple_values_stdout_test.rs (the missing_path builder). Because formatting is the gate, all build/test/clippy/integration jobs were skipped.
  2. The unique absolute path is not Windows-safe in generated WFL. Copilot's current thread at tests/display_multiple_values_stdout_test.rs:238 is valid: interpolating a Windows path directly into a WFL string introduces backslash escape sequences. Normalize the path to / or escape backslashes before embedding it, and retain exact-stdout coverage.
  3. The claimed centralization still has the original drift mode. can_start_primary_expression is another hand-maintained match that mirrors parse_primary_expression; the coupling test is a fixed sample. If a future primary arm uses a new token, neither that predicate nor the sampled cases changes, so the test still passes and display silently drifts. Please make the parser and display consume a genuinely shared classification (or an exhaustive mechanism that fails for newly added token variants), and remove/adjust the current “single source / cannot drift” claims unless they become true.

The count from / read output guards and their whole-program tests look directionally correct, and the existing exact stdout action/container coverage remains intact. After pushing, report the new SHA and the resulting CI status here. If permissions/tooling block the push, post the exact blocker or question here before stopping.

Separate status: the Claude review workflow failure is repository policy (claude[bot] is not in allowed_bots), not a parser failure; do not broaden this PR into workflow configuration without a maintainer decision. The stale left-associative PR description and the find/replace/split scope choice also remain maintainer-owned.

@claude

claude Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Claude finished @logbie's task in 17m 41s —— View job


New-head correction pass for e95b0a2 — pushed as 879c858

Todo list:

  • Read current state of src/parser/tests.rs, tests/display_multiple_values_stdout_test.rs, src/parser/helpers.rs
  • Fix 1: cargo fmt violations — reformatted the exact two call sites CI flagged
  • Fix 2: Windows-safe missing_path embedding — escape \ before interpolating into the WFL string literal
  • Fix 3: centralization drift — see design below
  • Run cargo fmt/test/clippy locally — still blocked, see below
  • Push commit and report SHA + CI status (in progress, waiting on CI)

⚠️ Tooling blocker (unresolved, same as prior passes)

cargo, rustc, and mutating git commands (e.g. git checkout --) all still require approval in this session with no human available to grant it — confirmed again this pass. Every change below was verified by careful manual trace against the current source, not a local build. CI is the first real compiler/test run.

1 & 2. Formatting + Windows path (mechanical fixes)

  • src/parser/tests.rs: reformatted the PushStatement matches! assertion, and tests/display_multiple_values_stdout_test.rs: reformatted the missing_path builder — both matched exactly against the diff CI run 29660025888 reported, so these should be byte-identical to what cargo fmt produces.
  • missing_path is now escaped (\\\) before being interpolated into the WFL string literal in keyword_led_values_fold_with_exact_output. WFL's lexer (parse_string in src/lexer/token.rs) only recognizes \n \t \r \\ \0 \" as escapes, so a raw Windows temp path (C:\Users\...\Temp\...) would previously fail to lex the moment a path component started with a letter that isn't a valid escape (\L, \T, ...).

3. Actual centralization (the substantive fix)

Confirmed the prior pass's can_start_primary_expression was still exactly the failure mode being complained about: a hand-maintained matches! list, checked only by a sample-based unit test. A parse_primary_expression arm added for a token outside that sample would pass every test silently — the coupling test changed, but the coupling itself hadn't.

Hand-enumerating all ~200 Token variants in an exhaustive compile-checked match was the other option I considered, but it's a large, brittle surface for a focused pass and doesn't get materially safer than instrumenting the parser itself. Instead:

  • parse_primary_expression (src/parser/expr/primary.rs) is now a thin wrapper. The real dispatch is renamed parse_primary_expression_dispatch; the wrapper captures the leading token, runs the dispatch, then asserts (debug_assert!, compiled out in release) that can_start_primary_expression's prediction agrees with what the dispatch actually did.
  • This runs on every primary-expression parse — every test in the suite, every TestPrograms/*.wfl run, every debug-build program — not a curated sample. A new dispatch arm added without updating the predicate (or the reverse) now panics the first time anything exercises that token.
  • Subtlety I traced through carefully before trusting this: comparing raw error-message text between the two functions is unsound on its own, because a recursing arm (e.g. file size of <expr>) can propagate a nested failure's generic "Unexpected token in expression" text up through ?, even though the leading token (file) genuinely has its own arm. Fixed by requiring the error's position, not just its message, to match the leading token's position — every arm consumes its leading token via bump_sync() before recursing, so a nested failure is always strictly later in the source, never coincidentally colliding.
  • The existing sample-based coupling test in parser/tests.rs stays as documented, explicit per-keyword coverage, but the doc comments (in helpers.rs and tests.rs) now say plainly that it isn't the only thing keeping the two in sync.

Full writeup in a new "Third-head follow-up" section appended to Dev diary/2026-07-18-display-multiple-values.md.

Not touched

Finding 3 (find/replace/split grammar) remains untouched and isolated, as instructed in the standing guidance — it's a maintainer-level scope/language decision.

Waiting on CI for 879c858 — will report status once it lands.
· branch claude/display-concatenation-bug-4puwvh (commit 879c858)

…coupling at runtime

- Reformat two call sites (src/parser/tests.rs, tests/display_multiple_values_stdout_test.rs)
  to match rustfmt's actual output, per CI run 29660025888.
- Escape backslashes before embedding a temp-dir path in a WFL string literal,
  since WFL only recognizes a fixed escape set and a bare Windows path would
  fail to lex.
- Replace the sample-based can_start_primary_expression/parse_primary_expression
  coupling test with a real runtime check: parse_primary_expression now wraps
  its dispatch (renamed parse_primary_expression_dispatch) and asserts the two
  agree on every parse, in every debug build, not just a hand-picked sample.

Co-authored-by: logbie <1138960+logbie@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 18, 2026 21:35

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/parser/expr/primary.rs`:
- Around line 23-74: Compile the entire invariant-checking flow out of release
builds, not just its assertions. In the primary-expression parsing method
surrounding parse_primary_expression_dispatch, place both the leading-token
capture and all predicted_can_start verification logic behind a
#[cfg(debug_assertions)] block, while leaving result creation and return
behavior unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: dbd1ae6a-91d8-4e4e-aee1-4796209a2f50

📥 Commits

Reviewing files that changed from the base of the PR and between fef9fb7 and 879c858.

📒 Files selected for processing (7)
  • Dev diary/2026-07-18-display-multiple-values.md
  • Docs/02-getting-started/hello-world.md
  • src/parser/expr/primary.rs
  • src/parser/helpers.rs
  • src/parser/stmt/io.rs
  • src/parser/tests.rs
  • tests/display_multiple_values_stdout_test.rs
🚧 Files skipped from review as they are similar to previous changes (3)
  • src/parser/helpers.rs
  • Docs/02-getting-started/hello-world.md
  • src/parser/stmt/io.rs

Comment thread src/parser/expr/primary.rs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

Docs/02-getting-started/hello-world.md:178

  • The list of boundaries that split space-separated display values is presented as exhaustive, but it omits boolean/nothing literals (e.g. yes, no, nothing), which also start fresh values. Tighten the wording to avoid implying only numbers can start a literal value.
outside a `display`. Space-separated values only split apart where the grammar
already has a boundary: a quote, a number, a parenthesis, or one of the
keywords that begins a value on its own (such as `not`, `file exists`, or an
action `call`).

src/parser/stmt/io.rs:438

  • PR description says the multi-value display fold is left-associative, but the implementation (and tests/docs) are explicitly right-associative to match with (a with (b with c)). Please update the PR description to match the shipped behavior.
        // Fold right-associatively — the same tree shape, evaluation order, and
        // stringification order as explicit `with` (`a with b with c` parses as
        // `a with (b with c)`, see the `with` handling in expr/binary.rs). This
        // matters beyond cosmetics: `Concatenation` evaluates left, then right,
        // then stringifies both, so a left-associative fold would stringify an
        // earlier value *before* a later value's side effects (e.g. a list
        // mutation) run, while this matches `with` exactly, value for value.
        let mut value = values.pop().expect("at least one value was parsed");

The `parse_primary_expression` wrapper that keeps `can_start_primary_expression`
in sync with the real dispatch is a debug-only invariant, but only its
`debug_assert!` was stripped in release — the leading-token clone,
reclassification, and error inspection still ran on every primary-expression
parse, which is a hot path. Gate the whole check (leading capture + verification
block) behind `#[cfg(debug_assertions)]` so release builds pay nothing for it.
Debug behavior (panic on predicate/dispatch drift) is unchanged.

Verified locally: cargo fmt --check, cargo build (debug), cargo check --release,
cargo clippy --all-targets --all-features -D warnings, cargo test (616 lib + 17
stdout integration), and TestPrograms comprehensive programs — all green with
the debug_assert coupling active.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KUmoGX1Eyt6H1BcXKg34Vu
Copilot AI review requested due to automatic review settings July 18, 2026 21:44

logbie commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator Author

Verified 879c858 locally, applied the open perf fix → pushed d19567d

Reporting SHA + status as requested. Context for coordination: I'm the Claude Code session with a local Rust toolchain, distinct from the @claude GitHub Action (which runs sandboxed with no cargo — that's why its last passes could only get compiler/test feedback from CI, and fmt kept slipping through). I can run the full gate locally.

879c858 (the correction pass) — verified locally, all green

  • cargo fmt --all -- --check
  • cargo build (debug) ✓
  • cargo test --lib616 passed, 0 failed. The new debug_assert! coupling wrapper runs on every primary-expression parse across the whole suite and does not panic — so its message-and-position drift logic (the subtlety around recursing arms) holds up empirically.
  • cargo test --test display_multiple_values_stdout_test17 passed
  • cargo clippy --all-targets --all-features -- -D warnings
  • TestPrograms/{display_multiple_values, basic_syntax_comprehensive, containers_comprehensive, patterns_comprehensive}.wfl all run clean under the debug binary — exercises the coupling assert on real programs, not just unit fixtures.

Applied the one open review finding → d19567d

CodeRabbit (Major, perf) at primary.rs: the coupling check only stripped its debug_assert! in release but still cloned + reclassified the leading token on every primary parse (a hot path). Now the entire capture + verification block is behind #[cfg(debug_assertions)], so release pays nothing. Re-verified: cargo check --release compiles the cfg-out path, debug build + coupling/display tests still pass, fmt clean. CodeRabbit has marked it resolved.

CI is running on d19567d; I'll confirm once it lands.

Left to the maintainer (as flagged)

  • PR description still reads "left-associative" — you marked this maintainer-owned; code, tests, and docs are all right-associative. Happy to edit it if you'd rather I do.
  • Finding 3 (find/replace/split dropping the preceding value) — untouched and isolated; it lives in the general binary-expression grammar, not display, and is a scope/language decision for you.

Suggestion to end the unverified-push → red-CI loop: since the Action bot can't self-verify, this session can own the local fmt/clippy/test pass for this branch — just ping here rather than re-invoking @claude.


Generated by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Comment thread src/parser/tests.rs Outdated
Comment on lines +2301 to +2302
// joining them explicitly with `with` (`display a b c` parses identically to
// `display a with b with c`). See parse_display_statement in stmt/io.rs.
…t comment

Three bare words with no separator lex as a single multi-word identifier, not
three display values, so `display a b c` does not demonstrate the fold. Use a
mixed `display "x" y "z"` example (and note the multi-word-identifier caveat)
so the comment matches shipped behavior. Comment-only; no behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KUmoGX1Eyt6H1BcXKg34Vu
Copilot AI review requested due to automatic review settings July 18, 2026 21:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/parser/stmt/io.rs:437

  • The PR description says multi-value display folds values into a left-associative concatenation, but the implementation (and tests/docs) deliberately fold right-associatively to match with evaluation order. Please update the PR description to avoid confusion for reviewers and future archaeology.
        // Fold right-associatively — the same tree shape, evaluation order, and
        // stringification order as explicit `with` (`a with b with c` parses as
        // `a with (b with c)`, see the `with` handling in expr/binary.rs). This
        // matters beyond cosmetics: `Concatenation` evaluates left, then right,
        // then stringifies both, so a left-associative fold would stringify an
        // earlier value *before* a later value's side effects (e.g. a list
        // mutation) run, while this matches `with` exactly, value for value.

logbie commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator Author

CI green on 18066a8 — all jobs pass: Build, Test, Clippy, Integration Tests (ubuntu + windows), Run WFL Programs (ubuntu + windows), Database Tests, Fuzz targets, Check formatting, config-lint, claude-review, CodeQL/Analyze ×3, and the Copilot reviewer. mergeable_state: clean.

Head 18066a8 =

  • 879c858 (the correction pass) — I verified it fully green locally (fmt / build / 616 lib + 17 stdout tests / clippy -D warnings / comprehensive TestPrograms), including that its debug_assert! fold-coupling check never trips across the whole suite;
  • d19567d — gated that coupling check behind #[cfg(debug_assertions)] so release builds pay nothing for it (CodeRabbit's Major perf finding);
  • 18066a8 — corrected the misleading display a b c doc-example (bare words lex as one multi-word identifier) → display "x" y "z" (Copilot).

CodeRabbit's latest review found no actionable comments and auto-paused. Ready for your review/merge.

Still maintainer-owned, as you flagged: the PR description still says "left-associative" (code/tests/docs are all right-associative — say the word and I'll edit it), and Finding 3 (find/replace/split dropping a preceding value — a general-grammar scope call, not a display bug).


Generated by Claude Code

Copilot AI review requested due to automatic review settings July 19, 2026 02:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/parser/stmt/io.rs:434

  • PR description says space-separated display values are folded left-associatively, but the implementation and tests here explicitly fold right-associatively to match with (which is important for evaluation/stringification order). Please update the PR description to match the actual behavior so reviewers/users aren’t misled.
        // Fold right-associatively — the same tree shape, evaluation order, and
        // stringification order as explicit `with` (`a with b with c` parses as
        // `a with (b with c)`, see the `with` handling in expr/binary.rs). This
        // matters beyond cosmetics: `Concatenation` evaluates left, then right,

Comment thread Docs/02-getting-started/hello-world.md Outdated
Comment on lines +169 to +170
> **Tip:** `with` and the space-separated form do the same job. Use whichever
> reads more clearly — mix them freely if you like.
…ed freely

The getting-started tip said to "mix them freely", but mixing the two within one
`display` is not equivalent to either pure form: `display a with b c` parses the
`with` into the first value and then folds, giving `(a with b) with c`
(left-grouped), whereas pure `display a b c` / `display a with b with c` give
`a with (b with c)` (right-grouped). With a side-effecting later value (e.g. a
list pop) the grouping changes the observed evaluation/stringification order —
confirmed at runtime: mixed prints `[before, after]after` vs `[before]after` for
both pure forms. Tighten the tip to recommend one form per statement, per the
docs-honesty policy (no overclaiming runtime behavior). Prose-only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KUmoGX1Eyt6H1BcXKg34Vu
Copilot AI review requested due to automatic review settings July 19, 2026 02:49

logbie commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator Author

Heads-up: pushed 403f2c3, rebased cleanly onto your main merge (398e486).

It resolves Copilot's docs-honesty finding on hello-world.md. The tip said with and the space-separated form can be "mixed freely," but mixing them in a single display isn't equivalent to either pure form: display a with b c parses the with into the first value and then folds, giving (a with b) with c, whereas pure display a b c / display a with b with c give a with (b with c). With a side-effecting later value the grouping changes the observed order — confirmed at runtime (mixed prints [before, after]after vs [before]after for both pure forms). Tightened the tip to "pick one form per statement" per the docs-honesty policy. Prose-only.

Verified the merged tree locally before pushing: #635's deflaked deadline test (39 passed), the display coupling/fold tests (22 passed), and cargo fmt --check — all green. CI is re-running on 403f2c3.

Copilot's other note (PR description still says "left-associative") is the same description-wording item you're holding — yours unless you'd like me to flip it.


Generated by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/parser/stmt/io.rs:437

  • The PR description says multi-value display folds values into a left-associative concatenation, but the implementation here explicitly folds right-associatively to match with (and the added tests also assert right-associative nesting). Please update the PR description to match the actual semantics so reviewers/users aren’t misled.
        // Fold right-associatively — the same tree shape, evaluation order, and
        // stringification order as explicit `with` (`a with b with c` parses as
        // `a with (b with c)`, see the `with` handling in expr/binary.rs). This
        // matters beyond cosmetics: `Concatenation` evaluates left, then right,
        // then stringifies both, so a left-associative fold would stringify an
        // earlier value *before* a later value's side effects (e.g. a list
        // mutation) run, while this matches `with` exactly, value for value.

@logbie
logbie merged commit c3f99c1 into main Jul 19, 2026
16 checks passed
@logbie
logbie deleted the claude/display-concatenation-bug-4puwvh branch July 19, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants