Skip to content

Follow-up: implement features behind 4 skipped aspirational tests + fix 10 broken keyword_reference docs examples #555

Description

@logbie

Follow-up tracking issue from the TestPrograms triage on branch claude/test-failures-triage-9f7pys (see Dev diary/2026-07-03-testprograms-triage-and-fixes.md).

During that triage, wfl was fixed to exit nonzero on runtime errors (exit 1) and parse errors (exit 2) — previously both exited 0, so the programs below were "passing" in CI without ever parsing or completing. They now carry a // CI-SKIP: <reason> first-line header so the suite stays green, but they should be unskipped once the work below is done.

1. Aspirational tests exercising unimplemented features

Each of these tests was written TDD-style against features that don't exist yet. Remove the CI-SKIP header once the feature lands.

  • TestPrograms/web_server_session_test.wfl — session middleware:
    • configure sessions on <server> with timeout <ms> and storage <kind>
    • enable csrf protection on <server>
    • enable secure cookies on <server>
  • TestPrograms/web_server_websocket_test.wfl — websocket support:
    • listen for websockets on port <port> as <server>
    • websocket connection/broadcast handling
    • also uses create list with "a", "b", "c" (comma-separated create-list-with form)
  • TestPrograms/direct_index_comprehensive.wfl — direct index syntax from PR Enhances Language Features and Documentation Structure #135 plus container forms:
    • direct indexing: numbers 0, chained nested 0 0, func_result 0, test_obj.items 0
    • create action <name>: blocks, bare add "a" inside them
    • store result as create list <name>: ... end list (block as expression)
    • create instance <name> from <Container>, test_obj initialize method-call form
    • when error occurs: clause form
  • TestPrograms/error_handling_comprehensive.wfl — advanced error handling:
    • finally: blocks on try
    • error objects: catch with error_info: exposing error_info.type / .message / .line
    • a runtime-catchable "invalid pattern" path (the current test expects a parse error to be caught at runtime, which can't work — needs rethinking)

2. Broken docs examples (TestPrograms/docs_examples/keyword_reference/)

10 of the 11 example files have pre-existing parse errors despite the "ALL code examples MUST be validated with MCP tools" policy (_meta/manifest.json claims they were validated). They were masked by the old exit-0-on-parse-error behavior. Only pattern_examples.wfl actually runs.

Common problems:

  • reserved keywords used as variable names: status, content, command, process, port, server, test, empty
  • unsupported forms: define container ... (vs create container), create list called <name>, bare add "x" outside create-list blocks, property declarations in the wrong context

Files to fix and re-validate (then remove the CI-SKIP headers and re-run python scripts/validate_docs_examples.py):

  • comparison_examples.wfl
  • containers_examples.wfl
  • contextual_keywords_examples.wfl
  • control_flow_examples.wfl
  • declaration_examples.wfl
  • error_handling_examples.wfl
  • file_io_examples.wfl
  • operations_examples.wfl
  • process_examples.wfl
  • web_network_examples.wfl

Since these mirror Docs/reference/keyword-reference.md and Docs/reference/reserved-keywords.md, decide per case whether the example is wrong (fix the example) or whether the keyword should be contextual (fix the parser and both reference docs together, per the two-tier policy).

Also worth noting

  • The multi-token-operator token-eating issue in parse_binary_expression (operator tokens are consumed during detection, before the precedence check can reject them) is still latent for exotic nestings; the precedence rebalance removed the common and-chain case.
  • Web-server tests that need a real HTTP client (simple_web_server, test_request_properties, test_simple_static, test_static_files, test_web_server_response, web_server_request_response_test, web_server_middleware_test, web_server_content_length_test, simple_timeout_test, header_access_test, web_server_example, comprehensive_web_server_demo, web_server_comprehensive_test) are CI-SKIPped; extending scripts/run_web_tests.sh to drive more of them with curl would restore coverage.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions