Add README.md with project overview and status - #10
Merged
Conversation
Co-Authored-By: bsbyrd@logbie.com <bsbyrd@logbie.com>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
⚙️ Control Options:
|
logbie
pushed a commit
that referenced
this pull request
Jul 12, 2026
…breach Round-3 maintainer review (scoping + front-end findings): - (#9) The REPL now installs the per-command budget as the current-thread budget for the WHOLE pipeline (lex/parse/analyze/type-check/interpret), not just interpretation, so the front-end checkpoints actually see it (they saw None before, since interpret() installed the guard only for its own phase). - (#10) A shared-budget breach during type checking is propagated on a distinct fatal channel (TypeChecker::take_budget_error); the CLI consults it and stops, instead of printing it as a non-fatal type-check "warning" and continuing. - (#8) More front-end checkpoints: Analyzer::analyze (backs the type checker and every load/include/execute-file) checks the budget at its phase boundary, and parse_statement checks recursively (covering deeply nested block bodies the top-level parse loop never revisits) while advancing a token so the parser's no-progress invariant holds. All checkpoints are exemption-aware. - (P2 repl:95) Check the prospective buffer length with checked_add BEFORE push_str, so a single oversized pasted line is never copied into the buffer. - (P2 docs) Fix the detailed max_pattern_steps default (5000000). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A
logbie
added a commit
that referenced
this pull request
Jul 12, 2026
…609) * feat: add shared ExecutionBudget consolidating runtime resource caps Replace a dozen isolated, per-subsystem resource ceilings (and several that were simply unbounded) with one coherent ExecutionBudget object that travels with a run through parsing, evaluation, pattern matching, web handling, and module loading. The budget owns every dimension: - Deadline and cooperative cancellation (was Interpreter.max_duration/op_count) - Interpreter operation ceiling (new, opt-in; default unlimited) - Recursion and import depth (recursion was unguarded in release) - execute-file depth (was MAX_EXECUTE_FILE_DEPTH) - Pattern transitions and active states (was MAX_STEPS; states now bounded) - Source, request-body, and response byte caps (source/response were unbounded) - Pending HTTP requests (was web_server_request_queue_bound) - WebSocket queue and connection limits (were unbounded channels/registry) Design: - src/exec/budget.rs: ExecutionBudget is Send+Sync (atomics only), so an Arc clones into the multi-threaded web transport without any Rc/RefCell crossing a thread boundary. The interpreter core stays !Send. BudgetLimits maps the existing .wflcfg keys plus nine new budget keys; BudgetExceeded keeps the historic timeout wording/ErrorKind so existing handling matches. - The interpreter runs on a dedicated 1 GiB-stack thread so max_call_depth (default 1000) turns runaway recursion into a clean, catchable error instead of a native stack overflow (an 8 MiB stack overflows near depth 40). Backward compatibility: the three pre-existing knobs keep their defaults exactly; new ceilings default generously; max_operations is off by default. All 515 lib tests, the new budget suite, and 107 TestPrograms pass. Docs: configuration-reference (new keys + Execution budget section), web-servers (response/WebSocket limits), and a Dev Diary entry. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: address PR review — budget spans execute-file, nested source caps, catchable-error stack Follow-up fixes from automated PR review on #609: - Share the ExecutionBudget with the `execute file` child interpreter (clone the parent Arc) so the deadline, operation ceiling, and cancellation span the whole run instead of resetting per executed file. - Enforce max_source_size on nested sources (load module / include / execute file) via file metadata *before* reading; the CLI top-level check now also reads metadata first, so oversized sources are refused without allocating. - budget_error only force-clears the call stack for the terminal deadline; a catchable ResourceLimit (e.g. the recursion ceiling) leaves the stack for call_function to unwind, so the depth counter stays correct after a caught recursion error under try/when. - Pattern VM checks the active-state ceiling after each expansion (fail fast) rather than only once the next generation is fully built. - Doc/comment accuracy: PatternVM::new no longer references the removed MAX_STEPS; the VM error-map comment no longer implies deadline sampling; the CLI comment says "same config" not "same budget instance"; and the duplicate "Execution budget (resource limits)" doc heading was renamed for a unique anchor. New regression tests: nested_execute_file_source_is_size_checked and execute_file_shares_the_parent_operation_budget. Full lib suite, budget suite, execute-file suite, and 107 TestPrograms pass; fmt + clippy clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(test): use forward-slash paths in execute-file budget tests (Windows CI) WFL treats backslash as an escape character in string literals, so embedding a Windows path (from Path::display()) into an `execute file at "..."` statement broke parsing on windows-latest. Normalize embedded paths to forward slashes, which the runtime accepts on every platform. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1): one budget per run + bounded source loader for every entry point Addresses maintainer P1-4, P1-5 and related Copilot findings. - One ExecutionBudget per run: main.rs builds a single budget from the timeout-capped run_config up front and reuses it for the pre-parse source check AND the interpreter (new Interpreter::with_config_and_budget + Interpreter::budget() cancellation handle). Its deadline clock now covers lexing/parsing/analysis/interpretation, and execute-file already shares it. - Bounded source loader: read at most max_source_size+1 bytes (CLI, load module, include, execute file, and now the REPL), so an oversized source is refused without allocating it — even when metadata is unavailable, stale, or reports 0 (special files). Replaces the metadata-only enforce_source_size. - u64->usize source length converted safely (overflow = too large). - charge_operation(false) (main-loop exemption) no longer increments the operation counter, so exempt work can't trigger a post-loop Operations breach; cancellation is still honored every call. - Large interpreter stack is no longer a hard prerequisite: --help/--version run on the normal stack, and a failed large-stack reservation (tight RLIMIT_AS / 32-bit) falls back to the default stack instead of refusing to start. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1-1): dedicated RAII recursion-depth counter; don't corrupt state on catch Addresses the maintainer's P1-1 (recursion guard bypassable after a caught limit) and Copilot's count-loop-state finding. - Enforcement recursion depth now lives in a dedicated `call_depth: Cell<usize>` incremented/decremented by an RAII CallDepthGuard in call_function, separate from the diagnostic `call_stack` (which may be force-cleared). The guard restores depth on every unwind, so a caught ResourceLimit can never leave the depth under-counted and pile onto still-live native frames. - budget_error no longer mutates any interpreter state. Every budget breach is catchable by a general try/when, so the call stack, count-loop flags, and recursion depth must unwind naturally; force-clearing them (the historic timeout behavior) corrupted an enclosing count loop and under-counted depth. - interpret() resets in_count_loop/current_count/call_depth up front so an uncaught terminal breach can't leak stale state into a reused interpreter (REPL). Regression test catching_a_recursion_limit_leaves_a_consistent_interpreter: a count loop survives 3 caught recursion errors, `count` stays readable, and re-recursing stays bounded (no stack overflow). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1-2): meter pattern transitions per-instruction on one shared budget Addresses the maintainer's P1-2 and the pattern error-swallowing findings. - Transitions are now charged per *instruction* inside step()'s dispatch loop (and per negative-lookahead iteration), so an epsilon-Jump cycle or a lookaround instruction chain is bounded — not just the outer NFA waves. - ONE shared meter (ExecutionBudget::pattern_steps, reset once per top-level op) is charged by nested lookaround/lookbehind VMs too, so their work counts against the same budget instead of resetting to zero. execute/find/find_all no longer reset; CompiledPattern's top-level methods reset once. - Budget breaches PROPAGATE instead of collapsing into a non-match: the VM's find/find_all return Result, CompiledPattern's *_with_budget return Result, and the interpreter's matches/find operators plus the stdlib pattern builtins (pattern_matches/find/find_all/split) surface a catchable ResourceLimit error. - A thread-local "current budget" (set for the run in interpret()) lets the stdlib pattern builtins — whose native signature has no budget parameter — honour the run's configured max_pattern_steps/max_pattern_states. - max_pattern_steps default raised to 5_000_000 to suit per-instruction granularity (was a per-wave 100_000); ordinary patterns clear it easily. - Active-state fan-out is checked after each expansion, including in the negative-lookahead loop. Tests: pattern_step_limit_is_enforced_and_propagated / _is_catchable / patterns_run_normally_under_default_budget; budget unit tests for the shared meter and the current-budget scope guard. 107 TestPrograms + full lib pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P2): register budget keys with ConfigChecker; document real MSRV - ConfigChecker::new now knows every ExecutionBudget key (plus the previously unregistered web_server_request_queue_bound), so `--configCheck` accepts them and `--configFix` no longer strips them — which for max_operations would have silently reverted a configured ceiling back to unlimited. Round-trip test test_budget_keys_are_known_and_survive_fix covers all new keys. - Record the real MSRV: the codebase uses `let`-chains (stable in Rust 1.88), so Cargo.toml gains rust-version = "1.88" and CLAUDE.md's stale "1.75+" is corrected, resolving the reviewer's MSRV-vs-let-chains concern. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1-3): stream HTTP body, global request guard + timeout, reliable WS close Addresses maintainer P1-3 and the remaining P2 web/REPL findings. - Streaming body limit: the request-body ceiling is enforced while the body streams in (warp::body::stream + read_body_capped), so a chunked body with no Content-Length is bounded too — an oversized body gets 413 without being fully buffered. The Content-Length fast-path 413 stays for well-behaved clients. - Global in-flight cap: admission now uses the shared ExecutionBudget RequestGuard, so the accepted-request cap is enforced across all listeners (not per-server). The guard is acquired before the body is read and held until the handler responds, the client disconnects, or the request times out. - Per-request timeout: a dequeued request that is never answered is shed with 504 after web_server_response_timeout_seconds (default 300, 0 disables), freeing its slot instead of pinning it forever. - Response cap checks the borrowed Text/Binary length before duplicating into content_bytes (bounds peak allocation), then re-checks materialized fallbacks. - WebSocket close reliability: the writer sends a best-effort close frame on any exit — explicit Close, empty channel (close server dropped the sender), or a Full queue that couldn't carry Close — so `close server` always terminates the socket. Outbound send/broadcast now log on a full/closed queue (matching docs). - REPL uses a no-deadline budget so later commands aren't rejected by a session-start timeout (each other ceiling still applies; Ctrl-C interrupts). - New web_server_response_timeout_seconds config key (registered with ConfigChecker); docs updated. Full lib + web + 107 TestPrograms pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * docs: record deep-review round (P1-1..P1-5 + P2) in Dev Diary Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1): per-match pattern meter with deadline sampling; keep VM API Round-2 maintainer review (pattern findings): - P1 (mod.rs:295): the pattern transition counter no longer lives on the shared run budget. Two matches that share one Arc<ExecutionBudget> (e.g. concurrent web handlers) previously reset each other's meter, letting one grant the other unbounded extra quota. Introduce a per-top-level-match PatternMeter that owns its own step/state counters and is cloned only into nested lookaround/lookbehind VMs; a second match under the same run budget gets an independent meter. - P1 (budget.rs:519): the meter now samples the wall-clock deadline on the transition stride (with the interpreter's main-loop exemption, tracked via a new deadline_exempt flag synced by set_in_main_loop) and propagates it as PatternError::Timeout -> ErrorKind::Timeout, so a single synchronous match cannot run past timeout_seconds and reports the historic timeout error. - P2 (vm.rs:267): active-state slots are reserved/released across the current, next, and nested frontiers via one RAII StateReservation, so the ceiling counts all simultaneously-live states (including under nesting), not just one output vector. - P2 (vm.rs:203): restore the public PatternVM API — find -> Option and find_all -> Vec compatible wrappers, with separately-named fallible try_find/try_find_all entry points; direct top-level ops reset their own meter while private no-reset runners feed nested lookarounds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1): bound HTTP request lifetime; prune abandoned requests Round-2 maintainer review (inbound HTTP findings): - P1 (interpreter:5751): establish one deadline at admission and apply its remaining time to the *body read* as well as the response wait. Previously only the response wait was bounded, so an unauthenticated client could open max_pending_requests chunked uploads and trickle bytes under the size cap forever, pinning every global in-flight slot (503 for everyone). A slow body now sheds with 408; the handler wait still sheds with 504, sharing the same deadline. - P1 (interpreter:5836): on timeout the dropped oneshot closes its sender, and the interpreter now (a) skips a dequeued request whose sender is already closed (client gave up) instead of running a zombie handler, and (b) prunes closed entries from pending_responses before registering a new one, so repeated timeouts can't accumulate dead map entries or handler work. - P2 (interpreter:6429): reject composite/opaque response bodies (lists, objects, …) with a clear error instead of materializing an unbounded `{:?}` string past the response cap; take the response sender into an RAII ResponseCompletion guard up front so every fallible step in `respond` resolves the request (500 on early error) rather than leaving the client to hang until timeout. - P2 (interpreter:203): keep read_body_capped's max+1 bound exact — copy at most the remaining allowance plus one sentinel byte from a chunk and return TooLarge without appending the rest, so one large Buf::chunk() can't grow the buffer to max+chunk_len. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1): bound WebSocket queued bytes; cancellable close handshake Round-2 maintainer review (WebSocket findings): - P1 (interpreter:373): bound WebSocket memory by BYTES, not only frame count. Add a budget-derived per-message size cap (web_socket_max_message_size, default 1 MiB) enforced on inbound and outbound text frames, plus a global queued-byte ceiling (web_socket_max_queued_bytes, default 16 MiB) reserved per queued frame via an RAII WsBytePermit that releases on send/consume/shed. Previously the event and per-connection outbound queues could each hold ws_queue_bound arbitrarily-large messages, permitting OOM-scale buffering. - P2 (interpreter:354): a close frame alone did not guarantee the reader task (blocked in ws_rx.next()) or its WsConnectionGuard terminated when the peer ignored the close handshake. Give each server a watch-based cancellation channel; the reader now select!s on it so `close server` wakes every connection, with a bounded (250 ms) close-handshake timeout before the writer is force-aborted. Adds config keys web_socket_max_message_size and web_socket_max_queued_bytes (both positive-integer validated; mapped into BudgetLimits). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1): share recursion depth across execute-file boundary Round-2 maintainer review (interpreter:7232): a child interpreter spawned by `execute file` started with call_depth = 0 (and interpret() reset it again), so a parent already near max_call_depth could run a child that consumed another full allowance — and nested execute files multiplied the native stack toward 1 GiB-thread overflow before any guard fired. The child now seeds its base recursion depth from the parent's live depth (base_call_depth), and interpret() resets to that base rather than 0, so the combined WFL call depth across nested execute-file runs is bounded by max_call_depth. Regression test execute_file_shares_the_parent_recursion_depth pins that a child cannot get a fresh depth allowance. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P2): config checker validates budget keys against loader ranges Round-2 maintainer review (checker.rs:560): the budget keys were registered as generic Integer, validated with parse::<i64>() and no per-key range, so max_operations = -1 or max_call_depth = 0 passed check/fix even though the loader rejects them (leaving max_operations unlimited), and valid u64 values above i64::MAX were wrongly rejected and could be "fixed" to 0/unlimited. Integer keys are now validated as non-negative u64 (the loader's usize/u64 domain), and each budget/web key enforces the loader's exact minimum via integer_min_for_key (max_operations and web_server_response_timeout_seconds allow 0; every other ceiling requires >= 1). Also add the Execution Budget category to get_settings_by_category so the config wizard no longer omits every setting in it, and register the two new web_socket_max_message_size / web_socket_max_queued_bytes keys. New test test_budget_keys_enforce_loader_ranges. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1/P2): REPL enforces source cap pre-lex; per-command budget + Ctrl-C Round-2 maintainer review (REPL findings): - P1 (repl.rs:149): enforce the source-size ceiling immediately after appending a line to the input buffer, before it is cloned/lexed/parsed — an oversized incomplete paste was previously retained and re-cloned/re-tokenized on every following line, defeating the pre-lexing cap. Also load the applicable `.wflcfg` (via load_config_with_global) instead of WflConfig::default(), so a configured smaller max_source_size / timeout is honored. - P2 (repl.rs:49): give each command a fresh budget (new wall-clock deadline, cleared counters) via reset_command_budget while preserving the session environment, instead of disabling the deadline for the whole session — so the first runaway command still times out. Wire Ctrl-C during execution to budget.cancel() (cooperative cancellation), racing the command future against tokio::signal::ctrl_c; Ctrl-C while awaiting input is still handled by rustyline. Adds Interpreter::set_budget/config accessors and a REPL budget-reset test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1): thread the budget through the front end Round-2 maintainer review (main.rs:828): starting the budget's clock did not make lexing, parsing, analysis, or type checking consult it — none received the budget, so `--analyze` and the dump modes could finish without checking, and a slow/oversized parse could not be cancelled mid-phase; the budget only measured elapsed time for later interpretation. main.rs now installs the run budget as the current-thread budget for the ENTIRE run (source read through interpretation), so every mode consults one budget. The parser polls it once per top-level statement, the type checker once per top-level statement, and the analyzer at its phase boundary — each honoring the deadline and cooperative cancellation and aborting cleanly on breach. The source read is already bounded (read_source_bounded), so lexing/parsing are bounded by max_source_size. New test analyze_mode_consults_the_budget_in_the_ front_end proves --analyze (which never interprets) surfaces a budget breach. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * docs: document round-2 budget changes (WS bytes, request deadline, front end) - Configuration reference: add web_socket_max_message_size and web_socket_max_queued_bytes (summary + detailed), fix the stale max_pattern_steps default (5_000_000). - Web servers guide: document the single admission deadline (408 body timeout / 504 handler timeout), abandoned-request pruning, WebSocket per-message and global queued-byte bounds, and the deterministic close-server handshake. - Dev Diary: record the second deep-review round (P1×8 + P2×7). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: address round-2 bot review (deadline exemption, yield, checker) - Nested `execute file` now preserves the parent's `deadline_exempt`: the child shares the budget and its interpret() clears the shared flag, which would make a parent still inside its own `main loop` enforce the wall-clock deadline on later pattern matches and time out spuriously. Save/restore around the child. - Add a throttled cooperative `tokio::task::yield_now()` in _execute_statement (outside a `main loop`) so a tight CPU-bound count/while/repeat loop returns control to the runtime — letting the REPL's Ctrl-C → budget.cancel() actually be delivered by the select! (the per-command deadline was already the safety net; this makes manual interruption work). - Config checker: enforce the loader's minimums for timeout_seconds and web_server_max_body_size (both >= 1); extend the range test. - Fix the stale budget_error doc that claimed the deadline force-releases the call stack — the helper mutates no state (every breach unwinds naturally). Skipped two nitpicks with reason: the lookbehind run_execute/run_find_all double-scan is bounded by the generous step ceiling and collapsing it to find_at_position(0) is not semantically equivalent under greedy/non-overlapping matching; the frontier-loop extraction is a pure refactor left to avoid churn. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1): main-loop exemption as a shared depth counter + RAII guard Round-3 maintainer review (#1,#2,#3,#13): the `main loop` deadline exemption was a bool split between the interpreter (`in_main_loop`) and the budget (`deadline_exempt`), which leaked on caught errors, didn't nest, wasn't inherited across `execute file`, and was snapshotted by the pattern meter. Replace it with one shared `main_loop_depth: AtomicUsize` on the budget plus an RAII `MainLoopGuard` (enter_main_loop): - (#2) the guard restores the depth on EVERY exit — normal, early return, a caught error unwinding through `?`, and nested loops — so the exemption is never leaked or cleared while an outer loop is active. - (#3) a child `execute file` shares the budget, so the parent's active main-loop exemption covers the child and the nested front end; the parser/type-checker/analyzer checkpoints now charge with `!is_deadline_exempt()`, so `execute file` from a server's main-loop handler no longer times out in the nested parse. Removes the brittle save/restore. - (#13) PatternMeter reads the exemption LIVE per sampled stride (no `enforce_deadline` snapshot), so reusing one VM across a main-loop boundary is correct. Test reuses one meter and toggles the budget. - (#1) `_execute_statement` yields on a dedicated per-statement scheduling counter that advances even inside a `main loop` (whose op counter is exempt), so a CPU-only main loop still yields and lets the REPL Ctrl-C fire. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1): harden WebSocket + HTTP lifecycle and peak allocation Round-3 maintainer review (web findings): - (#4) Cap the WebSocket transport itself: apply max_ws_message_bytes via warp Ws::max_message_size/max_frame_size before on_upgrade, so a fragmented text frame or ignored binary frame cannot allocate up to Tungstenite's defaults on the receive side. The queued-byte permit remains the global layer. - (#6) Fail closed when the Connect event cannot be admitted: unregister the connection and close the socket instead of leaving a live socket whose connect handler (app init/auth) never ran but which could still emit Message events. - (#11) Reserve the WS queued-byte permit on the *borrowed* payload length (ws_message_byte_len) before materializing the String, on both send and broadcast, so an oversized runtime value is never fully cloned first; an oversized broadcast is rejected before it is materialized once. - (#14) Drop cleanup for WflWebServer and WflWebSocketServer: abort the accept task (and signal WS close) so a dropped/replaced/setup-errored server can't orphan a bound listener; `close server` .take()s the handle so no double-abort. - (#5) Carry the global admission RequestGuard *inside* the queued WflHttpRequest (no longer Clone), so a queued body counts against the pending ceiling until the interpreter dequeues/drops it — a timed-out route future no longer un-accounts a still-queued body. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix(P1): scope budget across REPL/front end; fatal type-check budget breach Round-3 maintainer review (scoping + front-end findings): - (#9) The REPL now installs the per-command budget as the current-thread budget for the WHOLE pipeline (lex/parse/analyze/type-check/interpret), not just interpretation, so the front-end checkpoints actually see it (they saw None before, since interpret() installed the guard only for its own phase). - (#10) A shared-budget breach during type checking is propagated on a distinct fatal channel (TypeChecker::take_budget_error); the CLI consults it and stops, instead of printing it as a non-fatal type-check "warning" and continuing. - (#8) More front-end checkpoints: Analyzer::analyze (backs the type checker and every load/include/execute-file) checks the budget at its phase boundary, and parse_statement checks recursively (covering deeply nested block bodies the top-level parse loop never revisits) while advancing a token so the parser's no-progress invariant holds. All checkpoints are exemption-aware. - (P2 repl:95) Check the prospective buffer length with checked_add BEFORE push_str, so a single oversized pasted line is never copied into the buffer. - (P2 docs) Fix the detailed max_pattern_steps default (5000000). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * docs: record round-3 review (main-loop depth, WS/HTTP lifecycle, scoping) Dev Diary: document the third deep-review round — the main-loop-depth + RAII redesign, WebSocket/HTTP lifecycle hardening, budget scoping across the REPL and front end, fatal type-check budget breaches, and the two deferred items (task-local budget and pattern text-once) with rationale. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: poll the execution budget inside recursive front-end traversal The round-3 front-end checkpoints only covered phase entry / top-level statements: `Analyzer::analyze` polled the budget once, and `TypeChecker::check_types` polled once per top-level statement, but both then recurse (`analyze_statement`, `check_statement_types`) into every nested block, loop, `try`, action, and container-method body without re-checking. A single deeply nested top-level statement could run the analyzer or type checker to completion without honoring the run's deadline / cancellation / operation budget — unlike the parser, whose checkpoint already lives inside the recursively-called `parse_statement`. Move the poll into the recursive methods, mirroring the parser: - Analyzer: `analyze_statement` polls at its top; a new `budget_exhausted` flag (reset per run) records the breach once and short-circuits the rest of the traversal so a large nested body yields one error, not one per statement. The phase-boundary poll in `analyze` is retained. - Type checker: `check_statement_types` polls at its top, guarded by the existing `budget_error` channel (reset per run) as the record-once latch. The top-level loop no longer charges separately — it stops iterating once a breach is recorded, including one surfaced deep in a nested body. Both polls stay exemption-aware. `max_operations` defaults to unlimited, so ordinary programs are unaffected; the extra per-statement charging only applies when a user opts into an operation cap. Add two library-level regression tests that parse a program whose single top-level `count` loop holds 64 nested statements before installing a budget (so the parser cannot consume the cap), then run under a tiny operation cap: one proves the analyzer trips from inside the nested body, the other uses `with_analyzer` to prove the type checker trips from the recursive `check_statement_types` poll onto the `budget_error` channel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: make a type-check budget breach a fatal typed result, not a side channel A shared-budget breach recorded during type checking was an optional side channel (`take_budget_error`): the `include from` caller printed the type errors as non-fatal warnings and then executed the included program, so a deadline / cancellation / resource breach hit while checking an included file was silently followed by running it. And when `TypeChecker::new()`'s internal `Analyzer::analyze` failed, the breach was rendered as ordinary `TypeError`s while the budget channel stayed empty — so even a caller that checked the side channel saw `None`. Make the distinction impossible to miss at the type level: `check_types` now returns `Result<(), TypeCheckError>` where `TypeCheckError::Budget` (fatal — stop the run) is separate from `TypeCheckError::Types` (ordinary diagnostics). The analyzer records its breach on a typed `budget_error` latch (`take_budget_error`), which `check_types` propagates as the fatal variant, closing the analysis-phase gap. Every caller now distinguishes the two: - `include from`: a budget breach is fatal (return the catchable resource/timeout error) instead of a printed warning; ordinary type diagnostics stay non-fatal warnings as before. - `load module`, main CLI, REPL: budget breach aborts the run; type diagnostics keep their existing handling. - LSP (report-only) renders either via `into_diagnostics()`. Tests: analysis-phase and type-check-phase breaches now assert the fatal `TypeCheckError::Budget` variant; existing type-checker/LSP call sites updated to the typed result. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * perf: collect pattern input once, checkpoint before preprocessing `PatternVM::step` re-ran `text.chars().collect()` on every transition, so a single match against a large runtime input performed O(text) allocation and copying per step. Pattern input is a runtime value that is NOT bounded by `max_source_size` (it can be an unbounded file read, HTTP/client data, or a constructed string), and `max_pattern_steps` bounds the transition count, not the O(text) work done before each charge — so a ~1 MiB input could incur enormous repeated conversions between sampled deadline checks. Materialize the input into `Vec<char>` exactly once at the top of each runner (`run_execute` / `run_find` / `run_find_all`) and thread the `&[char]` slice through `execute_at_position`, `find_at_position`, and `step`. Charge the budget once *before* that collection so an already-expired deadline or cancelled run aborts without paying to materialize a large input. Capture extraction and lookbehind slicing now reuse the same slice, and the position loops bound on `chars.len()` (character length) instead of `text.len()` (byte length), which also removes redundant no-op start positions on multibyte input. Tests: matching by character index across multibyte input; and a cancelled run aborting before materializing a 1 MiB input. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: hold the HTTP admission slot until the response completes Moving the in-flight admission guard into the queued `WflHttpRequest` fixed timed-out bodies that were still queued, but once the interpreter dequeued a request, `WaitForRequestStatement` moved only `response_sender` into `pending_responses` and dropped the local request (and its guard) when the statement returned. So the global admission gate reopened at dequeue: a handler could dequeue requests in a loop without responding, immediately readmitting new work while every route task and pending sender stayed unfinished until its timeout. Carry the admission slot with the parked response for its whole post-dequeue life. `pending_responses` now stores a `PendingResponse { sender, admission }`, and `respond` moves the slot into the `ResponseCompletion` guard, so it is released only when the response actually completes — on success, on the `Drop` 500 fallback, on disconnect/timeout pruning (a dropped pruned entry releases it), or on shutdown (the map drops). The slot now spans the full request lifetime, dequeue through response completion. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: guarantee a WebSocket Disconnect for every admitted Connect The connect event is fail-closed, so every live connection has run (or will run) its `on websocket connect` handler, which may initialize per-connection application state. The disconnect event — the only place that state is cleaned up — used a lossy `try_send`, so a momentarily-full event queue could drop it, leaking application state and leaving a queued Connect with no matching Disconnect. Deliver the disconnect with a blocking `send().await` instead: it resolves as the interpreter drains the bounded event queue, and returns `Err` only when the server has shut down (its receiver dropped), where no disconnect handler remains to run anyway. Message events keep their lossy `try_send` (dropping a message under overload is acceptable backpressure); only the lifecycle pair is made guaranteed. Under normal load the send completes immediately, so there is no behavior change except under the full-queue race this fixes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: scope the run budget task-locally so interleaved runs cannot cross-contaminate `Interpreter` is re-exported from the crate root and is `!Send`, so a library embedder can legally `join!` or `spawn_local` two interpreter futures on one thread. The current budget was installed in a thread-local held across every `.await` (the interpreter run and the REPL command), so when two runs interleaved on a thread the second overwrote the first's current budget and restored stale state — the runs cross-contaminated. Install the run budget in a `tokio::task_local!` instead, via a new `ExecutionBudget::scope(budget, future)` that wraps the run. Task-local state is per-future, so each interleaved run keeps seeing its own budget across awaits. `interpret` is now a thin wrapper that scopes the budget around the run body (`interpret_inner`); an `execute file` child nests its own scope. The REPL command is likewise wrapped in `scope`. `ExecutionBudget::enter` (thread-local) is retained only as a synchronous fallback for the CLI front-end (lex/parse/analyze/type-check), which runs to completion without awaiting on a single-future runtime and therefore cannot interleave. `current()` consults the task-local scope first and this fallback only when no async scope is active — and since every async run establishes a scope that shadows it, the fallback can never cross-contaminate a run. Test: two runs with distinct budgets interleaved via `join!` on a current-thread runtime each keep observing only their own budget across an `.await` — the property a thread-local violates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: budget-checkpoint the lexer and recursive expression parsing/analysis The front-end checkpoints only polled the budget at statement boundaries and during recursive *statement* traversal, so one huge expression — a million-element list, a long flat operator chain — was a single statement that lexed, parsed, analyzed, and type-checked to completion after the deadline. The lexer had no budget call at all, so `wfl --lex` processed the whole (source-size-capped) input past its deadline and a same-task Ctrl-C could not interrupt lexing. Add strided/recursive checkpoints across the front end: - Lexer: every 4096 tokens, consult the run budget and stop tokenizing cooperatively on a deadline/cancellation/operation breach (it returns `Vec`, so it yields the tokens gathered so far; the truncated stream then trips the parser's checkpoint). - Parser: a strided checkpoint at every primary-expression parse (the universal operand leaf — list elements, operator-chain terms, call arguments), so a single giant expression is interruptible, returning the breach as a `ParseError`. - Analyzer / type checker: poll the budget at the top of `analyze_expression` and `infer_expression_type`, latched via `budget_error`, so a huge expression tree inside one statement is interruptible during analysis and type checking. All polls are exemption-aware and no-ops when no run budget is installed, so normal programs (default `max_operations` unlimited) are unaffected. Test: a single 6000-element list literal trips the parser checkpoint under a tiny operation cap, where the old parser ran to completion. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: expose the large interpreter stack as a public helper for embedders The interpreter's async recursion is stack-heavy (~1 MiB per WFL call), so its `max_call_depth` only turns runaway recursion into a clean, catchable error on a stack large enough to reach the limit. The 1 GiB stack was set up privately in the CLI's `main`, so a library embedder driving `Interpreter::interpret` directly on an ordinary 8 MiB thread with the default `max_call_depth` (1000) could crash the host process with a native stack overflow that no depth limit can catch. Extract the large-stack thread setup into a public `wfl::run_with_interpreter_stack(work)` helper (and a `INTERPRETER_STACK_SIZE` constant), documented as the entry point embedders should wrap their runtime + `interpret` call in — the same one the CLI now uses (so the logic is shared, not duplicated). It returns `Err` only if the large stack cannot be reserved, so a caller can fall back or lower `max_call_depth`. The `Interpreter` type gains a `# Stack safety` doc section pointing embedders at it. Test: recursion deep enough to overflow the default test-thread stack completes when driven through the helper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * docs: Dev Diary entry for the fifth review round (maintainer P1s) Document the seven fixes landed for the maintainer's fourth-pass review: typed fatal type-check result, pattern collect-once, HTTP admission lifetime, guaranteed WebSocket disconnect, task-local budget scoping, lexer/expression front-end checkpoints, and the public large-stack embedder helper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: address issue #611 — lexer typed outcome, HTTP admission release, and three residuals The fifth review round's own remediation left two P1 merge blockers plus three correctness/API residuals (issue #611). Fix all five, each with a regression test. P1 — lexer must not turn a budget breach into a successful partial stream: The strided lexer checkpoint `break`'d on a breach and returned the prefix, so `wfl --lex` dumped it and exited 0, and a prefix ending at a statement boundary could parse/execute. The loop is now a private `lex_positions_core` taking a checkpoint closure: `lex_wfl_with_positions` supplies a no-op (never truncates — LSP/tooling/tests unaffected) and the new `lex_wfl_with_positions_checked` returns `Result<_, BudgetExceeded>`. Every production execution caller (CLI run and `--lex`, nested execute-file/include/load-module, REPL) propagates it. At each stride the deadline and cancellation are checked directly (not via `charge_operation`'s sampling, which nested in the 4096-token stride could postpone them by millions of tokens); the operation ceiling is charged separately. P1 — a timed-out pending HTTP request can no longer wedge admission: The guard was parked in the interpreter's pending map and pruned only when a later dequeued request was inserted; once the cap filled with unanswered requests, no new request could be admitted to trigger the prune, pinning every slot forever. The guard now rides with the warp transport future (which outlives the dequeue while awaiting the response), so the slot is held through handling and released when that future ends — respond, response timeout, or client disconnect — independently of any future admission. The bounded request mpsc still caps queued bodies. P2 — analyzer entry-time breach preserves the typed channel: the phase-boundary checkpoint now stores `budget_error` before returning, so `TypeChecker::new()` sees a cancelled/exhausted entry breach as fatal `TypeCheckError::Budget`, not misclassified `Types`. P2 — Unicode lookbehind stays in character indices: the full-slice test compared `MatchResult::end` (chars) against `text_slice.len()` (bytes), inverting positive/negative lookbehind on multibyte input; it now compares the slice's character length. The public `CompiledPattern::find` doc example that sliced `&text[m.start..m.end]` with char offsets is fixed to extract by chars. P2 — the default public interpreter path is stack-safe: `Interpreter::new()` now caps recursion at the conservative `DEFAULT_EMBED_CALL_DEPTH` (12) instead of the config default (1000), which only stays catchable on the CLI's 1 GiB stack; `with_config`/`with_config_and_budget` still honor the configured depth so the CLI reaches 1000. A deep program on an ordinary stack now returns a catchable call-depth error instead of aborting. Tests: typed lexer outcomes (cancel/deadline/operations), no-truncation, and `--lex` exit; a live-server admission-reopens-after-timeout regression; a cancelled-budget analyzer entry test; positive/negative Unicode lookbehind over a 2-byte char; and an ordinary-8-MiB-stack embedding test. Dev Diary updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A * fix: consult the run budget at the lexer boundary, not only on a full stride The strided lexer checkpoint fired only at token 4096, 8192, … so a source shorter than one stride never consulted the budget: an already cancelled or already expired budget lexed to a successful `Ok`. This was most visible in `--lex`, which has no later parser/analyzer checkpoint and so could dump a short source and exit 0 under an already-breached run budget. `lex_positions_core` now invokes the checkpoint once before the first token (catching an already-breached budget for any input length) and once after the last (catching a breach during the final partial stride), keeping the strided checks for long inputs. The non-budgeted `lex_wfl_with_positions` still supplies a no-op checkpoint, so it is unaffected. Tests: `lex_wfl_with_positions_checked("display 1\n")` under an already-cancelled budget and under an already-expired deadline both abort with the typed breach at the entry checkpoint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y498gapRYjXfGSciauq49A --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a README.md file to the project with an overview of WFL, its current status, and usage instructions.
Link to Devin run: https://app.devin.ai/sessions/8f29314c0a8c434fbd9d247f479fcff9
Requested by: bsbyrd@logbie.com