Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 52 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,59 @@
- name: Run All Integration Tests
run: cargo test --test '*' --verbose

# Database integration tests against live PostgreSQL and MariaDB servers.
# SQLite database tests need no services and already run everywhere via
# `cargo test`; this job exercises the env-gated PostgreSQL/MariaDB paths.
database-tests:
name: Database Tests (PostgreSQL + MariaDB)
runs-on: ubuntu-latest
needs: fmt
timeout-minutes: 15
services:
postgres:
image: postgres:16
env:
POSTGRES_USER: wfl
POSTGRES_PASSWORD: wfl
POSTGRES_DB: wfl_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U wfl"
--health-interval 10s
--health-timeout 5s
--health-retries 5
mariadb:
image: mariadb:11
env:
MARIADB_USER: wfl
MARIADB_PASSWORD: wfl
MARIADB_DATABASE: wfl_test
MARIADB_ROOT_PASSWORD: root
ports:
- 3306:3306
options: >-
--health-cmd "healthcheck.sh --connect --innodb_initialized"
--health-interval 10s
--health-timeout 5s
--health-retries 5
env:
WFL_TEST_POSTGRES_URL: postgres://wfl:wfl@localhost:5432/wfl_test
WFL_TEST_MYSQL_URL: mysql://wfl:wfl@localhost:3306/wfl_test
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable

- name: Cache Cargo registry and target directory
uses: Swatinem/rust-cache@v2
with:
shared-key: database-tests

- name: Run Database Tests
run: cargo test --test database_test --verbose

# Run WFL test programs to verify the interpreter works correctly
run-wfl-programs:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
name: Run WFL Programs
strategy:
matrix:
Expand Down Expand Up @@ -414,7 +465,7 @@
bump-version:
name: Bump Version
runs-on: ubuntu-latest
needs: [fmt, clippy-and-test, integration-tests, run-wfl-programs]
needs: [fmt, clippy-and-test, integration-tests, database-tests, run-wfl-programs]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
permissions:
contents: write
Expand Down
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,26 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Request objects from `wait for request` now carry `method`, `path`, `client_ip`, `body` and `headers` properties (in addition to the existing standalone variables)
- Errors in executed files (missing file, parse errors, runtime errors) are catchable in the parent with `try`/`when`, including `when file not found`
- Nesting depth guard (4 levels) protects against a file that executes itself
- Built-in database support for SQLite, PostgreSQL, and MariaDB/MySQL backed by sqlx connection pooling:
- `open database at "<url>" as db` (alias: `connect to database at ... as ...`) routed by URL scheme (`sqlite://`, `sqlite::memory:`, `postgres://`, `postgresql://`, `mysql://`, `mariadb://`)
- `store rows as query db with "<sql>" [and parameters [...]]` returns a list of row objects keyed by column name
- `store result as execute db with "<sql>" [and parameters [...]]` returns `{affected_rows, last_insert_id}` (`last_insert_id` is `nothing` on PostgreSQL — use `RETURNING`)
- `close database db`
- Parameters always bind through the database driver (never string interpolation), so SQL injection via values is not possible; placeholders are driver-native (`?` for SQLite/MariaDB, `$1` for PostgreSQL)
- Type-aware decoding: integers/floats/decimals → number, `NULL` → `nothing`, `BOOLEAN` → boolean, `BLOB`/`BYTEA` → binary, `DATE`/`TIME`/`TIMESTAMP` → date/time/datetime
- Database errors are catchable with `try`/`when error`
- Note: `store <name> as query <handle> with ...` and `store <name> as execute <handle> with ...` are now reserved statement shapes; a multi-word variable whose name starts with the word `query`, followed by a `with` concatenation, would previously have parsed as an expression
- Web route parameter helpers in the standard library:
- `path_params of <path> and "<template>"` extracts `:name` segment captures (plus trailing `*name` wildcards) as an object, or returns `nothing` on no match; captures are percent-decoded and query strings are ignored
- `path_matches of <path> and "<template>"` returns a boolean for routing conditionals
- CI job running the database test suite against live PostgreSQL 16 and MariaDB 11 service containers (`WFL_TEST_POSTGRES_URL` / `WFL_TEST_MYSQL_URL` gate the backend-specific tests)
- New documentation: `Docs/04-advanced-features/databases.md`; route-parameters section in `Docs/04-advanced-features/web-servers.md`

### Fixed
- `respond to ... with <content> and status <code> and content_type <type>` previously parsed the status as the boolean expression `<code> and content_type`, which failed at runtime and left the HTTP request unanswered; status/content_type values now parse as primary expressions
- `header "<Name>" of <request>` is now case-insensitive; warp normalizes header names to lowercase, so canonically-spelled names like `User-Agent` always returned nothing on real requests. Absent headers now compare equal to `nothing`
- The static analyzer now marks variables inside list literals (e.g. `parameters [user_name]`) as used
- `scripts/run_web_tests.sh` exited before running any test due to `set -e` combined with `((var++))` arithmetic increments

## [25.9.1] - 2025-09-20

Expand Down
4 changes: 4 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ log = "0.4.20"
rustyline = "12.0.0"
tokio = { version = "1.35.1", features = ["full"] }
reqwest = { version = "0.11.24", features = ["json"] }
sqlx = { version = "0.8.1", features = ["runtime-tokio-rustls", "sqlite", "mysql", "postgres"] }
sqlx = { version = "0.8.1", features = ["runtime-tokio-rustls", "sqlite", "mysql", "postgres", "chrono"] }
serde_json = "1.0.114"
warp = "0.3.7"
uuid = { version = "1.6.1", features = ["v4"] }
Expand Down
97 changes: 97 additions & 0 deletions Dev diary/2026-06-12-database-bindings-and-route-parameters.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Database Bindings, Route Parameters, and Web Server Fixes

**Date:** 2026-06-12

## What Changed

### Database support (SQLite, PostgreSQL, MariaDB)

The sqlx dependency that has been sitting in Cargo.toml is now exposed to WFL
programs:

```wfl
open database at "sqlite://app.db" as db
store inserted as execute db with "INSERT INTO users (name) VALUES (?)" and parameters ["Alice"]
store users as query db with "SELECT * FROM users"
close database db
```

- Backends route by URL scheme: `sqlite://` / `sqlite::memory:`,
`postgres://` / `postgresql://`, `mysql://` / `mariadb://` (MariaDB uses
the MySQL driver).
- Runtime lives in `src/interpreter/database.rs` as an explicit `DbPool` enum
(deliberately not sqlx `Any`, which has lossy type mapping). Pools hang off
`IoClient` behind string handles, exactly like file handles.
- Rows decode to lists of objects keyed by column name; SQL `NULL` maps to
the runtime value of the `nothing` literal (`Value::Null`).
- Parameters always go through driver-level `.bind()` — SQL injection via
values is structurally impossible. Placeholders are driver-native (`?` vs
`$1`); we do not rewrite SQL text.
- No new lexer keywords: `connect` and `query` are contextual with strict
lookahead, so existing programs (including variables named `query`) parse
unchanged.

### Route parameters for the web server

New stdlib helpers in `src/stdlib/web.rs`:

```wfl
store params as path_params of path and "/users/:id"
check if params is nothing:
respond to req with "Not Found" and status 404
otherwise:
respond to req with params["id"]
end check
```

`:name` captures one segment (percent-decoded), trailing `*name` captures the
rest, `path_matches` gives a boolean. Implemented as a split-segment matcher —
the pattern VM would have been overkill.

### Web server fixes (FRAMEWORK_FINAL_REPORT follow-up)

Investigating the archived framework report's blockers against live servers
turned up two real bugs, both now fixed with regression tests:

1. **respond status clause swallowed the rest of the line.**
`respond to req with "x" and status 404 and content_type "text/plain"`
parsed the status as the boolean expression `404 and content_type`, which
failed at runtime (undefined variable) and left the request unanswered —
every 404 path in the comprehensive demo was silently broken. Status and
content_type values now parse as primary expressions.
2. **Header access never matched on real requests.** warp lowercases header
names, but the lookup was exact-match, so `header "User-Agent" of req`
always returned nothing. Lookup is now case-insensitive, and absent
headers compare equal to `nothing`.

The report's headline claim ("`wait for request comes in on ...` does not
parse") no longer reproduces; parser regression tests in
`tests/main_loop_parser_test.rs` lock in the try/catch-inside-main-loop
shapes.

## Why

Strategic direction: close primitive-level gaps in the runtime (database
access first) and let higher-level framework features live as WFL packages
later, rather than building a Laravel-style framework into the core.

## Testing

- TDD throughout — every feature/fix started from a failing test.
- `tests/database_parser_test.rs`, `tests/database_test.rs` (SQLite suites run
everywhere; PostgreSQL/MariaDB suites gate on `WFL_TEST_POSTGRES_URL` /
`WFL_TEST_MYSQL_URL` and were verified against live PostgreSQL 16 and
MariaDB 10.11/11 servers).
- `tests/route_params_test.rs`, `tests/respond_statement_parser_test.rs`,
`tests/header_access_runtime_test.rs` (real warp server round-trips).
- E2E: `TestPrograms/database_sqlite_test.wfl` (runs in the standard program
suites) and `TestPrograms/web_route_params_test.wfl` (driven by
`scripts/run_web_tests.sh|ps1` with curl assertions).
- CI: new `database-tests` job with postgres:16 and mariadb:11 service
containers.

## Docs

`Docs/04-advanced-features/databases.md` (new, all examples parse-validated;
the complete example runs), route-parameters section in `web-servers.md`,
keyword reference notes for the reserved statement shapes, CHANGELOG.
Loading
Loading