Repository navigation
fix(middleware): enforce database-backed rate limits with portable DDL and purge - #2913
Conversation
…L and purge
Database storage in the RateLimiter middleware never actually enforced:
$ensureTable used MySQL-only DDL, swallowed every error, and latched
tableVerified unconditionally, while $dbIncrement relied on a duplicate-key
throw that never fired because store_key only had a plain (non-unique)
index. Result: fixedWindow + database did no limiting on MySQL, and
fail-closed 429'd ALL traffic on every other engine.
Changes (all in vendor/wheels/middleware/RateLimiter.cfc):
- $queryOptions(): lazily resolve application.wheels.dataSourceName once
and pass it to all rate-limit queries, so database storage works without
a default datasource (apps with this.datasource keep working via {}).
- $ensureTable(): now boolean and fail-closed-aware. Probes for an
existing table first (accepts legacy MySQL tables with the old id
column), creates with per-engine types via $detectDatabaseType()
(copied from Job.cfc's CI-proven map; SQL Server gets DATETIME, never
TIMESTAMP/rowversion; Oracle gets VARCHAR2), re-probes once on create
failure (concurrent node), logs errors, and only latches tableVerified
on verified success. Failed attempts are throttled so a broken config
doesn't run DDL per request but can still recover.
- All three $db* strategies honor failOpen via $handleError('table
unavailable') when the table can't be verified, instead of throwing
per-request DML exceptions.
- $dbIncrement(): UPDATE-first counter algorithm (UPDATE, SELECT MAX,
INSERT only when no row, re-read once on insert race). Enforces
correctly on every engine, with or without a unique index, against old
or new table shapes. No UNIQUE index is added on purpose: the sliding
window strategy stores one row per request under the same store_key.
- $dbPurgeExpired(): throttled global purge of expired rows. Deviation
from the triage plan: the cutoff trails Now() by windowSeconds instead
of purging at Now(), because the token bucket strategy stores its
last-refill time in expires_at - purging at Now() would delete live
buckets. A bucket idle longer than windowSeconds is fully refilled, so
deleting it is semantically a no-op.
- init(): drop the verbatim duplicate windowSeconds/maxRequests
validation blocks (merge artifact, zero behavior change).
New spec vendor/wheels/tests/specs/middleware/RateLimiterDatabaseSpec.cfc
verified RED against the pre-fix code (6/8 fail or error on Lucee 7 +
SQLite) and GREEN after (8/8).
Behavioral note for the changelog: MySQL apps using storage=database
believed they had rate limiting - over-limit traffic now actually gets
429s; non-MySQL apps stop 429ing all traffic.
Deferred follow-up (to be filed as an issue): dialect-specific atomic
upsert to close the bounded first-insert race undercount, schema redesign
with a row-type discriminator, and locking for the DB tokenBucket /
slidingWindow read-modify-write races.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <peter@alurium.com>
…-create The database storage section claimed the wheels_rate_limits table is auto-created via CREATE TABLE IF NOT EXISTS; the RateLimiter now probes for the table first and creates it with engine-appropriate DDL when missing. Reword the parenthetical to match the actual mechanism. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Peter Amiri <peter@alurium.com>
There was a problem hiding this comment.
Wheels Bot — Reviewer A
TL;DR: This PR fixes a genuine, serious bug where storage="database" in RateLimiter never enforced limits on MySQL (each increment inserted a fresh row with counter=1 rather than accumulating) and self-DoS'd every other engine (MySQL-only DDL failed silently, tableVerified latched anyway, and all subsequent per-request DML against the missing table threw — with failOpen=false that 429'd everything). The fix is architecturally sound: probe-then-create DDL, UPDATE-first increment, per-engine type map, and lazy datasource resolution. Cross-engine invariants are all honored. Overall verdict: comment (no blocking issues; a few targeted nits worth addressing before merge).
Correctness
1. counter INT is nullable in the generated DDL
vendor/wheels/middleware/RateLimiter.cfc (new $ensureTable, around line 947):
CREATE TABLE wheels_rate_limits (
store_key #local.varcharType#(255) NOT NULL,
counter INT, -- nullable, no default
expires_at #local.datetimeType#
)If counter is ever NULL (edge case: external tooling, manual repair of a corrupted row), UPDATE ... SET counter = counter + 1 produces NULL on every SQL engine, and the subsequent SELECT MAX(counter) over a single-NULL-row returns NULL — IsNumeric returns false — so $dbUpdateAndCount returns -1. That triggers $dbTryInsert, which inserts a new row (no UNIQUE constraint) rather than recovering the existing one. This is handled gracefully in practice, but the schema could be more defensive:
counter INT NOT NULL DEFAULT 0,2. Non-atomic UPDATE + SELECT in $dbUpdateAndCount — missing in-code rationale comment
vendor/wheels/middleware/RateLimiter.cfc, $dbUpdateAndCount (around line 665):
QueryExecute("UPDATE wheels_rate_limits SET counter = counter + 1 WHERE store_key = :storeKey", ...);
local.qCount = QueryExecute("SELECT MAX(counter) AS counter FROM wheels_rate_limits WHERE store_key = :storeKey", ...);The PR body correctly calls this out as a bounded first-window race (deferred to #2911), but there is no in-code comment explaining the intentional non-atomicity. A future maintainer seeing a SELECT immediately after an UPDATE will instinctively try to "fix" it with a dialect-specific RETURNING or OUTPUT clause, which would break the sliding-window constraint (store_key is non-unique by design). A one-liner comment above the SELECT would preserve the reasoning:
// Non-atomic by design: a UNIQUE index is blocked by slidingWindow's one-row-per-store_key
// model. Bounded undercount on concurrent first requests is tracked in #2911.
local.qCount = QueryExecute("SELECT MAX(counter) ...");3. $dbPurgeExpired cutoff creates 2× window dead-time for fixed/sliding-window rows
vendor/wheels/middleware/RateLimiter.cfc, $dbPurgeExpired (line ~880):
{cutoff: {value: DateAdd("s", -variables.windowSeconds, Now()), cfsqltype: "cf_sql_timestamp"}}The cutoff is Now() − windowSeconds. Fixed-window rows have expires_at = Now() + windowSeconds at insert time, so they are only eligible for deletion at Now() = expires_at + windowSeconds — a full two window-lengths after they were created. Token-bucket rows store last_refill_time in expires_at, so the trailing cutoff is correct and necessary (as the comment explains). For fixed/sliding-window rows the lag is harmless functionally, but it means the table holds twice as many live rows as necessary on a busy app with a long window. Not a bug, but worth a note in the docblock for future tuning.
Cross-Engine
4. "epoch seconds" comment is incorrect on Adobe CF
vendor/wheels/middleware/RateLimiter.cfc, line 124:
// Throttle markers for database housekeeping (epoch seconds via GetTickCount() / 1000).On Lucee, GetTickCount() delegates to System.currentTimeMillis() (genuine epoch ms). On Adobe CF, it returns milliseconds since JVM startup. Dividing by 1000 gives JVM-uptime seconds on Adobe, not epoch seconds. The throttle arithmetic (nowSeconds - lastDbPurge < cleanupThrottleSeconds) is a relative comparison and works correctly on both engines regardless of which epoch the origin is. The comment misleads readers into thinking they have an absolute wall-clock value. Suggested replacement:
// Throttle markers for database housekeeping (monotone seconds via GetTickCount() / 1000;
// both values use the same GetTickCount() origin, so relative comparisons are engine-agnostic).Tests
5. Missing CHANGELOG [Unreleased] entry
The current CHANGELOG.md has no [Unreleased] section — it opens directly with the 4.0.3 release block. This fix addresses a data-loss / self-DoS regression that affects any app that deployed storage="database" since the feature shipped (#1931). It belongs in an [Unreleased] / ### Fixed entry so downstream users can identify the fix when 4.0.4 ships.
6. No test for the concurrent-first-insert re-read path
vendor/wheels/tests/specs/middleware/RateLimiterDatabaseSpec.cfc
The path where $dbTryInsert returns false (lost the race to a concurrent insert) and $dbUpdateAndCount is called a second time (lines 632–635 of RateLimiter.cfc) has no test coverage. This is a minor gap given the race requires genuine concurrency, but a unit-test using prepareMock / $mock to make the first QueryExecute INSERT throw (simulating a duplicate-key failure) and then verifying the re-read is taken would close it cheaply.
Commits
Both commit headers are well-formed and under the 100-char limit:
fix(middleware): enforce database-backed rate limits with portable DDL and purge— 79 chars ✓docs(web/guides): reword rate-limit table auto-creation to probe-then-create— 73 chars ✓
Types and scopes are from the allowlist in commitlint.config.js. ✓
What's done well
- The UPDATE-first +
SELECT MAX()algorithm correctly enforces on every engine without requiring a UNIQUE index (which would break the sliding-window one-row-per-request design). The old INSERT-first-then-catch pattern could never enforce because the duplicate-key signal required a UNIQUE constraint that wasn't there. cfdbinfois called with direct named attributes, notattributeCollection = arguments— correctly dodges the Adobe CF 2023/2025 rejection (cross-engine invariant #10 inCLAUDE.md).- The DDL string interpolation (
#local.varcharType#,#local.datetimeType#) is safe because both variables come from a hardcoded if/else chain; no user input reaches SQL. - The test spec correctly uses
var state = {blocked: "", errored: false}andstate.blocked = .../state.errored = trueinsidetry/catchblocks, matching the BoxLanglocal.Xin-catch scope invariant (#11 inCLAUDE.md). prepareMock(limiter)+limiter.$property(...)to inject a bogus datasource name is a clean way to force the table-unavailable code path without needing to tear down a real schema.- The
lastTableAttempt > 0guard in$ensureTablecorrectly avoids throttling the very first attempt (0 > 0 is false), so a fresh app doesn't delay the first table creation bycleanupThrottleSeconds.
Wheels Bot — Reviewer B (round 1)A's review is largely solid — the correctness and cross-engine findings are well-grounded. However, there is one false positive and one significant missed issue that prevent alignment. SycophancyNone detected. A used a False positivesFinding 3 (purge dead-time — "worth a note in the docblock"): The docblock for
A's request that the PR author add this note is asking for something already written. This finding should be retracted. Missed issuesRemoval of The PR silently deletes both guards: This is a regression in documented API behavior. Verdict alignmentA called this ConvergenceNot aligned. A's false positive on Finding 3 is minor and retractable, but the missed validation-removal is a real behavioral regression that elevates the appropriate verdict from |
There was a problem hiding this comment.
Wheels Bot — Reviewer A response (round 1)
B's analysis is correct on both counts. I concede the false positive on Finding 3 and concede the missed validation-removal, which is a behavioral regression that blocks merge. Updated verdict: request-changes.
Conceded points
-
Finding 3 retracted (false positive). The docblock for
$dbPurgeExpiredin the diff already reads: "For fixed/sliding window rows the extra lag is harmless." My finding asked for something already written. B is correct — retracted. -
Missed: removal of both
Wheels.RateLimiter.InvalidConfigurationguards (vendor/wheels/middleware/RateLimiter.cfc, diff hunk@@ -90,22 +90,6 @@). The PR deletes:if (arguments.windowSeconds <= 0)— the deleted comment explicitly warns this "divides by zero in the fixedWindow / tokenBucket math."if (arguments.maxRequests < 0)—maxRequests = 0as a kill-switch is preserved downstream, but negative values are now silently accepted.
CLAUDE.md§ "Rate Limiting" is explicit: "windowSecondsmust be > 0;maxRequestsmust be >= 0. Invalid values throwWheels.RateLimiter.InvalidConfigurationat construction.maxRequests = 0is a valid kill-switch." The deletion looks accidental — both guards sat immediately above thevariables.maxRequests = ...assignments and were likely swept up in the surrounding refactor. Both must be restored before merge.
Defended points
- Finding 1 (
counter INTnullable): The diff still showscounter INT,with noNOT NULL DEFAULT 0. The risk (NULL counter →$dbUpdateAndCountreturns -1 → spurious$dbTryInsertretry) is real, if minor. Not retracted. - Finding 2 (non-atomic UPDATE + SELECT rationale comment): The comment gap is still present in the diff. Without it, a future maintainer will reach for a RETURNING/OUTPUT clause that would break the sliding-window one-row-per-request model. Not retracted.
- Finding 4 (GetTickCount() comment): The
// epoch seconds via GetTickCount() / 1000comment remains in the diff. On Adobe CF,GetTickCount()is JVM-uptime milliseconds, not epoch milliseconds; the throttle comparison is still correct but the comment misleads. Not retracted. - Findings 5 and 6 (CHANGELOG and test gap): Unchanged nits — still valid.
Updated verdict
request-changes — the removal of both documented Wheels.RateLimiter.InvalidConfiguration guards is a behavioral regression (and a divide-by-zero risk for windowSeconds <= 0) that must be restored before merge.
Manually triggered baseline refresh via .github/workflows/refresh-visual-baselines.yml on branch peter/review-ratelimiter-db-enforcement. Run when an intentional content/layout change makes the visual-regression check fail. The new PNG(s) under web/tests/visual-baselines/ are now the expected rendering; re-run the failing visual-regression job to flip the check green.
The refresh-visual-baselines workflow pushed with GITHUB_TOKEN, which does not trigger pull_request workflows, leaving the required Bot PR TDD Gate check unreported on the new head. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Peter Amiri <peter@alurium.com>
There was a problem hiding this comment.
Wheels Bot — Reviewer A
TL;DR: This PR correctly fixes a serious enforcement bug in storage="database": on MySQL, every increment inserted a fresh row with counter=1 so no request was ever limited; on all other engines, the MySQL-only DDL was swallowed silently, tableVerified latched anyway, and per-request DML against the missing table threw — with failOpen=false this 429'd all traffic. The UPDATE-first + SELECT MAX() + insert-on-miss algorithm is sound, $ensureTable() is now boolean and fail-closed-aware, and the cross-engine DDL type map is borrowed from the CI-proven Job.cfc pattern. One housekeeping note: the "validation guard removal" raised in a previous review cycle is a false positive — windowSeconds <= 0 and maxRequests < 0 guards are present in the current file at lines 65–79, unchanged from develop; the PR diff only removes an older duplicate copy with simpler messages that sat after the storage/proxyStrategy checks. The two follow-up bot commits (chore(web): refresh visual baseline(s) (blog) and chore: retrigger CI after baseline-refresh bot push) introduce no code changes. Overall verdict: comment — no blocking correctness, cross-engine, or security issues; three nits worth addressing.
Correctness
1. counter INT — nullable column, no DEFAULT 0
vendor/wheels/middleware/RateLimiter.cfc, line 947:
CREATE TABLE wheels_rate_limits (
store_key #local.varcharType#(255) NOT NULL,
counter INT,
expires_at #local.datetimeType#
)counter is nullable with no default. The framework's own code paths always supply an explicit value (counter = counter + 1, or counter = 1 on insert), so a NULL counter row cannot arise from normal operation. But an externally inserted or manually repaired row with counter = NULL will confuse $dbUpdateAndCount: UPDATE ... SET counter = counter + 1 leaves NULL unchanged, SELECT MAX(counter) over a single NULL row returns NULL, IsNumeric(NULL) is false, so the function returns -1 — triggering a $dbTryInsert that inserts a duplicate row (no UNIQUE index). Suggested hardening:
counter INT NOT NULL DEFAULT 0,Cross-engine
2. GetTickCount() comment claims epoch seconds — inaccurate on Adobe CF
vendor/wheels/middleware/RateLimiter.cfc, line 124:
// Throttle markers for database housekeeping (epoch seconds via GetTickCount() / 1000).On Lucee, GetTickCount() delegates to System.currentTimeMillis() (genuine epoch milliseconds). On Adobe CF, GetTickCount() returns milliseconds since JVM startup — dividing by 1000 gives JVM-uptime seconds, not epoch seconds. The throttle comparison is a relative subtraction and works correctly on both engines because both operands share the same GetTickCount() origin. The comment is harmless in practice but misleads readers into treating the value as a wall-clock timestamp. Suggested replacement:
// Throttle markers for database housekeeping (relative seconds via GetTickCount() / 1000;
// both operands share the same origin, so relative comparisons are engine-agnostic even though
// Adobe CF's GetTickCount() is JVM-uptime-based rather than epoch-based).3. $dbUpdateAndCount non-atomic design — rationale missing from docblock
vendor/wheels/middleware/RateLimiter.cfc, lines 633–634:
} else {
// Lost the first-insert race to a concurrent request — re-read once.
local.count = $dbUpdateAndCount(arguments.storeKey);
}The $dbUpdateAndCount docblock explains the IsNumeric / MAX() sentinel but does not explain why the UPDATE and SELECT are intentionally non-atomic (a UNIQUE index that would enable a dialect-specific RETURNING/OUTPUT clause is blocked by the sliding-window strategy's one-row-per-request model). Without that note, a future maintainer will reach for a "fix" — RETURNING, OUTPUT, or a transaction — that would silently break the sliding-window path. The $ensureTable docblock already carries the one-line explanation (store_key intentionally has a plain (non-unique) index); a cross-reference in $dbUpdateAndCount's docblock would close the gap:
/**
* Increment the counter for a store key and return the resulting count.
* Returns -1 when no counter row exists yet (MAX() over zero rows returns a single
* row with NULL, so IsNumeric — not recordCount — is the reliable "no row" signal).
*
* UPDATE + SELECT is intentionally non-atomic: a UNIQUE index (needed for a
* dialect-specific RETURNING/OUTPUT) is blocked by the sliding-window strategy,
* which stores one row per request under the same store_key. The bounded first-window
* undercount under concurrent first requests is tracked in #2911.
*/Docs
4. No [Unreleased] CHANGELOG entry
CHANGELOG.md opens directly with the [4.0.3] block; there is no [Unreleased] section. This fix resolves a production self-DoS regression (non-MySQL apps using storage="database" get 429'd on every request) that affects any app using database-backed rate limiting since the feature shipped. An [Unreleased] ### Fixed entry would let downstream users identify the fix when 4.0.4 ships.
Commits
Both code-bearing commit headers are well-formed:
fix(middleware): enforce database-backed rate limits with portable DDL and purge— 79 chars, valid type + scope ✓docs(web/guides): reword rate-limit table auto-creation to probe-then-create— 73 chars, valid type + scope ✓
What's done well
- The UPDATE-first +
SELECT MAX()+ insert-on-miss algorithm enforces correctly on every engine without requiring a UNIQUE index that the sliding-window strategy would violate. cfdbinfouses direct named attributes (notattributeCollection = arguments) — correct for Adobe CF 2023/2025 (cross-engine invariant #10 inCLAUDE.md).- DDL string interpolation (
#local.varcharType#,#local.datetimeType#) is safe: both variables originate from a hardcoded if/else chain on$detectDatabaseType()output, never from user input. - The spec uses
var state = {blocked: "", errored: false}withstate.blocked = ...insidetry/catch— correct for the BoxLanglocal.X-in-catch scope invariant (#11 inCLAUDE.md). $queryOptions()lazy datasource resolution correctly avoids the construction-timeapplication.wheels.dataSourceNameavailability problem.- The
lastTableAttempt > 0guard ensures a fresh app does not delay its first table-creation attempt bycleanupThrottleSeconds.
Wheels Bot — Reviewer B (round 1)A's review is well-grounded on the updated SHA. The key factual question from the prior review cycle — whether the validation guards were deleted — is correctly resolved: the diff removes lines 94–108 of develop's SycophancyNone detected. A's "What's done well" section is earned — each point maps to a real cross-engine invariant or a known pitfall (cfdbinfo attribute style, BoxLang local-in-catch scope, DDL injection safety), and A named the specific framework invariant each satisfies. This is substantive, not flattery. False positivesNone detected. A's refutation of the prior-cycle "validation guard removal" finding is correct: Missed issuesMinor: No cross-engine, security, or behavioral regressions were missed. Verdict alignmentA's ConvergenceAligned. A's review is accurate, the false-positive from the prior SHA has been correctly retracted, and the one item B adds (the |
Summary
storage="database"in the RateLimiter middleware never actually enforced. Pre-fix,$ensureTable()(vendor/wheels/middleware/RateLimiter.cfc:762on develop) used MySQL-only DDL (CREATE TABLE IF NOT EXISTS, inlineINDEXclauses), swallowed every error, and latchedtableVerifiedunconditionally — and$dbIncrement()(:610) relied on a duplicate-key throw that could never fire becausestore_keyonly had a plain (non-unique) index. Net effect:counter=1, and no request was ever limited — apps believed they had multi-node rate limiting and had none.tableVerifiedlatched anyway, and per-request DML against the missing table threw — withfailOpen=false(the default) this 429'd all traffic, a self-inflicted denial of service.Source
Internal multi-agent framework review, 2026-06-09, finding T8 (db-ratelimiter-enforce).
Fix
All in
vendor/wheels/middleware/RateLimiter.cfc:$ensureTable()is now boolean and fail-closed-aware: probe-first (SELECT ... WHERE 1=0), per-engine DDL types via$detectDatabaseType()(ported from the CI-proven map inJob.cfc/channel/DatabaseAdapter.cfc— RateLimiter was the last sibling carrying the old pattern), separateCREATE INDEXstatements, re-probe once on create failure (concurrent node), andtableVerifiedonly latches on verified success. Failed attempts are throttled (10s) so a broken config doesn't run DDL per request but can recover. Legacy MySQL-created tables (with the oldidcolumn) are accepted by the probe and written compatibly.$dbIncrement()replaces the never-firing duplicate-key dependence with UPDATE-first +SELECT MAX(counter)+ insert-on-miss + one re-read — enforces on every engine, with or without a unique index, and against legacy tables.$db*strategies route table-unavailable through$handleError(), sofailOpenis honored both ways instead of throwing raw per-request DML exceptions.$queryOptions()lazily resolvesapplication.wheels.dataSourceNameonce and passes it to all rate-limit queries.Now() - windowSecondsheadroom; any tokenBucket row idle >= windowSeconds is fully refilled, so deletion is a semantic no-op, and fixedWindow rows are only purged after their window has rotated.Behavior changes (changelog call-outs)
storage="database"now actually get 429s — over-limit traffic that previously passed is now limited.storage="database"stop 429ing ALL traffic; the limiter now works (or honorsfailOpenif the table genuinely can't be created).application.wheels.dataSourceNameover the Application.cfc default datasource (consistent withJob.cfc); apps relying solely onthis.datasourcekeep working via the{}fallback.Deliberately deferred → #2911
UNIQUE(store_key)index is blocked by slidingWindow's one-row-per-request design).storage="database".Also reworded one stale line in
web/sites/guides/src/content/docs/v4-0-0/digging-deeper/rate-limiting.mdxthat described the oldCREATE TABLE IF NOT EXISTSmechanism (now probe-then-create).Tests
New
vendor/wheels/tests/specs/middleware/RateLimiterDatabaseSpec.cfc(8 cases): fixedWindow/slidingWindow/tokenBucket enforcement through database storage, single-row counter accumulation,$ensureTablesuccess reporting, expired-row purge, and fail-closed/fail-open behavior when the table is unavailable. Verified RED against pre-fix code on Lucee 7 + SQLite (6/8 fail or error — enforcement cases pass requests that must be limited; fail-open/closed cases hit raw DML throws) and GREEN after (8/8). The pre-existingRateLimiterSpecDB kill-switch case still passes. Local verification was Lucee 7 + SQLite only; the per-PR compat matrix is the real cross-engine gate.Cross-engine notes
cfdbinfousage mirrorsJob.cfcexactly; noattributeCollection = argumentsanywhere (Adobe 2023/2025 invariant).Left(str, 0); spec hoists closures instead of passing them inline as constructor named args; catch-state in specs uses struct fields (BoxLanglocal-in-catch invariant).DATETIME(neverTIMESTAMP/rowversion); Oracle getsVARCHAR2; type map is the CI-proven one fromJob.cfc.🤖 Generated with Claude Code