Repository navigation
feat: disable ops and resources sanitizers by default in deno test - #33250
Conversation
…33158) ## Summary Changes the default for `sanitizeOps` and `sanitizeResources` from `true` to `false` in `Deno.test()`. This aligns with what most users expect -- tests no longer fail due to leaked async ops or resources unless explicitly opted in. ### Before ```ts // This test FAILS by default -- the timer leak triggers the ops sanitizer Deno.test("my test", () => { setTimeout(() => {}, 1000); }); ``` ``` ERRORS my test => ./main.ts:1:6 error: Leaks detected: - A timer was started in this test, but never completed. ``` ### After ```ts // This test PASSES by default -- no sanitizers enabled Deno.test("my test", () => { setTimeout(() => {}, 1000); }); ``` To re-enable sanitizers, users have three options: **1. CLI flags (e.g. for CI)** ```sh deno test --sanitize-ops --sanitize-resources ``` **2. Environment variables (e.g. for CI)** ```sh DENO_TEST_SANITIZE_OPS=1 DENO_TEST_SANITIZE_RESOURCES=1 deno test ``` **3. Per-test opt-in** ```ts Deno.test({ name: "my test", sanitizeOps: true, sanitizeResources: true, fn() { setTimeout(() => {}, 1000); }, }); ``` ### What's unchanged - `sanitizeExit` default remains `true` - Per-test `sanitizeOps: true` / `sanitizeResources: true` continues to work - Test steps still inherit sanitizer settings from their parent test Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add `test.sanitizeOps` and `test.sanitizeResources` options to deno.json(c) for configuring test sanitizers globally. Also add `Deno.test.sanitizer()` API for module-level sanitizer configuration. Precedence (highest to lowest): 1. Per-test options (sanitizeOps/sanitizeResources on Deno.test()) 2. Module-level (Deno.test.sanitizer()) 3. CLI flags (--sanitize-ops/--sanitize-resources) 4. Environment variables (DENO_TEST_SANITIZE_OPS/DENO_TEST_SANITIZE_RESOURCES) 5. Config file (deno.json test.sanitizeOps/test.sanitizeResources) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tests for deno.json sanitizer config: - sanitizeOps enabled in config catches timer leaks - sanitizeResources enabled in config catches resource leaks - no config means sanitizers are off (default) - both sanitizers enabled in config Tests for Deno.test.sanitizer() module-level API: - module-level ops sanitizer catches timer leaks - module-level resources sanitizer catches resource leaks - per-test sanitizeOps: false overrides module-level - module-level sanitizer overrides config/CLI defaults - module-level can disable what CLI enables Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Is there demand for this change. Genuine question. I personally prefer how it is already. |
Yes, you can't run most of tests using npm packages without sanitizer tripping. At the end of the end very few people care about that; they're mostly useful for internal code. |
lunadogbot
left a comment
There was a problem hiding this comment.
Read the precedence chain end-to-end (flags.rs -> WorkspaceTestOptions::resolve in cli/args/mod.rs -> Deno[Deno.internal].testSanitize* injected by cli/tools/test/mod.rs:configure_main_worker -> 40_test.js:testInner defaults that read moduleSanitizeOps ?? Deno[Deno.internal].testSanitizeOps ?? false -> per-test override via {...defaults, ...userOptions}). The five-level precedence the body advertises does work in the order listed, with one important asymmetry I'd want explicit before merging.
The CLI/env/config layer is enable-only. In WorkspaceTestOptions::resolve the resolved value is test_flags.sanitize_ops || env_var || config_sanitize_ops. Any one of those three setting true wins; none of them can set false. So the user pattern "my deno.json sets sanitizeOps: true but I want to disable it for one CLI invocation" doesn't work -- there is no --no-sanitize-ops flag, and DENO_TEST_SANITIZE_OPS=0 doesn't disable it (the check is == Some("1"), anything else is treated as unset). The module-level and per-test layers in JS can disable (Deno.test.sanitizer({ ops: false }) works, and per-test sanitizeOps: false works), so the only escape hatch from a config-enabled sanitizer is to add an opt-out at the test source. That's probably fine for the v2.8 ergonomics, but worth either documenting or adding --no-sanitize-ops for symmetry.
Other things I checked and consider fine:
Deno[Deno.internal].testSanitizeOpsis only set when truthy (the Rust side doesif options.sanitize_ops { execute_script_static(...) }), so the JS?? falsebottom default kicks in correctly.Deno.test.sanitizer()overwrites the module-level value on each call -- the helper text in the dts doesn't say "call once at top of file", so a user who calls it after registering some tests would have those earlier tests pick up the old defaults. Not a bug, just worth a one-liner in the dts that recommends calling at module top.- Default flip from
truetofalseforsanitizeOps/sanitizeResourcesinlib.deno.ns.d.tsmatches the runtime change.
LSP-driven test runs hard-code sanitize_ops: false, sanitize_resources: false in cli/lsp/testing/execution.rs:333 -- intentional, since the LSP doesn't surface sanitizer failures usefully, but worth a code comment so it doesn't get "fixed" later.
|
Before landing enable sanitizer on all internal unit tests we run |
lunadogbot
left a comment
There was a problem hiding this comment.
Re-reviewed at 167d1c38. The new commit enables sanitizeOps/sanitizeResources in tests/config/deno.json so Deno's own internal test suite keeps catching real ops/resource leaks even though the user-facing default flips to off. Keys match the serde renames in SerializedTestConfig (sanitizeOps/sanitizeResources), so this is correctly wired.
My earlier COMMENTED notes (no --no-sanitize-ops symmetry; recommend calling Deno.test.sanitizer() at module top in the .dts; comment the LSP hard-coded false) were optional polish, not blockers. Happy to land as-is.
fibibot
left a comment
There was a problem hiding this comment.
Substance walks. The five-layer precedence (per-test → module-level → CLI → env → config) does work in the documented order, with one important asymmetry that @lunadogbot already flagged in their initial COMMENT and waved through as optional polish — I'd lean toward making it a blocker rather than nice-to-have. Also one validation gap on the new Deno.test.sanitizer() API and a documentation/implementation drift in the PR body.
Substantive observations inline.
This is a default-flip on Deno.test() (sanitizeOps/sanitizeResources from true → false) plus three new public surfaces (Deno.test.sanitizer() JS API, --sanitize-ops/--sanitize-resources CLI flags, DENO_TEST_SANITIZE_OPS/DENO_TEST_SANITIZE_RESOURCES env vars, test.sanitizeOps/test.sanitizeResources config keys). That's escalation territory regardless of CI state — this is a v2.8 ergonomics change that affects every existing user.
CI
101 pass, 14 pending, 0 fail. Holding COMMENT per CI-must-complete; substance otherwise tracking @lunadogbot's APPROVE.
| no_run: test_flags.no_run, | ||
| shuffle: test_flags.shuffle, | ||
| trace_leaks: test_flags.trace_leaks, | ||
| sanitize_ops: test_flags.sanitize_ops |
There was a problem hiding this comment.
The CLI/env/config layer is enable-only. test_flags.sanitize_ops || env_var || config_sanitize_ops means any one setting true wins; none can set false.
Concrete failure mode:
// deno.json
{ "test": { "sanitizeOps": true } }User wants to disable sanitizers for one CLI invocation:
deno test --no-sanitize-ops # ← no such flag
DENO_TEST_SANITIZE_OPS=0 deno test # ← treated as "unset", not as "disable" (the check is == Some("1"))
The only escape hatch is editing the test source (per-test or module-level Deno.test.sanitizer({ ops: false })). That's surprising for users who set defaults in deno.json and expect CLI to override.
@lunadogbot flagged this in their initial review and marked it as optional polish; I'd argue it's actually a blocker because the docs in the PR body claim "Precedence (highest to lowest): ... 3. CLI flags ... 5. Config file" — which is not what the implementation does. CLI can't override config-set-true. That's the kind of doc-vs-implementation drift users hit and file issues about.
Minimal fix: add --no-sanitize-ops / --no-sanitize-resources clap_overrides_with flags that flip a Option<bool> instead of plain bool, and resolve as test_flags.sanitize_ops_override.unwrap_or(env_var || config_sanitize_ops). Then both directions of override work.
Alternatively: document explicitly that the CLI/env/config layer is enable-only and only the JS-side per-test/module-level layers can disable. That at least makes the PR body match reality.
| registerHook("afterAll", fn); | ||
| }; | ||
|
|
||
| test.sanitizer = function (options) { |
There was a problem hiding this comment.
Deno.test.sanitizer() doesn't validate its input. Three failure modes:
Deno.test.sanitizer(); // → TypeError: Cannot read properties of undefined (reading 'ops')
Deno.test.sanitizer(null); // → TypeError: Cannot read properties of null (reading 'ops')
Deno.test.sanitizer("true"); // → silent no-op (string has no `.ops` / `.resources`)Since this is a new public API on Deno.test, validation matches the rest of the codebase's conventions:
if (typeof options !== "object" || options === null) {
throw new TypeError("options must be an object");
}Or webidl.requiredArguments + webidl.converters.any shape if you want the same validation rigor as Deno.test() itself.
Also worth a comment in the .dts file recommending Deno.test.sanitizer() is called at module top before any Deno.test() registrations — calling it after registers tests with the stale defaults silently. @lunadogbot raised this too.
| @@ -331,6 +331,8 @@ impl TestRun { | |||
| filter, | |||
| shuffle: None, | |||
| trace_leaks: false, | |||
There was a problem hiding this comment.
Pre-existing concern from @lunadogbot's earlier COMMENT — the LSP hard-codes sanitize_ops: false, sanitize_resources: false (intentional, since LSP-driven test runs don't usefully surface sanitizer failures). Worth a one-line comment so this doesn't get "fixed" later by someone who doesn't know the rationale.
// LSP test runs intentionally disable sanitizers — the LSP UI doesn't
// surface sanitizer failures usefully and they'd just generate noise.
sanitize_ops: false,
sanitize_resources: false,
fibibot
left a comment
There was a problem hiding this comment.
Substance walks cleanly. The default flip (sanitizeOps/sanitizeResources true → false) at cli/js/40_test.js:268–274 lands via the ?? chain moduleSanitizeOps ?? Deno[Deno.internal].testSanitizeOps ?? false, which correctly implements the documented precedence (per-test → module → CLI/env → config → built-in default). The reland additions (config-file support in cli/schemas/config-file.v1.json:927–940, Deno.test.sanitizer({ ops, resources }) API at 40_test.js:434–443, CLI flags + env-var plumbing through TestFlags → TestSpecifierOptions → worker bootstrap) are wired consistently across all the touch points (LSP execution path included at cli/lsp/testing/execution.rs:333–334).
CI 134/137 success at this head, no failures. Updated spec-test fixtures (5 files) reflect the new defaults — the previously-checking-sanitizer-output tests now opt in via --sanitize-ops or per-test options, which is exactly the migration the PR establishes for users.
Escalation: every trigger fires here
This isn't a fibibot-autonomous APPROVE. The change crosses the autonomous-bar in five distinct ways:
- Breaking change to default behavior of a stable public API (
Deno.test()) — sanitizers default flips fromtruetofalse. Existing tests that relied on the implicit default (rather than explicitly settingsanitizeOps: true) will silently stop catching leaks after upgrade. - New CLI flags:
--sanitize-ops,--sanitize-resourcesondeno test. - New env vars:
DENO_TEST_SANITIZE_OPS,DENO_TEST_SANITIZE_RESOURCES. - New public Deno API:
Deno.test.sanitizer(options)module-level setter. - New config-file surface:
test.sanitizeOps/test.sanitizeResourcesindeno.json(c).
Given this is a reland of #33158 (which was reverted in #33215), the breaking-default-flip part has presumably already been through one round of maintainer review. But the new in this reland items (config-file fields, module-level API, formal precedence chain) are post-revert additions that need fresh eyes. Worth pinging @nathanwhit, who has the most recent ownership of cli/tools/test/mod.rs and cli/js/40_test.js, on the precedence-chain semantics specifically — the ?? cascade is correct as written, but the user-facing precedence doc in the PR body is the public contract, and any divergence between the doc and 40_test.js:268 would be a long-tail bug.
One small substantive question worth answering before merge: is Deno.test.sanitizer({ ops: false }) additive with --sanitize-ops? Reading testInner's default chain, moduleSanitizeOps = false (set explicitly via Deno.test.sanitizer({ ops: false })) takes precedence over Deno[Deno.internal].testSanitizeOps = true (set by --sanitize-ops). So a module-level false overrides a CLI --sanitize-ops, which is opposite to the typical "CLI > config" mental model. That matches your stated precedence chain (module > CLI), but it's worth being explicit about it in lib.deno.ns.d.ts's docstring so users don't get caught out.
- validate Deno.test.sanitizer() argument and throw TypeError on non-object input (previously crashed with a confusing "Cannot read properties of undefined" on undefined/null, and silently no-op'd on strings or numbers) - clarify in WorkspaceTestOptions::resolve that the CLI/env/config layer is enable-only — opting out requires the JS-side per-test or module-level API - comment LSP test runner's hard-coded `sanitize_ops: false` / `sanitize_resources: false` so the rationale isn't lost - document in the .dts that Deno.test.sanitizer() should be called at module top, since tests registered before the call inherit the previous defaults - add spec test covering the new input validation
Deno Individual Contributor License AgreementAll contributors have signed the CLA. Thank you! This is an automated message from CLA Assistant |
|
Addressed @fibibot's three inline review comments on top of @bartlomieju's branch, plus a related .dts docstring note that @lunadogbot raised in their initial review:
Verified locally: |
|
CI all green on @bartlomieju — ready for merge whenever you're ready. Happy to follow up with a separate PR adding |
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as off-topic.
This comment was marked as off-topic.
fibibot
left a comment
There was a problem hiding this comment.
Three inline comments from my prior review at 167d1c38 are addressed at 4e5fb12:
cli/args/mod.rs:219-223— five-line comment now spells out the enable-only nature of the CLI/env/config layers and points at the JS opt-out (Deno.test.sanitizer({ ops: false })or per-testsanitizeOps: false).cli/js/40_test.js:437-441—Deno.test.sanitizer()validates input and throwsTypeError("options must be an object")onundefined/null/non-object.cli/lsp/testing/execution.rs:334-336— added the rationale comment for LSP hard-codingsanitize_{ops,resources}: false.
New tests/specs/test/sanitizer_module_level/invalid_options.ts exercises the TypeError throw for no-arg, null, undefined, string, and number inputs.
lib.deno.ns.d.ts:1261-1264 also picked up a "call before Deno.test() registrations" note that matches the implementation — moduleSanitizeOps/moduleSanitizeResources are captured per-registration in the ?? chain at 40_test.js:271-274, so tests registered before a sanitizer() call use the prior defaults.
CI fully green at this head.
|
Gentle bump @bartlomieju — this has been mergeable: clean with 2 approvals and all 134 required checks green since 2026-05-14. Happy to rebase if it's gotten stale; otherwise ready for merge whenever you have a moment. |
|
@bartlomieju — friendly nudge, this is still sitting mergeable: clean with 2 approvals and all 134 required checks green since the 14th. Happy to rebase if it'''s gotten stale, but otherwise ready for merge whenever you have a moment. |
|
I think the main issue here was there was no way to easily globally turn this off. The introduction of the new |
…enoland#33250) Reland of denoland#33158, which was reverted in denoland#33215 to be relanded for Deno v2.8. The fixes from denoland#33215 (internal timers for TLS handshake, sanitizer test fixes) are already in main. --- Changes the default for `sanitizeOps` and `sanitizeResources` from `true` to `false` in `Deno.test()`. This aligns with what most users expect -- tests no longer fail due to leaked async ops or resources unless explicitly opted in. ## New in this reland ### Config file support Sanitizers can now be configured in `deno.json(c)`: ```jsonc { "test": { "sanitizeOps": true, "sanitizeResources": true } } ``` ### Module-level API `Deno.test.sanitizer()` allows configuring sanitizers per-module: ```ts // Enable sanitizers for all tests in this file Deno.test.sanitizer({ ops: true, resources: true }); Deno.test("my test", () => { // ops and resources sanitizers are enabled }); Deno.test({ name: "override per-test", sanitizeOps: false, fn() { // this test opts out of ops sanitizer }, }); ``` ### Precedence (highest to lowest) 1. Per-test options (`sanitizeOps`/`sanitizeResources` on `Deno.test()`) 2. Module-level (`Deno.test.sanitizer()`) 3. CLI flags (`--sanitize-ops`/`--sanitize-resources`) 4. Environment variables (`DENO_TEST_SANITIZE_OPS`/`DENO_TEST_SANITIZE_RESOURCES`) 5. Config file (`deno.json` `test.sanitizeOps`/`test.sanitizeResources`) --------- Co-authored-by: crowlbot <280062030+crowlbot@users.noreply.github.com>
Reland of #33158, which was reverted in #33215 to be relanded for Deno v2.8.
The fixes from #33215 (internal timers for TLS handshake, sanitizer test fixes) are already in main.
Changes the default for
sanitizeOpsandsanitizeResourcesfromtruetofalseinDeno.test(). This aligns with what most users expect -- tests no longer fail due to leaked async ops or resources unless explicitly opted in.New in this reland
Config file support
Sanitizers can now be configured in
deno.json(c):{ "test": { "sanitizeOps": true, "sanitizeResources": true } }Module-level API
Deno.test.sanitizer()allows configuring sanitizers per-module:Precedence (highest to lowest)
sanitizeOps/sanitizeResourcesonDeno.test())Deno.test.sanitizer())--sanitize-ops/--sanitize-resources)DENO_TEST_SANITIZE_OPS/DENO_TEST_SANITIZE_RESOURCES)deno.jsontest.sanitizeOps/test.sanitizeResources)