Skip to content

feat: disable ops and resources sanitizers by default in deno test - #33250

Merged
crowlKats merged 8 commits into
mainfrom
reland/disable-sanitizers-default
May 20, 2026
Merged

crowlKats merged 8 commits into
mainfrom
reland/disable-sanitizers-default

Conversation

@bartlomieju

@bartlomieju bartlomieju commented Apr 12, 2026 •

Copy link
Copy Markdown
Member

Reland of #33158, which was reverted in #33215 to be relanded for Deno v2.8.

The fixes from #33215 (internal timers for TLS handshake, sanitizer test fixes) are already in main.


Changes the default for sanitizeOps and sanitizeResources from true to false in Deno.test(). This aligns with what most users expect -- tests no longer fail due to leaked async ops or resources unless explicitly opted in.

New in this reland

Config file support

Sanitizers can now be configured in deno.json(c):

{
  "test": {
    "sanitizeOps": true,
    "sanitizeResources": true
  }
}

Module-level API

Deno.test.sanitizer() allows configuring sanitizers per-module:

// Enable sanitizers for all tests in this file
Deno.test.sanitizer({ ops: true, resources: true });

Deno.test("my test", () => {
  // ops and resources sanitizers are enabled
});

Deno.test({
  name: "override per-test",
  sanitizeOps: false,
  fn() {
    // this test opts out of ops sanitizer
  },
});

Precedence (highest to lowest)

  1. Per-test options (sanitizeOps/sanitizeResources on Deno.test())
  2. Module-level (Deno.test.sanitizer())
  3. CLI flags (--sanitize-ops/--sanitize-resources)
  4. Environment variables (DENO_TEST_SANITIZE_OPS/DENO_TEST_SANITIZE_RESOURCES)
  5. Config file (deno.json test.sanitizeOps/test.sanitizeResources)

bartlomieju and others added 4 commits April 12, 2026 13:11
…33158)

## Summary

Changes the default for `sanitizeOps` and `sanitizeResources` from
`true` to `false` in `Deno.test()`. This aligns with what most users
expect -- tests no longer fail due to leaked async ops or resources
unless explicitly opted in.

### Before

```ts
// This test FAILS by default -- the timer leak triggers the ops sanitizer
Deno.test("my test", () => {
  setTimeout(() => {}, 1000);
});
```

```
ERRORS

my test => ./main.ts:1:6
error: Leaks detected:
  - A timer was started in this test, but never completed.
```

### After

```ts
// This test PASSES by default -- no sanitizers enabled
Deno.test("my test", () => {
  setTimeout(() => {}, 1000);
});
```

To re-enable sanitizers, users have three options:

**1. CLI flags (e.g. for CI)**
```sh
deno test --sanitize-ops --sanitize-resources
```

**2. Environment variables (e.g. for CI)**
```sh
DENO_TEST_SANITIZE_OPS=1 DENO_TEST_SANITIZE_RESOURCES=1 deno test
```

**3. Per-test opt-in**
```ts
Deno.test({
  name: "my test",
  sanitizeOps: true,
  sanitizeResources: true,
  fn() {
    setTimeout(() => {}, 1000);
  },
});
```

### What's unchanged

- `sanitizeExit` default remains `true`
- Per-test `sanitizeOps: true` / `sanitizeResources: true` continues to
work
- Test steps still inherit sanitizer settings from their parent test

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add `test.sanitizeOps` and `test.sanitizeResources` options to deno.json(c)
for configuring test sanitizers globally. Also add `Deno.test.sanitizer()`
API for module-level sanitizer configuration.

Precedence (highest to lowest):
1. Per-test options (sanitizeOps/sanitizeResources on Deno.test())
2. Module-level (Deno.test.sanitizer())
3. CLI flags (--sanitize-ops/--sanitize-resources)
4. Environment variables (DENO_TEST_SANITIZE_OPS/DENO_TEST_SANITIZE_RESOURCES)
5. Config file (deno.json test.sanitizeOps/test.sanitizeResources)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tests for deno.json sanitizer config:
- sanitizeOps enabled in config catches timer leaks
- sanitizeResources enabled in config catches resource leaks
- no config means sanitizers are off (default)
- both sanitizers enabled in config

Tests for Deno.test.sanitizer() module-level API:
- module-level ops sanitizer catches timer leaks
- module-level resources sanitizer catches resource leaks
- per-test sanitizeOps: false overrides module-level
- module-level sanitizer overrides config/CLI defaults
- module-level can disable what CLI enables

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@bartlomieju bartlomieju modified the milestones: 3.0, 2.8.0 Apr 12, 2026
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@bartlomieju bartlomieju changed the title reland: feat: disable ops and resources sanitizers by default in deno test feat: disable ops and resources sanitizers by default in deno test Apr 12, 2026
@iuioiua

iuioiua commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

Is there demand for this change. Genuine question. I personally prefer how it is already.

@bartlomieju

Copy link
Copy Markdown
Member Author

Is there demand for this change. Genuine question. I personally prefer how it is already.

Yes, you can't run most of tests using npm packages without sanitizer tripping. At the end of the end very few people care about that; they're mostly useful for internal code.

@lunadogbot lunadogbot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the precedence chain end-to-end (flags.rs -> WorkspaceTestOptions::resolve in cli/args/mod.rs -> Deno[Deno.internal].testSanitize* injected by cli/tools/test/mod.rs:configure_main_worker -> 40_test.js:testInner defaults that read moduleSanitizeOps ?? Deno[Deno.internal].testSanitizeOps ?? false -> per-test override via {...defaults, ...userOptions}). The five-level precedence the body advertises does work in the order listed, with one important asymmetry I'd want explicit before merging.

The CLI/env/config layer is enable-only. In WorkspaceTestOptions::resolve the resolved value is test_flags.sanitize_ops || env_var || config_sanitize_ops. Any one of those three setting true wins; none of them can set false. So the user pattern "my deno.json sets sanitizeOps: true but I want to disable it for one CLI invocation" doesn't work -- there is no --no-sanitize-ops flag, and DENO_TEST_SANITIZE_OPS=0 doesn't disable it (the check is == Some("1"), anything else is treated as unset). The module-level and per-test layers in JS can disable (Deno.test.sanitizer({ ops: false }) works, and per-test sanitizeOps: false works), so the only escape hatch from a config-enabled sanitizer is to add an opt-out at the test source. That's probably fine for the v2.8 ergonomics, but worth either documenting or adding --no-sanitize-ops for symmetry.

Other things I checked and consider fine:

  • Deno[Deno.internal].testSanitizeOps is only set when truthy (the Rust side does if options.sanitize_ops { execute_script_static(...) }), so the JS ?? false bottom default kicks in correctly.
  • Deno.test.sanitizer() overwrites the module-level value on each call -- the helper text in the dts doesn't say "call once at top of file", so a user who calls it after registering some tests would have those earlier tests pick up the old defaults. Not a bug, just worth a one-liner in the dts that recommends calling at module top.
  • Default flip from true to false for sanitizeOps/sanitizeResources in lib.deno.ns.d.ts matches the runtime change.

LSP-driven test runs hard-code sanitize_ops: false, sanitize_resources: false in cli/lsp/testing/execution.rs:333 -- intentional, since the LSP doesn't surface sanitizer failures usefully, but worth a code comment so it doesn't get "fixed" later.

@bartlomieju

Copy link
Copy Markdown
Member Author

Before landing enable sanitizer on all internal unit tests we run

@lunadogbot lunadogbot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 167d1c38. The new commit enables sanitizeOps/sanitizeResources in tests/config/deno.json so Deno's own internal test suite keeps catching real ops/resource leaks even though the user-facing default flips to off. Keys match the serde renames in SerializedTestConfig (sanitizeOps/sanitizeResources), so this is correctly wired.

My earlier COMMENTED notes (no --no-sanitize-ops symmetry; recommend calling Deno.test.sanitizer() at module top in the .dts; comment the LSP hard-coded false) were optional polish, not blockers. Happy to land as-is.

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Substance walks. The five-layer precedence (per-test → module-level → CLI → env → config) does work in the documented order, with one important asymmetry that @lunadogbot already flagged in their initial COMMENT and waved through as optional polish — I'd lean toward making it a blocker rather than nice-to-have. Also one validation gap on the new Deno.test.sanitizer() API and a documentation/implementation drift in the PR body.

Substantive observations inline.

This is a default-flip on Deno.test() (sanitizeOps/sanitizeResources from true → false) plus three new public surfaces (Deno.test.sanitizer() JS API, --sanitize-ops/--sanitize-resources CLI flags, DENO_TEST_SANITIZE_OPS/DENO_TEST_SANITIZE_RESOURCES env vars, test.sanitizeOps/test.sanitizeResources config keys). That's escalation territory regardless of CI state — this is a v2.8 ergonomics change that affects every existing user.

CI

101 pass, 14 pending, 0 fail. Holding COMMENT per CI-must-complete; substance otherwise tracking @lunadogbot's APPROVE.

Comment thread cli/args/mod.rs
no_run: test_flags.no_run,
shuffle: test_flags.shuffle,
trace_leaks: test_flags.trace_leaks,
sanitize_ops: test_flags.sanitize_ops

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CLI/env/config layer is enable-only. test_flags.sanitize_ops || env_var || config_sanitize_ops means any one setting true wins; none can set false.

Concrete failure mode:

// deno.json
{ "test": { "sanitizeOps": true } }

User wants to disable sanitizers for one CLI invocation:

deno test --no-sanitize-ops      # ← no such flag
DENO_TEST_SANITIZE_OPS=0 deno test  # ← treated as "unset", not as "disable" (the check is == Some("1"))

The only escape hatch is editing the test source (per-test or module-level Deno.test.sanitizer({ ops: false })). That's surprising for users who set defaults in deno.json and expect CLI to override.

@lunadogbot flagged this in their initial review and marked it as optional polish; I'd argue it's actually a blocker because the docs in the PR body claim "Precedence (highest to lowest): ... 3. CLI flags ... 5. Config file" — which is not what the implementation does. CLI can't override config-set-true. That's the kind of doc-vs-implementation drift users hit and file issues about.

Minimal fix: add --no-sanitize-ops / --no-sanitize-resources clap_overrides_with flags that flip a Option<bool> instead of plain bool, and resolve as test_flags.sanitize_ops_override.unwrap_or(env_var || config_sanitize_ops). Then both directions of override work.

Alternatively: document explicitly that the CLI/env/config layer is enable-only and only the JS-side per-test/module-level layers can disable. That at least makes the PR body match reality.

Comment thread cli/js/40_test.js
registerHook("afterAll", fn);
};

test.sanitizer = function (options) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deno.test.sanitizer() doesn't validate its input. Three failure modes:

Deno.test.sanitizer();          // → TypeError: Cannot read properties of undefined (reading 'ops')
Deno.test.sanitizer(null);      // → TypeError: Cannot read properties of null (reading 'ops')
Deno.test.sanitizer("true");    // → silent no-op (string has no `.ops` / `.resources`)

Since this is a new public API on Deno.test, validation matches the rest of the codebase's conventions:

if (typeof options !== "object" || options === null) {
  throw new TypeError("options must be an object");
}

Or webidl.requiredArguments + webidl.converters.any shape if you want the same validation rigor as Deno.test() itself.

Also worth a comment in the .dts file recommending Deno.test.sanitizer() is called at module top before any Deno.test() registrations — calling it after registers tests with the stale defaults silently. @lunadogbot raised this too.

@@ -331,6 +331,8 @@ impl TestRun {
filter,
shuffle: None,
trace_leaks: false,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-existing concern from @lunadogbot's earlier COMMENT — the LSP hard-codes sanitize_ops: false, sanitize_resources: false (intentional, since LSP-driven test runs don't usefully surface sanitizer failures). Worth a one-line comment so this doesn't get "fixed" later by someone who doesn't know the rationale.

// LSP test runs intentionally disable sanitizers — the LSP UI doesn't
// surface sanitizer failures usefully and they'd just generate noise.
sanitize_ops: false,
sanitize_resources: false,

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Substance walks cleanly. The default flip (sanitizeOps/sanitizeResources true → false) at cli/js/40_test.js:268–274 lands via the ?? chain moduleSanitizeOps ?? Deno[Deno.internal].testSanitizeOps ?? false, which correctly implements the documented precedence (per-test → module → CLI/env → config → built-in default). The reland additions (config-file support in cli/schemas/config-file.v1.json:927–940, Deno.test.sanitizer({ ops, resources }) API at 40_test.js:434–443, CLI flags + env-var plumbing through TestFlags → TestSpecifierOptions → worker bootstrap) are wired consistently across all the touch points (LSP execution path included at cli/lsp/testing/execution.rs:333–334).

CI 134/137 success at this head, no failures. Updated spec-test fixtures (5 files) reflect the new defaults — the previously-checking-sanitizer-output tests now opt in via --sanitize-ops or per-test options, which is exactly the migration the PR establishes for users.

Escalation: every trigger fires here

This isn't a fibibot-autonomous APPROVE. The change crosses the autonomous-bar in five distinct ways:

  1. Breaking change to default behavior of a stable public API (Deno.test()) — sanitizers default flips from true to false. Existing tests that relied on the implicit default (rather than explicitly setting sanitizeOps: true) will silently stop catching leaks after upgrade.
  2. New CLI flags: --sanitize-ops, --sanitize-resources on deno test.
  3. New env vars: DENO_TEST_SANITIZE_OPS, DENO_TEST_SANITIZE_RESOURCES.
  4. New public Deno API: Deno.test.sanitizer(options) module-level setter.
  5. New config-file surface: test.sanitizeOps / test.sanitizeResources in deno.json(c).

Given this is a reland of #33158 (which was reverted in #33215), the breaking-default-flip part has presumably already been through one round of maintainer review. But the new in this reland items (config-file fields, module-level API, formal precedence chain) are post-revert additions that need fresh eyes. Worth pinging @nathanwhit, who has the most recent ownership of cli/tools/test/mod.rs and cli/js/40_test.js, on the precedence-chain semantics specifically — the ?? cascade is correct as written, but the user-facing precedence doc in the PR body is the public contract, and any divergence between the doc and 40_test.js:268 would be a long-tail bug.

One small substantive question worth answering before merge: is Deno.test.sanitizer({ ops: false }) additive with --sanitize-ops? Reading testInner's default chain, moduleSanitizeOps = false (set explicitly via Deno.test.sanitizer({ ops: false })) takes precedence over Deno[Deno.internal].testSanitizeOps = true (set by --sanitize-ops). So a module-level false overrides a CLI --sanitize-ops, which is opposite to the typical "CLI > config" mental model. That matches your stated precedence chain (module > CLI), but it's worth being explicit about it in lib.deno.ns.d.ts's docstring so users don't get caught out.

- validate Deno.test.sanitizer() argument and throw TypeError on
  non-object input (previously crashed with a confusing
  "Cannot read properties of undefined" on undefined/null, and
  silently no-op'd on strings or numbers)
- clarify in WorkspaceTestOptions::resolve that the CLI/env/config
  layer is enable-only — opting out requires the JS-side per-test
  or module-level API
- comment LSP test runner's hard-coded `sanitize_ops: false` /
  `sanitize_resources: false` so the rationale isn't lost
- document in the .dts that Deno.test.sanitizer() should be called
  at module top, since tests registered before the call inherit the
  previous defaults
- add spec test covering the new input validation
@deno-cla-assistant

deno-cla-assistant Bot commented May 12, 2026 •

Copy link
Copy Markdown

Deno Individual Contributor License Agreement

All contributors have signed the CLA. Thank you!

Re-run CLA check


This is an automated message from CLA Assistant

@crowlbot

Copy link
Copy Markdown
Contributor

Addressed @fibibot's three inline review comments on top of @bartlomieju's branch, plus a related .dts docstring note that @lunadogbot raised in their initial review:

  • cli/js/40_test.js — Deno.test.sanitizer() now throws TypeError when called with undefined, null, or a non-object. Previously crashed with Cannot read properties of undefined (reading 'ops') or silently no-op'd on strings/numbers. Added a spec test (tests/specs/test/sanitizer_module_level/invalid_options.{ts,out}) covering the new validation.
  • cli/args/mod.rs — replaced the misleading // Precedence (highest wins): CLI flag > env var > deno.json config with a comment explaining that this layer is enable-only: any source setting true turns the sanitizer on, but none of them can turn it back off. To disable a sanitizer enabled here, opt out from JS via Deno.test.sanitizer({ ops: false }) at the module level or { sanitizeOps: false } per-test. (Did not add --no-sanitize-ops / --no-sanitize-resources flags — happy to do that in a follow-up if the consensus is that the CLI/env/config layer should support symmetric disable.)
  • cli/lsp/testing/execution.rs — added a comment explaining the LSP test runner intentionally disables sanitizers (the LSP UI doesn't surface sanitizer failures usefully), so it doesn't get "fixed" later.
  • cli/tsc/dts/lib.deno.ns.d.ts — added a note in Deno.test.sanitizer()'s docstring that it should be called at the top of the module before any Deno.test() registrations, since tests registered before the call inherit the previous defaults.

Verified locally: cargo check and the full sanitizer_module_level spec suite (6 tests including the new invalid_options) pass.

@crowlbot

Copy link
Copy Markdown
Contributor

CI all green on 4e5fb12 (rerun cleared the unrelated linux-aarch64 bundle_watch_html_entry watcher flake — pre-existing esbuild aarch64 exec-format-error, not touched by this PR). PR shows mergeable: clean, with @lunadogbot's prior approval still standing.

@bartlomieju — ready for merge whenever you're ready. Happy to follow up with a separate PR adding --no-sanitize-ops / --no-sanitize-resources CLI flags for symmetric disable, if the team decides the CLI/env/config layer should support that.

@fibibot

This comment was marked as spam.

@fibibot

This comment was marked as off-topic.

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three inline comments from my prior review at 167d1c38 are addressed at 4e5fb12:

  • cli/args/mod.rs:219-223 — five-line comment now spells out the enable-only nature of the CLI/env/config layers and points at the JS opt-out (Deno.test.sanitizer({ ops: false }) or per-test sanitizeOps: false).
  • cli/js/40_test.js:437-441 — Deno.test.sanitizer() validates input and throws TypeError("options must be an object") on undefined/null/non-object.
  • cli/lsp/testing/execution.rs:334-336 — added the rationale comment for LSP hard-coding sanitize_{ops,resources}: false.

New tests/specs/test/sanitizer_module_level/invalid_options.ts exercises the TypeError throw for no-arg, null, undefined, string, and number inputs.

lib.deno.ns.d.ts:1261-1264 also picked up a "call before Deno.test() registrations" note that matches the implementation — moduleSanitizeOps/moduleSanitizeResources are captured per-registration in the ?? chain at 40_test.js:271-274, so tests registered before a sanitizer() call use the prior defaults.

CI fully green at this head.

@crowlbot

Copy link
Copy Markdown
Contributor

Gentle bump @bartlomieju — this has been mergeable: clean with 2 approvals and all 134 required checks green since 2026-05-14. Happy to rebase if it's gotten stale; otherwise ready for merge whenever you have a moment.

@crowlbot

Copy link
Copy Markdown
Contributor

@bartlomieju — friendly nudge, this is still sitting mergeable: clean with 2 approvals and all 134 required checks green since the 14th. Happy to rebase if it'''s gotten stale, but otherwise ready for merge whenever you have a moment.

@crowlKats
crowlKats merged commit 8443ee0 into main May 20, 2026
268 of 270 checks passed
@crowlKats
crowlKats deleted the reland/disable-sanitizers-default branch May 20, 2026 12:47
@dsherret

Copy link
Copy Markdown
Contributor

I think the main issue here was there was no way to easily globally turn this off. The introduction of the new "sanitizeOps" and "sanitizeResources" in the deno.json and providing a suggestion to set them to false probably would have been sufficient and helped keep people's code high quality. Plus, LLMs help fix these issues so easily now (I'm skeptical there was still a demand to turn this off). Now this is more boilerplate to add to a deno.json.

littledivy pushed a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…enoland#33250)

Reland of denoland#33158, which was reverted in denoland#33215 to be relanded for Deno
v2.8.

The fixes from denoland#33215 (internal timers for TLS handshake, sanitizer test
fixes) are already in main.

---

Changes the default for `sanitizeOps` and `sanitizeResources` from
`true` to `false` in `Deno.test()`. This aligns with what most users
expect -- tests no longer fail due to leaked async ops or resources
unless explicitly opted in.

## New in this reland

### Config file support

Sanitizers can now be configured in `deno.json(c)`:

```jsonc
{
  "test": {
    "sanitizeOps": true,
    "sanitizeResources": true
  }
}
```

### Module-level API

`Deno.test.sanitizer()` allows configuring sanitizers per-module:

```ts
// Enable sanitizers for all tests in this file
Deno.test.sanitizer({ ops: true, resources: true });

Deno.test("my test", () => {
  // ops and resources sanitizers are enabled
});

Deno.test({
  name: "override per-test",
  sanitizeOps: false,
  fn() {
    // this test opts out of ops sanitizer
  },
});
```

### Precedence (highest to lowest)

1. Per-test options (`sanitizeOps`/`sanitizeResources` on `Deno.test()`)
2. Module-level (`Deno.test.sanitizer()`)
3. CLI flags (`--sanitize-ops`/`--sanitize-resources`)
4. Environment variables
(`DENO_TEST_SANITIZE_OPS`/`DENO_TEST_SANITIZE_RESOURCES`)
5. Config file (`deno.json` `test.sanitizeOps`/`test.sanitizeResources`)

---------

Co-authored-by: crowlbot <280062030+crowlbot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants