Skip to content

feat(ext/fetch): emit Network.* inspector events for fetch() - #34220

Merged
bartlomieju merged 6 commits into
mainfrom
feat/inspector-network-fetch
May 19, 2026
Merged

bartlomieju merged 6 commits into
mainfrom
feat/inspector-network-fetch

Conversation

@bartlomieju

@bartlomieju bartlomieju commented May 18, 2026 •

Copy link
Copy Markdown
Member

Building on the Network CDP domain landed in #32707 and the body
buffering hooks in #34201, this wires the global fetch() into the
inspector so that DevTools and node:inspector clients actually observe
outgoing HTTP traffic. Until now Deno exposed the Network.* emitters
but nothing inside Deno called them, so attached frontends saw no events
and Network.getResponseBody had no data to return.

When the inspector is attached, each fetch() now emits
requestWillBeSent, responseReceived, dataReceived, and either
loadingFinished or loadingFailed, including the post-data and
response-body bookkeeping the buffer needs. The cross-extension bridge
lives on internals.__inspectorNetwork, installed by ext/node's
inspector polyfill, so ext/fetch doesn't depend on ext/node and the
non-attached path is one isEnabled() check. Enables
parallel/test-inspector-network-fetch.js; node:http / node:http2 /
WebSocket instrumentation will follow.

When the inspector is attached, the global fetch() now emits
Network.requestWillBeSent / responseReceived / dataReceived /
loadingFinished / loadingFailed events, including request and response
body buffering so Network.getRequestPostData and Network.getResponseBody
return real content. This matches Node's --experimental-network-inspection
behavior for the fetch path.

The bridge lives on `internals.__inspectorNetwork`, populated by
ext/node/polyfills/inspector.js when node:inspector is loaded, so
ext/fetch doesn't need to depend on ext/node. The lookup is one method
call (`isEnabled()`) when no debugger is attached.

Fixes a couple of pre-existing bugs that were latent until the fetch test
exercised them: `Network.requestWillBeSent`'s captured initiator was
serializing as `initiator.stackTrace` instead of `initiator.stack` and
returning `{ type: "other" }` when the stack was empty; the
`is_utf8_charset` check rejected `UTF-8` (uppercase) which fetch's
implicit `Content-Type: text/plain;charset=UTF-8` always produces.

The node_compat runner now forwards `--inspect=PORT` (rewriting bare
ports to 127.0.0.1) and pairs it with --unsafely-ignore-certificate-errors
so the inspector-network-* tests' self-signed HTTPS server works without
relying on undici's global dispatcher.

Enables tests/node_compat/parallel/test-inspector-network-fetch.js.
`capture_initiator` only stripped the `file://` prefix on Unix, so on
Windows frame URLs like `file:///C:/...` failed to match Node CJS
`__filename` (`C:\...`) and broke the test-inspector-network-fetch CI
job. Use ModuleSpecifier::to_file_path so the conversion produces the
native OS path everywhere.
Replaces `url::Url::to_file_path` (forbidden by clippy in this crate
because it doesn't work in Wasm) with a small helper that strips the
`file://` prefix and, on Windows, drops the leading `/` and flips
forward slashes to backslashes. Uses `sys_traits::impls::is_windows`
for the platform check.
…k-fetch

# Conflicts:
#	ext/node/polyfills/inspector.js

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly some inline notes — see below. Higher-level things that didn't fit on a specific line:

  • Repetitive try/catch: there are 7 copies of try { inspectorNetwork.xxx({...}); } catch { /* ignore */ } in 26_fetch.js. A 3-line safeEmit(fn, params) helper would cut a lot of noise and make future audits easier. Not a blocker.
  • Tee memory pressure: bodyStream.tee() doubles the in-memory footprint of response bodies while the inspector is attached. I assume the buffer cap from #34201 applies before any Uint8Array reaches dataReceived({data: value}) — worth a quick confirm.
  • Redirect chain visibility: real DevTools emits requestWillBeSent with a redirectResponse for each hop. The current !recursive gate correctly suppresses re-entries, but you lose visibility into redirect chains. Probably fine for a v1; worth a follow-up issue.
  • Test coverage: the diff only enables parallel/test-inspector-network-fetch.js; no Deno-native test covers the recursive suppression, the streaming-body dataSent path (see inline), or the tee() failure fallback. A small unit-style test in tests/specs/ would guard the wiring.

Things I checked and liked: disabled-path cost is one method call; {__proto__: null} on the headers object is good prototype hygiene; the lowerCaseNames toggle (lowercase for request, canonical Set-Cookie for response) matches CDP/DevTools convention; the stackTrace → stack rename is a real CDP-schema bugfix; background drain swallows errors so instrumentation can't leak unhandled rejections to user code.

Comment thread ext/fetch/26_fetch.js Outdated
Comment on lines +392 to +395
if (hasPostData) {
inspectorNetwork.dataSent({
requestId: inspectorRequestId,
finished: true,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likely bug — gating on the wrong condition.

The comment two lines up says streaming bodies "take the chunked Network.dataSent path instead and aren't covered here," but this branch fires dataSent({finished:true}) whenever hasPostData is true — including the streaming case where postDataText is undefined. Result: Network.getRequestPostData will see is_request_finished == true with no data ever supplied, and return an empty/garbage payload instead of the documented "Request data is not finished yet" rejection.

Suggest gating on postDataText !== undefined (or the explicit Uint8Array branch) so streaming bodies stay un-finished until the chunked path lands.

if (postDataText !== undefined) {
  inspectorNetwork.dataSent({
    requestId: inspectorRequestId,
    finished: true,
  });
}

Comment thread ext/fetch/26_fetch.js Outdated
Comment on lines +358 to +364
let requestCharset;
for (let i = 0; i < req.headerList.length; i++) {
if (byteLowerCase(req.headerList[i][0]) === "content-type") {
requestCharset = parseContentTypeForCdp(req.headerList).charset ||
undefined;
break;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead loop. The outer for walks headers just to gate a call to parseContentTypeForCdp, which walks the headers itself. When content-type is absent, parseContentTypeForCdp returns { charset: "" }, and the || undefined here converts that to undefined anyway. So the whole block collapses to:

let requestCharset = parseContentTypeForCdp(req.headerList).charset || undefined;

Identical behavior, one pass over the header list instead of (potentially) two.

Comment thread ext/node/ops/inspector.rs
Comment on lines +231 to +235
serde_json::json!({
"type": "script",
"stack": {
"callFrames": call_frames,
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two unrelated changes squashed here, one of which is a behavior change worth a comment:

  1. stackTrace → stack — real bugfix; CDP's Network.Initiator schema uses stack (typed as Runtime.StackTrace). 👍
  2. Empty-frames branch removed — previously call_frames.is_empty() returned {type:"other"}; now it unconditionally returns {type:"script", stack:{callFrames:[]}}. A CDP client reasonably expecting type:"other" when there's no callstack will be surprised.

Either restore the is_empty() branch, or leave a one-line justification ("node_compat tests expect script even with no frames" or similar) so the next reader doesn't second-guess it.

Comment thread ext/node/ops/inspector.rs
Comment on lines +177 to +186
fn file_url_to_os_path(s: String) -> String {
let Some(after_scheme) = s.strip_prefix("file://") else {
return s;
};
// file:///path/to/file -> /path/to/file (Unix)
// file:///C:/path -> C:\path (Windows)
if sys_traits::impls::is_windows() {
after_scheme.trim_start_matches('/').replace('/', "\\")
} else {
after_scheme.to_string()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two corner-case gaps worth at least a // TODO:

  • No percent-decoding. file:///path%20with%20spaces/foo.js becomes /path%20with%20spaces/foo.js, which won't match __filename for paths containing spaces or non-ASCII. Rare in CI, real in user code.
  • UNC paths (file://server/share/x) on Windows yield server\share\x instead of \\server\share\x — strip_prefix("file://") strips both slashes, so the UNC marker is gone. Probably out of scope for this PR, but worth a comment so the next person doesn't think it's handled.

Comment thread tests/node_compat/mod.rs Outdated
// on undici's `rejectUnauthorized: false` global dispatcher,
// which Deno's fetch doesn't honor. Disable cert validation
// here so the HTTPS branch of the test can complete.
deno_args.push("--unsafely-ignore-certificate-errors".to_string());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scope is broader than the comment suggests. This appends --unsafely-ignore-certificate-errors to every test that uses --inspect, not just test-inspector-network-fetch.js. config.jsonc already lists five test-inspector-network-* tests, and any future inspector test that deliberately sets up TLS will silently lose validation.

Consider attaching the flag per-test via a config.jsonc override (e.g. an extra_deno_args field) rather than baking it into the flag-parser. That keeps the security knob explicit at the call site.

Comment thread tests/node_compat/mod.rs Outdated
Comment on lines +544 to +551
let mapped = if let Some(rest) = n.strip_prefix("--inspect=") {
if rest.is_empty() || rest.chars().all(|c| c.is_ascii_digit()) {
format!("--inspect=127.0.0.1:{}", rest)
} else {
format!("--inspect={}", rest)
}
} else {
"--inspect=127.0.0.1:0".to_string()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small nits:

  • chars().all(_) already returns true on an empty iterator, so rest.is_empty() || is redundant.
  • When rest is empty, the format string produces --inspect=127.0.0.1: (no port), which is an invalid flag. If --inspect= is reachable in practice, you probably want to map it to :0 the same way the bare --inspect arm does — e.g. via a match on strip_prefix(...):
let mapped = match n.strip_prefix("--inspect=") {
    None | Some("") => "--inspect=127.0.0.1:0".to_string(),
    Some(rest) if rest.chars().all(|c| c.is_ascii_digit()) => {
        format!("--inspect=127.0.0.1:{}", rest)
    }
    Some(rest) => format!("--inspect={}", rest),
};

26_fetch.js:
- Centralize all `try { inspectorNetwork.xxx(...) } catch {}` blocks
  behind a `safeEmit(fn, params)` helper (7 sites cut to one shape).
- Gate the post-data `Network.dataSent({finished:true})` flush on
  `postDataText !== undefined` so streaming bodies (where postDataText
  stays undefined) correctly stay un-finished, preserving the
  "Request data is not finished yet" rejection until the chunked
  dataSent path lands.
- Collapse the dead content-type-header loop: parseContentTypeForCdp
  already walks the headers once.

inspector.rs:
- Add a comment justifying the always-`type:"script"` branch (test
  contract — we only reach capture_initiator from JS-emitted events).
- Add TODO notes on file_url_to_os_path covering percent-decoding and
  Windows UNC paths.

mod.rs / config.jsonc / schema.json:
- Drop the auto-attached `--unsafely-ignore-certificate-errors` from the
  `--inspect=*` flag mapper (it was bleeding into every inspector test).
  Introduce a per-test `extraDenoArgs` config field and move the cert
  flag onto `parallel/test-inspector-network-fetch.js` only, with a
  reason recorded inline.
- Rewrite the `--inspect=*` parser as a match so empty (`--inspect=`)
  maps to `127.0.0.1:0` instead of an invalid `--inspect=127.0.0.1:`.
@bartlomieju

Copy link
Copy Markdown
Member Author

Pushed ce5fe95 addressing the inline notes. Quick responses to the
three higher-level points:

Tee memory pressure — confirmed: the per-request cap (NETWORK_BUFFER_MAX_BYTES_PER_REQUEST = 10MB in libs/core/inspector.rs) is enforced inside capture_network_event before bytes are appended to entry.data, and the FIFO eviction at 32 requests caps total. The tee'd inspector branch is drained back-to-back in a background reader, so at any instant only the in-flight chunk and the user-branch's stream queue are alive in JS — there's no doubled long-lived buffer. The only growth risk is a user that holds the response without consuming, which is the same risk untee'd fetch already has; the inspector branch keeps draining regardless and the Rust cap protects the persistent storage.

Redirect chain visibility — agreed, this is v1 scope. Filed follow-up: I'll add redirectResponse on the recursive call from httpRedirectFetch so DevTools sees the hops, in a separate PR.

Test coverage — also follow-up. The current node_compat test covers the happy path end-to-end; for the streaming-body dataSent path and the tee failure fallback I'll add focused unit tests in tests/unit/fetch_test.ts (with Deno.test driving the inspector via op_inspector_emit_protocol_event directly) once the underlying inspector ops are accessible from the test harness.

Two pieces from the higher-level review that I previously deferred to
follow-ups; doing them here instead.

Redirect chain: mainFetch and httpRedirectFetch now thread an
`inspectorCtx { requestId, redirectResponse }` through the recursive
hop. A 302 hop reuses the original requestId, suppresses its own
`responseReceived`, and the next hop's `requestWillBeSent` carries the
30x as `redirectResponse`. Matches Chrome DevTools' contract.

Streaming body: `hasPostData` was wrongly gated on `reqBody !== null`,
so a `ReadableStream` body (where the bytes flow through `reqRid`)
ended up declaring `hasPostData: false` and the Rust capture promptly
flipped `is_request_finished = true`. Fixed; now streaming bodies
correctly stay un-finished and `getRequestPostData` rejects with
"Request data is not finished yet", as documented.

New spec test `tests/specs/inspector/fetch_events/` exercises both
behaviors: it spins up a `Deno.serve` that 302s, asserts the requestId
correlates across hops and the intermediate 302 doesn't fire its own
`responseReceived`; and it sends a `ReadableStream` POST body and
verifies `Network.getRequestPostData` rejects.
@bartlomieju

Copy link
Copy Markdown
Member Author

Fair, fixed in e7ec6f3 — both in this PR, not deferred:

Redirect chain — mainFetch and httpRedirectFetch now thread an inspectorCtx { requestId, redirectResponse } through the recursive hop. A 302 reuses the original requestId, suppresses its own responseReceived, and the next requestWillBeSent carries the previous 30x as redirectResponse. Matches Chrome's contract.

Streaming body coverage — turned up a real bug while writing the test: hasPostData was gated on reqBody !== null, so a ReadableStream body (where bytes flow through reqRid) declared hasPostData: false and Rust flipped is_request_finished = true immediately. Now hasPostData = reqBody !== null || reqRid !== null; streaming bodies stay un-finished and getRequestPostData rejects with "Request data is not finished yet" as documented.

New spec test tests/specs/inspector/fetch_events/ covers both: a Deno.serve 302 chain verifying the requestId correlates across hops and the intermediate 302 doesn't fire its own responseReceived; and a ReadableStream POST body verifying Network.getRequestPostData rejects with the expected message.

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — every point from the previous review was addressed:

  • dataSent({finished:true}) now correctly gates on postDataText !== undefined; hasPostData = reqBody !== null || reqRid !== null keeps streaming-body requests un-finished as intended.
  • The redundant content-type loop is gone — single direct call.
  • capture_initiator empty-frames change has the inline explanation it needed.
  • file_url_to_os_path has explicit TODOs for the percent-decoding and UNC gaps.
  • --unsafely-ignore-certificate-errors is now per-test via extraDenoArgs in config.jsonc (schema updated too), so the flag only applies to test-inspector-network-fetch.js.
  • The --inspect= empty-case bug is fixed with a clean match.
  • safeEmit(fn, params) replaces the seven try/catch blocks at every emit site.
  • Redirect chains are now correctly implemented (shared requestId, redirectResponse on the next hop, intermediate responseReceived suppressed) — matches DevTools' contract.
  • New tests/specs/inspector/fetch_events/ spec test covers both the redirect chain and the streaming-body "not finished yet" behavior.

All 136 CI checks pass (the unstable mergeable_state appears to be stale). Ready to land.

@bartlomieju
bartlomieju merged commit 8aec4d5 into main May 19, 2026
268 of 270 checks passed
@bartlomieju
bartlomieju deleted the feat/inspector-network-fetch branch May 19, 2026 01:38
bartlomieju added a commit that referenced this pull request May 19, 2026
…34222)

Follow-up to #34220, on top of the same `internals.__inspectorNetwork`
bridge. When the inspector is attached, constructing a `WebSocket` now
emits `Network.webSocketCreated`, the resolved handshake response
emits `Network.webSocketHandshakeResponseReceived` with real
status/statusText/headers (`op_ws_create` returns them now), and every
code path that dispatches the user-visible `CloseEvent` also emits
`Network.webSocketClosed`. DevTools and `node:inspector` clients see
WebSocket traffic alongside the fetch events.

The compat test `parallel/test-inspector-network-websocket.js` is
ignored with a reason: it asserts the initiator stack contains a
frame matching `/undici/`, which is a Node-internal detail of how
Node's WebSocket is built on undici. Deno's WebSocket is native, so no
such frame exists; the rest of the assertions pass (verified with a
standalone smoke test).
bartlomieju pushed a commit that referenced this pull request May 20, 2026
## Summary

Wires `node:http`'s `ClientRequest` into the inspector so DevTools and
`node:inspector` clients observe outgoing http/https traffic from
`http.request` / `http.get` / `https.request` / `https.get`. Builds on
the Network CDP domain landed in #32707, the body buffering hooks in
#34201, and the fetch() instrumentation in #34220.

When the inspector is attached, each client request now emits:
- `Network.requestWillBeSent` (from the `ClientRequest` constructor, so
the user's call-site stays on the V8 stack for `initiator` capture)
- `Network.responseReceived` (from `parserOnIncomingClient`, with `type:
"Other"` to distinguish from `type: "Fetch"`)
- `Network.dataReceived` (per body chunk, by wrapping `res.push` rather
than attaching a `data` listener — keeps the response paused until the
consumer attaches its own listener, asserted by the upstream test)
- `Network.loadingFinished` (on `res.push(null)`) or
`Network.loadingFailed` (from `emitErrorEvent`)

When the inspector is detached, the cost is one
`internals.__inspectorNetwork.isEnabled()` call.

### Inspector server teardown fix

Also fixes a pre-existing panic in `InspectorServer::drop`:
`shutdown_server_tx.send(()).expect(...)` panicked when the server
thread had already exited via `stop()` -> `reset_tx`, which manifests as
a flaky exit-1 from the new node_compat test (`for i in 1..10; do deno
run --inspect=0 test-inspector-network-http.js; done` reproduced it ~1/5
times). The Drop now treats a missing receiver as a no-op.

Co-authored-by: divybot <divybot@users.noreply.github.com>
Co-authored-by: Divy Srivastava <me@littledivy.com>
littledivy pushed a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…d#34220)

Building on the Network CDP domain landed in denoland#32707 and the body
buffering hooks in denoland#34201, this wires the global `fetch()` into the
inspector so that DevTools and `node:inspector` clients actually observe
outgoing HTTP traffic. Until now Deno exposed the `Network.*` emitters
but nothing inside Deno called them, so attached frontends saw no events
and `Network.getResponseBody` had no data to return.

When the inspector is attached, each `fetch()` now emits
`requestWillBeSent`, `responseReceived`, `dataReceived`, and either
`loadingFinished` or `loadingFailed`, including the post-data and
response-body bookkeeping the buffer needs. The cross-extension bridge
lives on `internals.__inspectorNetwork`, installed by ext/node's
inspector polyfill, so ext/fetch doesn't depend on ext/node and the
non-attached path is one `isEnabled()` check. Enables
`parallel/test-inspector-network-fetch.js`; node:http / node:http2 /
WebSocket instrumentation will follow.
littledivy pushed a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…enoland#34222)

Follow-up to denoland#34220, on top of the same `internals.__inspectorNetwork`
bridge. When the inspector is attached, constructing a `WebSocket` now
emits `Network.webSocketCreated`, the resolved handshake response
emits `Network.webSocketHandshakeResponseReceived` with real
status/statusText/headers (`op_ws_create` returns them now), and every
code path that dispatches the user-visible `CloseEvent` also emits
`Network.webSocketClosed`. DevTools and `node:inspector` clients see
WebSocket traffic alongside the fetch events.

The compat test `parallel/test-inspector-network-websocket.js` is
ignored with a reason: it asserts the initiator stack contains a
frame matching `/undici/`, which is a Node-internal detail of how
Node's WebSocket is built on undici. Deno's WebSocket is native, so no
such frame exists; the rest of the assertions pass (verified with a
standalone smoke test).
littledivy added a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…nd#34231)

## Summary

Wires `node:http`'s `ClientRequest` into the inspector so DevTools and
`node:inspector` clients observe outgoing http/https traffic from
`http.request` / `http.get` / `https.request` / `https.get`. Builds on
the Network CDP domain landed in denoland#32707, the body buffering hooks in
denoland#34201, and the fetch() instrumentation in denoland#34220.

When the inspector is attached, each client request now emits:
- `Network.requestWillBeSent` (from the `ClientRequest` constructor, so
the user's call-site stays on the V8 stack for `initiator` capture)
- `Network.responseReceived` (from `parserOnIncomingClient`, with `type:
"Other"` to distinguish from `type: "Fetch"`)
- `Network.dataReceived` (per body chunk, by wrapping `res.push` rather
than attaching a `data` listener — keeps the response paused until the
consumer attaches its own listener, asserted by the upstream test)
- `Network.loadingFinished` (on `res.push(null)`) or
`Network.loadingFailed` (from `emitErrorEvent`)

When the inspector is detached, the cost is one
`internals.__inspectorNetwork.isEnabled()` call.

### Inspector server teardown fix

Also fixes a pre-existing panic in `InspectorServer::drop`:
`shutdown_server_tx.send(()).expect(...)` panicked when the server
thread had already exited via `stop()` -> `reset_tx`, which manifests as
a flaky exit-1 from the new node_compat test (`for i in 1..10; do deno
run --inspect=0 test-inspector-network-http.js; done` reproduced it ~1/5
times). The Drop now treats a missing receiver as a no-op.

Co-authored-by: divybot <divybot@users.noreply.github.com>
Co-authored-by: Divy Srivastava <me@littledivy.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants