Repository navigation
feat(ext/fetch): emit Network.* inspector events for fetch() - #34220
Conversation
When the inspector is attached, the global fetch() now emits
Network.requestWillBeSent / responseReceived / dataReceived /
loadingFinished / loadingFailed events, including request and response
body buffering so Network.getRequestPostData and Network.getResponseBody
return real content. This matches Node's --experimental-network-inspection
behavior for the fetch path.
The bridge lives on `internals.__inspectorNetwork`, populated by
ext/node/polyfills/inspector.js when node:inspector is loaded, so
ext/fetch doesn't need to depend on ext/node. The lookup is one method
call (`isEnabled()`) when no debugger is attached.
Fixes a couple of pre-existing bugs that were latent until the fetch test
exercised them: `Network.requestWillBeSent`'s captured initiator was
serializing as `initiator.stackTrace` instead of `initiator.stack` and
returning `{ type: "other" }` when the stack was empty; the
`is_utf8_charset` check rejected `UTF-8` (uppercase) which fetch's
implicit `Content-Type: text/plain;charset=UTF-8` always produces.
The node_compat runner now forwards `--inspect=PORT` (rewriting bare
ports to 127.0.0.1) and pairs it with --unsafely-ignore-certificate-errors
so the inspector-network-* tests' self-signed HTTPS server works without
relying on undici's global dispatcher.
Enables tests/node_compat/parallel/test-inspector-network-fetch.js.
`capture_initiator` only stripped the `file://` prefix on Unix, so on Windows frame URLs like `file:///C:/...` failed to match Node CJS `__filename` (`C:\...`) and broke the test-inspector-network-fetch CI job. Use ModuleSpecifier::to_file_path so the conversion produces the native OS path everywhere.
Replaces `url::Url::to_file_path` (forbidden by clippy in this crate because it doesn't work in Wasm) with a small helper that strips the `file://` prefix and, on Windows, drops the leading `/` and flips forward slashes to backslashes. Uses `sys_traits::impls::is_windows` for the platform check.
…k-fetch # Conflicts: # ext/node/polyfills/inspector.js
fibibot
left a comment
There was a problem hiding this comment.
Mostly some inline notes — see below. Higher-level things that didn't fit on a specific line:
- Repetitive
try/catch: there are 7 copies oftry { inspectorNetwork.xxx({...}); } catch { /* ignore */ }in26_fetch.js. A 3-linesafeEmit(fn, params)helper would cut a lot of noise and make future audits easier. Not a blocker. - Tee memory pressure:
bodyStream.tee()doubles the in-memory footprint of response bodies while the inspector is attached. I assume the buffer cap from #34201 applies before anyUint8ArrayreachesdataReceived({data: value})— worth a quick confirm. - Redirect chain visibility: real DevTools emits
requestWillBeSentwith aredirectResponsefor each hop. The current!recursivegate correctly suppresses re-entries, but you lose visibility into redirect chains. Probably fine for a v1; worth a follow-up issue. - Test coverage: the diff only enables
parallel/test-inspector-network-fetch.js; no Deno-native test covers therecursivesuppression, the streaming-bodydataSentpath (see inline), or thetee()failure fallback. A small unit-style test intests/specs/would guard the wiring.
Things I checked and liked: disabled-path cost is one method call; {__proto__: null} on the headers object is good prototype hygiene; the lowerCaseNames toggle (lowercase for request, canonical Set-Cookie for response) matches CDP/DevTools convention; the stackTrace → stack rename is a real CDP-schema bugfix; background drain swallows errors so instrumentation can't leak unhandled rejections to user code.
| if (hasPostData) { | ||
| inspectorNetwork.dataSent({ | ||
| requestId: inspectorRequestId, | ||
| finished: true, |
There was a problem hiding this comment.
Likely bug — gating on the wrong condition.
The comment two lines up says streaming bodies "take the chunked Network.dataSent path instead and aren't covered here," but this branch fires dataSent({finished:true}) whenever hasPostData is true — including the streaming case where postDataText is undefined. Result: Network.getRequestPostData will see is_request_finished == true with no data ever supplied, and return an empty/garbage payload instead of the documented "Request data is not finished yet" rejection.
Suggest gating on postDataText !== undefined (or the explicit Uint8Array branch) so streaming bodies stay un-finished until the chunked path lands.
if (postDataText !== undefined) {
inspectorNetwork.dataSent({
requestId: inspectorRequestId,
finished: true,
});
}| let requestCharset; | ||
| for (let i = 0; i < req.headerList.length; i++) { | ||
| if (byteLowerCase(req.headerList[i][0]) === "content-type") { | ||
| requestCharset = parseContentTypeForCdp(req.headerList).charset || | ||
| undefined; | ||
| break; | ||
| } |
There was a problem hiding this comment.
Dead loop. The outer for walks headers just to gate a call to parseContentTypeForCdp, which walks the headers itself. When content-type is absent, parseContentTypeForCdp returns { charset: "" }, and the || undefined here converts that to undefined anyway. So the whole block collapses to:
let requestCharset = parseContentTypeForCdp(req.headerList).charset || undefined;Identical behavior, one pass over the header list instead of (potentially) two.
| serde_json::json!({ | ||
| "type": "script", | ||
| "stack": { | ||
| "callFrames": call_frames, | ||
| }, |
There was a problem hiding this comment.
Two unrelated changes squashed here, one of which is a behavior change worth a comment:
stackTrace→stack— real bugfix; CDP'sNetwork.Initiatorschema usesstack(typed asRuntime.StackTrace). 👍- Empty-frames branch removed — previously
call_frames.is_empty()returned{type:"other"}; now it unconditionally returns{type:"script", stack:{callFrames:[]}}. A CDP client reasonably expectingtype:"other"when there's no callstack will be surprised.
Either restore the is_empty() branch, or leave a one-line justification ("node_compat tests expect script even with no frames" or similar) so the next reader doesn't second-guess it.
| fn file_url_to_os_path(s: String) -> String { | ||
| let Some(after_scheme) = s.strip_prefix("file://") else { | ||
| return s; | ||
| }; | ||
| // file:///path/to/file -> /path/to/file (Unix) | ||
| // file:///C:/path -> C:\path (Windows) | ||
| if sys_traits::impls::is_windows() { | ||
| after_scheme.trim_start_matches('/').replace('/', "\\") | ||
| } else { | ||
| after_scheme.to_string() |
There was a problem hiding this comment.
Two corner-case gaps worth at least a // TODO:
- No percent-decoding.
file:///path%20with%20spaces/foo.jsbecomes/path%20with%20spaces/foo.js, which won't match__filenamefor paths containing spaces or non-ASCII. Rare in CI, real in user code. - UNC paths (
file://server/share/x) on Windows yieldserver\share\xinstead of\\server\share\x—strip_prefix("file://")strips both slashes, so the UNC marker is gone. Probably out of scope for this PR, but worth a comment so the next person doesn't think it's handled.
| // on undici's `rejectUnauthorized: false` global dispatcher, | ||
| // which Deno's fetch doesn't honor. Disable cert validation | ||
| // here so the HTTPS branch of the test can complete. | ||
| deno_args.push("--unsafely-ignore-certificate-errors".to_string()); |
There was a problem hiding this comment.
Scope is broader than the comment suggests. This appends --unsafely-ignore-certificate-errors to every test that uses --inspect, not just test-inspector-network-fetch.js. config.jsonc already lists five test-inspector-network-* tests, and any future inspector test that deliberately sets up TLS will silently lose validation.
Consider attaching the flag per-test via a config.jsonc override (e.g. an extra_deno_args field) rather than baking it into the flag-parser. That keeps the security knob explicit at the call site.
| let mapped = if let Some(rest) = n.strip_prefix("--inspect=") { | ||
| if rest.is_empty() || rest.chars().all(|c| c.is_ascii_digit()) { | ||
| format!("--inspect=127.0.0.1:{}", rest) | ||
| } else { | ||
| format!("--inspect={}", rest) | ||
| } | ||
| } else { | ||
| "--inspect=127.0.0.1:0".to_string() |
There was a problem hiding this comment.
Two small nits:
chars().all(_)already returnstrueon an empty iterator, sorest.is_empty() ||is redundant.- When
restis empty, the format string produces--inspect=127.0.0.1:(no port), which is an invalid flag. If--inspect=is reachable in practice, you probably want to map it to:0the same way the bare--inspectarm does — e.g. via amatchonstrip_prefix(...):
let mapped = match n.strip_prefix("--inspect=") {
None | Some("") => "--inspect=127.0.0.1:0".to_string(),
Some(rest) if rest.chars().all(|c| c.is_ascii_digit()) => {
format!("--inspect=127.0.0.1:{}", rest)
}
Some(rest) => format!("--inspect={}", rest),
};26_fetch.js:
- Centralize all `try { inspectorNetwork.xxx(...) } catch {}` blocks
behind a `safeEmit(fn, params)` helper (7 sites cut to one shape).
- Gate the post-data `Network.dataSent({finished:true})` flush on
`postDataText !== undefined` so streaming bodies (where postDataText
stays undefined) correctly stay un-finished, preserving the
"Request data is not finished yet" rejection until the chunked
dataSent path lands.
- Collapse the dead content-type-header loop: parseContentTypeForCdp
already walks the headers once.
inspector.rs:
- Add a comment justifying the always-`type:"script"` branch (test
contract — we only reach capture_initiator from JS-emitted events).
- Add TODO notes on file_url_to_os_path covering percent-decoding and
Windows UNC paths.
mod.rs / config.jsonc / schema.json:
- Drop the auto-attached `--unsafely-ignore-certificate-errors` from the
`--inspect=*` flag mapper (it was bleeding into every inspector test).
Introduce a per-test `extraDenoArgs` config field and move the cert
flag onto `parallel/test-inspector-network-fetch.js` only, with a
reason recorded inline.
- Rewrite the `--inspect=*` parser as a match so empty (`--inspect=`)
maps to `127.0.0.1:0` instead of an invalid `--inspect=127.0.0.1:`.
|
Pushed ce5fe95 addressing the inline notes. Quick responses to the Tee memory pressure — confirmed: the per-request cap ( Redirect chain visibility — agreed, this is v1 scope. Filed follow-up: I'll add Test coverage — also follow-up. The current node_compat test covers the happy path end-to-end; for the streaming-body |
Two pieces from the higher-level review that I previously deferred to
follow-ups; doing them here instead.
Redirect chain: mainFetch and httpRedirectFetch now thread an
`inspectorCtx { requestId, redirectResponse }` through the recursive
hop. A 302 hop reuses the original requestId, suppresses its own
`responseReceived`, and the next hop's `requestWillBeSent` carries the
30x as `redirectResponse`. Matches Chrome DevTools' contract.
Streaming body: `hasPostData` was wrongly gated on `reqBody !== null`,
so a `ReadableStream` body (where the bytes flow through `reqRid`)
ended up declaring `hasPostData: false` and the Rust capture promptly
flipped `is_request_finished = true`. Fixed; now streaming bodies
correctly stay un-finished and `getRequestPostData` rejects with
"Request data is not finished yet", as documented.
New spec test `tests/specs/inspector/fetch_events/` exercises both
behaviors: it spins up a `Deno.serve` that 302s, asserts the requestId
correlates across hops and the intermediate 302 doesn't fire its own
`responseReceived`; and it sends a `ReadableStream` POST body and
verifies `Network.getRequestPostData` rejects.
|
Fair, fixed in e7ec6f3 — both in this PR, not deferred: Redirect chain — Streaming body coverage — turned up a real bug while writing the test: New spec test |
fibibot
left a comment
There was a problem hiding this comment.
LGTM — every point from the previous review was addressed:
dataSent({finished:true})now correctly gates onpostDataText !== undefined;hasPostData = reqBody !== null || reqRid !== nullkeeps streaming-body requests un-finished as intended.- The redundant content-type loop is gone — single direct call.
capture_initiatorempty-frames change has the inline explanation it needed.file_url_to_os_pathhas explicit TODOs for the percent-decoding and UNC gaps.--unsafely-ignore-certificate-errorsis now per-test viaextraDenoArgsinconfig.jsonc(schema updated too), so the flag only applies totest-inspector-network-fetch.js.- The
--inspect=empty-case bug is fixed with a cleanmatch. safeEmit(fn, params)replaces the seven try/catch blocks at every emit site.- Redirect chains are now correctly implemented (shared
requestId,redirectResponseon the next hop, intermediateresponseReceivedsuppressed) — matches DevTools' contract. - New
tests/specs/inspector/fetch_events/spec test covers both the redirect chain and the streaming-body "not finished yet" behavior.
All 136 CI checks pass (the unstable mergeable_state appears to be stale). Ready to land.
…34222) Follow-up to #34220, on top of the same `internals.__inspectorNetwork` bridge. When the inspector is attached, constructing a `WebSocket` now emits `Network.webSocketCreated`, the resolved handshake response emits `Network.webSocketHandshakeResponseReceived` with real status/statusText/headers (`op_ws_create` returns them now), and every code path that dispatches the user-visible `CloseEvent` also emits `Network.webSocketClosed`. DevTools and `node:inspector` clients see WebSocket traffic alongside the fetch events. The compat test `parallel/test-inspector-network-websocket.js` is ignored with a reason: it asserts the initiator stack contains a frame matching `/undici/`, which is a Node-internal detail of how Node's WebSocket is built on undici. Deno's WebSocket is native, so no such frame exists; the rest of the assertions pass (verified with a standalone smoke test).
## Summary Wires `node:http`'s `ClientRequest` into the inspector so DevTools and `node:inspector` clients observe outgoing http/https traffic from `http.request` / `http.get` / `https.request` / `https.get`. Builds on the Network CDP domain landed in #32707, the body buffering hooks in #34201, and the fetch() instrumentation in #34220. When the inspector is attached, each client request now emits: - `Network.requestWillBeSent` (from the `ClientRequest` constructor, so the user's call-site stays on the V8 stack for `initiator` capture) - `Network.responseReceived` (from `parserOnIncomingClient`, with `type: "Other"` to distinguish from `type: "Fetch"`) - `Network.dataReceived` (per body chunk, by wrapping `res.push` rather than attaching a `data` listener — keeps the response paused until the consumer attaches its own listener, asserted by the upstream test) - `Network.loadingFinished` (on `res.push(null)`) or `Network.loadingFailed` (from `emitErrorEvent`) When the inspector is detached, the cost is one `internals.__inspectorNetwork.isEnabled()` call. ### Inspector server teardown fix Also fixes a pre-existing panic in `InspectorServer::drop`: `shutdown_server_tx.send(()).expect(...)` panicked when the server thread had already exited via `stop()` -> `reset_tx`, which manifests as a flaky exit-1 from the new node_compat test (`for i in 1..10; do deno run --inspect=0 test-inspector-network-http.js; done` reproduced it ~1/5 times). The Drop now treats a missing receiver as a no-op. Co-authored-by: divybot <divybot@users.noreply.github.com> Co-authored-by: Divy Srivastava <me@littledivy.com>
…d#34220) Building on the Network CDP domain landed in denoland#32707 and the body buffering hooks in denoland#34201, this wires the global `fetch()` into the inspector so that DevTools and `node:inspector` clients actually observe outgoing HTTP traffic. Until now Deno exposed the `Network.*` emitters but nothing inside Deno called them, so attached frontends saw no events and `Network.getResponseBody` had no data to return. When the inspector is attached, each `fetch()` now emits `requestWillBeSent`, `responseReceived`, `dataReceived`, and either `loadingFinished` or `loadingFailed`, including the post-data and response-body bookkeeping the buffer needs. The cross-extension bridge lives on `internals.__inspectorNetwork`, installed by ext/node's inspector polyfill, so ext/fetch doesn't depend on ext/node and the non-attached path is one `isEnabled()` check. Enables `parallel/test-inspector-network-fetch.js`; node:http / node:http2 / WebSocket instrumentation will follow.
…enoland#34222) Follow-up to denoland#34220, on top of the same `internals.__inspectorNetwork` bridge. When the inspector is attached, constructing a `WebSocket` now emits `Network.webSocketCreated`, the resolved handshake response emits `Network.webSocketHandshakeResponseReceived` with real status/statusText/headers (`op_ws_create` returns them now), and every code path that dispatches the user-visible `CloseEvent` also emits `Network.webSocketClosed`. DevTools and `node:inspector` clients see WebSocket traffic alongside the fetch events. The compat test `parallel/test-inspector-network-websocket.js` is ignored with a reason: it asserts the initiator stack contains a frame matching `/undici/`, which is a Node-internal detail of how Node's WebSocket is built on undici. Deno's WebSocket is native, so no such frame exists; the rest of the assertions pass (verified with a standalone smoke test).
…nd#34231) ## Summary Wires `node:http`'s `ClientRequest` into the inspector so DevTools and `node:inspector` clients observe outgoing http/https traffic from `http.request` / `http.get` / `https.request` / `https.get`. Builds on the Network CDP domain landed in denoland#32707, the body buffering hooks in denoland#34201, and the fetch() instrumentation in denoland#34220. When the inspector is attached, each client request now emits: - `Network.requestWillBeSent` (from the `ClientRequest` constructor, so the user's call-site stays on the V8 stack for `initiator` capture) - `Network.responseReceived` (from `parserOnIncomingClient`, with `type: "Other"` to distinguish from `type: "Fetch"`) - `Network.dataReceived` (per body chunk, by wrapping `res.push` rather than attaching a `data` listener — keeps the response paused until the consumer attaches its own listener, asserted by the upstream test) - `Network.loadingFinished` (on `res.push(null)`) or `Network.loadingFailed` (from `emitErrorEvent`) When the inspector is detached, the cost is one `internals.__inspectorNetwork.isEnabled()` call. ### Inspector server teardown fix Also fixes a pre-existing panic in `InspectorServer::drop`: `shutdown_server_tx.send(()).expect(...)` panicked when the server thread had already exited via `stop()` -> `reset_tx`, which manifests as a flaky exit-1 from the new node_compat test (`for i in 1..10; do deno run --inspect=0 test-inspector-network-http.js; done` reproduced it ~1/5 times). The Drop now treats a missing receiver as a no-op. Co-authored-by: divybot <divybot@users.noreply.github.com> Co-authored-by: Divy Srivastava <me@littledivy.com>
Building on the Network CDP domain landed in #32707 and the body
buffering hooks in #34201, this wires the global
fetch()into theinspector so that DevTools and
node:inspectorclients actually observeoutgoing HTTP traffic. Until now Deno exposed the
Network.*emittersbut nothing inside Deno called them, so attached frontends saw no events
and
Network.getResponseBodyhad no data to return.When the inspector is attached, each
fetch()now emitsrequestWillBeSent,responseReceived,dataReceived, and eitherloadingFinishedorloadingFailed, including the post-data andresponse-body bookkeeping the buffer needs. The cross-extension bridge
lives on
internals.__inspectorNetwork, installed by ext/node'sinspector polyfill, so ext/fetch doesn't depend on ext/node and the
non-attached path is one
isEnabled()check. Enablesparallel/test-inspector-network-fetch.js; node:http / node:http2 /WebSocket instrumentation will follow.