Skip to content

feat(ext/node): buffer Network.* bodies for inspector body-fetch commands - #34201

Merged
bartlomieju merged 3 commits into
mainfrom
feat/inspector-network-body-buffer
May 18, 2026
Merged

bartlomieju merged 3 commits into
mainfrom
feat/inspector-network-body-buffer

Conversation

@bartlomieju

Copy link
Copy Markdown
Member

The Node-style inspector Network domain shipped in #32707 plumbed event
broadcast for Network.dataReceived / dataSent / requestWillBeSent /
responseReceived, but the three CDP commands consumers actually use to
pull bodies back out — Network.getResponseBody,
Network.streamResourceContent, and Network.getRequestPostData — weren't
implemented, so any tool that called them got a -32601 method-not-found
error.

This adds a bounded per-process buffer (32 requests max, 10 MB per
request, FIFO eviction) on the inspector that captures bodies as the
events flow through op_inspector_emit_protocol_event, plus dispatcher
handlers for the three commands on both the local-session and WebSocket
paths. Charset handling mirrors Node: utf-8 returns a decoded string,
anything else returns base64. Raw Buffer/Uint8Array data passed to
inspector.Network.dataReceived/.dataSent is base64-encoded at the
polyfill layer to match the wire format consumers expect.

Clears four node_compat tests:
test-inspector-network-data-received.js,
test-inspector-network-data-sent.js,
test-inspector-network-arbitrary-data.js, and
test-inspector-emit-protocol-event.js. Wiring fetch / node:http / http2 /
ws as event producers is a follow-up that will build on this buffer.

… / getRequestPostData / streamResourceContent

Adds a bounded per-process buffer (32 requests max, 10 MB per request) that
captures Network.dataReceived / dataSent / requestWillBeSent / responseReceived
events as they pass through op_inspector_emit_protocol_event, keyed by CDP
requestId.

The inspector dispatcher now handles three additional CDP commands
locally (without forwarding to V8, which doesn't know about the Network
domain): Network.getResponseBody, Network.streamResourceContent, and
Network.getRequestPostData. Charset-aware decoding mirrors Node: utf-8
returns a string, anything else returns base64.

Also base64-encodes raw Buffer/Uint8Array data passed to
inspector.Network.dataReceived/.dataSent at the polyfill layer, matching
the wire format consumers expect.

Clears 4 inspector node_compat tests:
  test-inspector-network-data-received.js
  test-inspector-network-data-sent.js
  test-inspector-network-arbitrary-data.js
  test-inspector-emit-protocol-event.js
Buffer Network.dataReceived events until Network.streamResourceContent
flips the entry into streaming mode, after which subsequent events flow
over the wire instead of being re-buffered. Network.dataSent is purely
a capture event and is not broadcast. Track is_request_finished and
is_response_finished so getResponseBody/getRequestPostData reject when
the request/response is still in flight, and handle dataSent
{finished: true} to mark request completion. Erase entries on
loadingFailed, on getResponseBody, and on loadingFinished while
streaming. Drop the dead truncated field, strict-match utf-8 charset,
and route both dispatcher paths through dispatch_custom_domain. Throw
TypeError on unknown data shapes at the polyfill instead of silently
passing them through. Enable the data-received and data-sent
node_compat tests in config.jsonc.
Match Node's NetworkAgent::getRequestPostData, which returns
DispatchResponse::ServerError (-32000) for the
"Unable to serialize binary request body" case rather than InvalidParams
(-32602): the protocol doesn't model binary post bodies, so this is a
server-side limitation rather than a caller mistake. Other error paths
(request not found, not finished, streaming) remain InvalidParams,
matching network_agent.cc.
@bartlomieju
bartlomieju merged commit bf72384 into main May 18, 2026
136 checks passed
@bartlomieju
bartlomieju deleted the feat/inspector-network-body-buffer branch May 18, 2026 13:01
bartlomieju added a commit that referenced this pull request May 19, 2026
Building on the Network CDP domain landed in #32707 and the body
buffering hooks in #34201, this wires the global `fetch()` into the
inspector so that DevTools and `node:inspector` clients actually observe
outgoing HTTP traffic. Until now Deno exposed the `Network.*` emitters
but nothing inside Deno called them, so attached frontends saw no events
and `Network.getResponseBody` had no data to return.

When the inspector is attached, each `fetch()` now emits
`requestWillBeSent`, `responseReceived`, `dataReceived`, and either
`loadingFinished` or `loadingFailed`, including the post-data and
response-body bookkeeping the buffer needs. The cross-extension bridge
lives on `internals.__inspectorNetwork`, installed by ext/node's
inspector polyfill, so ext/fetch doesn't depend on ext/node and the
non-attached path is one `isEnabled()` check. Enables
`parallel/test-inspector-network-fetch.js`; node:http / node:http2 /
WebSocket instrumentation will follow.
bartlomieju pushed a commit that referenced this pull request May 20, 2026
## Summary

Wires `node:http`'s `ClientRequest` into the inspector so DevTools and
`node:inspector` clients observe outgoing http/https traffic from
`http.request` / `http.get` / `https.request` / `https.get`. Builds on
the Network CDP domain landed in #32707, the body buffering hooks in
#34201, and the fetch() instrumentation in #34220.

When the inspector is attached, each client request now emits:
- `Network.requestWillBeSent` (from the `ClientRequest` constructor, so
the user's call-site stays on the V8 stack for `initiator` capture)
- `Network.responseReceived` (from `parserOnIncomingClient`, with `type:
"Other"` to distinguish from `type: "Fetch"`)
- `Network.dataReceived` (per body chunk, by wrapping `res.push` rather
than attaching a `data` listener — keeps the response paused until the
consumer attaches its own listener, asserted by the upstream test)
- `Network.loadingFinished` (on `res.push(null)`) or
`Network.loadingFailed` (from `emitErrorEvent`)

When the inspector is detached, the cost is one
`internals.__inspectorNetwork.isEnabled()` call.

### Inspector server teardown fix

Also fixes a pre-existing panic in `InspectorServer::drop`:
`shutdown_server_tx.send(()).expect(...)` panicked when the server
thread had already exited via `stop()` -> `reset_tx`, which manifests as
a flaky exit-1 from the new node_compat test (`for i in 1..10; do deno
run --inspect=0 test-inspector-network-http.js; done` reproduced it ~1/5
times). The Drop now treats a missing receiver as a no-op.

Co-authored-by: divybot <divybot@users.noreply.github.com>
Co-authored-by: Divy Srivastava <me@littledivy.com>
littledivy pushed a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…ands (denoland#34201)

The Node-style inspector Network domain shipped in denoland#32707 plumbed event
broadcast for Network.dataReceived / dataSent / requestWillBeSent /
responseReceived, but the three CDP commands consumers actually use to
pull bodies back out — Network.getResponseBody,
Network.streamResourceContent, and Network.getRequestPostData — weren't
implemented, so any tool that called them got a -32601 method-not-found
error.

This adds a bounded per-process buffer (32 requests max, 10 MB per
request, FIFO eviction) on the inspector that captures bodies as the
events flow through op_inspector_emit_protocol_event, plus dispatcher
handlers for the three commands on both the local-session and WebSocket
paths. Charset handling mirrors Node: utf-8 returns a decoded string,
anything else returns base64. Raw Buffer/Uint8Array data passed to
inspector.Network.dataReceived/.dataSent is base64-encoded at the
polyfill layer to match the wire format consumers expect.

Clears four node_compat tests:
test-inspector-network-data-received.js,
test-inspector-network-data-sent.js,
test-inspector-network-arbitrary-data.js, and
test-inspector-emit-protocol-event.js
littledivy pushed a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…d#34220)

Building on the Network CDP domain landed in denoland#32707 and the body
buffering hooks in denoland#34201, this wires the global `fetch()` into the
inspector so that DevTools and `node:inspector` clients actually observe
outgoing HTTP traffic. Until now Deno exposed the `Network.*` emitters
but nothing inside Deno called them, so attached frontends saw no events
and `Network.getResponseBody` had no data to return.

When the inspector is attached, each `fetch()` now emits
`requestWillBeSent`, `responseReceived`, `dataReceived`, and either
`loadingFinished` or `loadingFailed`, including the post-data and
response-body bookkeeping the buffer needs. The cross-extension bridge
lives on `internals.__inspectorNetwork`, installed by ext/node's
inspector polyfill, so ext/fetch doesn't depend on ext/node and the
non-attached path is one `isEnabled()` check. Enables
`parallel/test-inspector-network-fetch.js`; node:http / node:http2 /
WebSocket instrumentation will follow.
littledivy added a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…nd#34231)

## Summary

Wires `node:http`'s `ClientRequest` into the inspector so DevTools and
`node:inspector` clients observe outgoing http/https traffic from
`http.request` / `http.get` / `https.request` / `https.get`. Builds on
the Network CDP domain landed in denoland#32707, the body buffering hooks in
denoland#34201, and the fetch() instrumentation in denoland#34220.

When the inspector is attached, each client request now emits:
- `Network.requestWillBeSent` (from the `ClientRequest` constructor, so
the user's call-site stays on the V8 stack for `initiator` capture)
- `Network.responseReceived` (from `parserOnIncomingClient`, with `type:
"Other"` to distinguish from `type: "Fetch"`)
- `Network.dataReceived` (per body chunk, by wrapping `res.push` rather
than attaching a `data` listener — keeps the response paused until the
consumer attaches its own listener, asserted by the upstream test)
- `Network.loadingFinished` (on `res.push(null)`) or
`Network.loadingFailed` (from `emitErrorEvent`)

When the inspector is detached, the cost is one
`internals.__inspectorNetwork.isEnabled()` call.

### Inspector server teardown fix

Also fixes a pre-existing panic in `InspectorServer::drop`:
`shutdown_server_tx.send(()).expect(...)` panicked when the server
thread had already exited via `stop()` -> `reset_tx`, which manifests as
a flaky exit-1 from the new node_compat test (`for i in 1..10; do deno
run --inspect=0 test-inspector-network-http.js; done` reproduced it ~1/5
times). The Drop now treats a missing receiver as a no-op.

Co-authored-by: divybot <divybot@users.noreply.github.com>
Co-authored-by: Divy Srivastava <me@littledivy.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant