Skip to content

feat: add mm network, a passive request log for pages and the service worker - #111

Open
cryptotavares wants to merge 1 commit into
cryptotavares/mmcli-p2-16-logsfrom
cryptotavares/mmcli-p2-17-network
Open

cryptotavares wants to merge 1 commit into
cryptotavares/mmcli-p2-16-logsfrom
cryptotavares/mmcli-p2-17-network

Conversation

@cryptotavares

@cryptotavares cryptotavares commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Important

Stacked on #110 (P2-16) → #109 (P2-15) → #107 (P2-28, ready to review on main). Review those first. This PR targets #110's branch and contains one commit.

Release validation needs, for each flow, the backend calls the wallet made, with status, duration, failures and JSON-RPC methods. On the Extension most of that traffic (gas, Blockaid, bridge, intents, Relay, Hyperliquid) comes from the MV3 service worker. mock-network only sees hosts it routes and records no timing. The closed #106 did this with Playwright context events. This PR uses our own CDP sessions (#107) instead, so it needs no minimum Playwright version, and a mobile source can later fill the same record format.

  • Source (src/events/network-source.ts): Network.requestWillBeSent / responseReceived / dataReceived / loadingFinished / loadingFailed on every page and the service worker, joined by (target.kind, target.targetId, requestId) into one record per request, updated in place. A record keeps its first seq and gets a settledSeq when it ends, that is, when no field of it can change any more (its rpcError read, when configured, included). An unfiltered read returns records in seq order; a filtered one returns settled records in settledSeq order.
    { seq, settledSeq?, t, target: { kind: 'page'|'sw', id, url }, method, url, type, status?, ms?, failure?, rpcMethods?, rpcError? }
    
    • t: request start (wallTime, epoch ms).
    • ms: CDP monotonic time from the start to the finish or the failure.
    • type: lowercase CDP resource type.
    • failure: CDP errorText (+ blocked reason, + cors: <CorsErrorStatus> when Chrome reports one), redacted; target closed when the page or worker goes away mid-request or the source restarts; abandoned when the request is dropped from the pending limit.
    • url: redacted with the P2-15 rules.
    • rpcMethods: always an array, parsed from the JSON-RPC request body (postData / postDataEntries, up to 64 KiB; single and batch). Absent for non-RPC requests. A method name is cut at 128 characters.
    • Only http(s) URLs are recorded. Others are counted in stats.ignored.
    • A redirect ends its hop with the redirect status and starts a new record (one record per hop).
  • rpcError { code, message }, opt-in: mm network configure --rpc-errors reads Network.getResponseBody for JSON-RPC requests only.
    • Limits: up to 64 KiB decoded (sum of Network.dataReceived.dataLength, else encodedDataLength), 8 reads at once, 2 s timeout. Skipped reads are counted in stats.rpcErrorsSkipped. Off by default. Network.enable caps the response buffers (128 KiB per resource, 1 MiB total) since no bodies are kept otherwise.
    • The read runs on the request's own target session, through the new CdpManager.sendToEventTarget, with the target lookup inside the timeout. It never re-attaches a gone target.
  • mm network [--cursor N | --since <iso|ms>] [--limit N] [--url <glob|text>] [--errors] [--settled] → { records, cursor, hasMore, gap, summary, stats: { pending, ignored, rpcErrorsSkipped } }.
    • --errors: status ≥ 400, failure or rpcError.
    • --url: a glob (*, **) or a substring, like mock-network requests --url (feat: record request bodies in mock-network #93).
    • Settled reads: an unfiltered read returns every record in start order (seq), pending requests included, and its cursor moves past all of them. A read with --url or --errors returns only settled requests, in settledSeq order: a request may become a match only when it ends (a 404, a failure, an rpcError), so it is returned once by the read that follows its end, even when it started before the cursor. The cursor stays a plain buffer position (one counter shared by appends and settles), so it also continues a read of the other kind once hasMore is false, and a record that settles after a read always lands above the returned cursor. --settled asks for that same settle-order read without a filter (value-less, like --errors): one call at the end of a step reads the final status of every request that ended since the mark, including ones started earlier; requests still open are not returned. An unsettled request is never returned by a filtered or --settled read, so a request that never ends never matches one. --since compares the request start (t), not when it ended (documented).
    • Pending requests end with their target: a request whose page closes, whose worker stops or that is still open when the source restarts is finalized with failure: "target closed" and stops counting in stats.pending; a request dropped from the pending limit is finalized with failure: "abandoned".
  • Starts with the session. Passive: it never routes, blocks or changes a request, and it stores no bodies. Read-only, browser-only tool. Also adds the zod schema, CLI help, README, and SKILL.md with the per-flow recipe (mark → network/logs --cursor, startedAt check, capture gap). Unknown read flags and missing values exit 2; configure takes exactly one of --rpc-errors / --no-rpc-errors.

Part of the mm CLI v2 effort: ticket P2-17 (Phase 2).

Shared #93 helper: --url uses mock-network's matchesUrlFilter, so mm network --url and mm mock-network requests --url match URLs the same way.

Breaking changes

None.

Consumer impact

Consumer Change needed Done on branch
Extension test/e2e/playwright/llm-workflow/ none n/a
Mobile tests/llm-workflow/ none (browser only; mobile source after P2-00 S1/S3) n/a

The Extension release-validation skill (Consensys/skills#122) can take one mm events mark at flow start and read mm network / mm logs at flow end, falling back to its mock-network recorder on older mm. The reports importer maps failure / text / rpcMethods / target.

Proof

Real daemon (createServer from the built dist/) + built CLI, headless Chromium, read-only, no funds. Rerun after the review fixes on the rebuilt dist/ of this head (ce385a3): 12/12 harness checks pass (the new 17.5 is the --settled end-to-end scenario), the redaction grep finds nothing (18 × [REDACTED]), and the 22 mm events|logs|network reads take a mean of 96 ms wall-clock (max 130 ms, CLI start included). A live late-settle check on the same dist/: a 2 s page fetch started before a filtered read (read returns 0 records, cursor 3), then the next read with cursor 3 returns it once (seq 3, settledSeq 4), and the read after that returns nothing.

Check Result
Worker fetch 200 / 404 / aborted and a page fetch: status or failure, ms, target.kind ✅ sw … 200 123ms, sw … 404 0ms, sw … net::ERR_ABORTED 60ms, page … 200 122ms
Single and batch JSON-RPC POST → rpcMethods ✅ ["eth_call"], ["eth_chainId","eth_blockNumber"]
--rpc-errors: execution reverted → rpcError ✅ {"code":-32000,"message":"execution reverted"}; the batch without errors has none; rpcErrorsSkipped 0
After ServiceWorker.stopAllWorkers (0 worker targets) and a wake ✅ new worker requests appear, including the worker's start-up ?boot=1 fetch
Relaunch ✅ changes summary.startedAt; seq restarts; an old cursor reads the new session from the start (gap: true)
Grep for the fake Infura key (path) and apiKey=supersecret (query) over every CLI output ✅ not found; 18 × [REDACTED]
--limit 3 paging ✅ 2 pages, no duplicates
--errors, --url glob, events clear keeps seq, bad flag ✅ (bad flag: exit 2)
--settled --cursor <mark>: only settled requests, in settledSeq order, with final status; a repeat with the returned cursor returns none ✅ 17.5: 6 settled (csv 22,26,27,29,32,33), settledSeq 29 is the execution reverted request, repeat 0 records, gap false
MetaMask test build 13.56.0 (dist/chrome, first screen, no wallet) ✅ worker GET https://accounts.api.cx.metamask.io/v2/supportedNetworks 200 60ms, chainid.network 200, refused e2e fixture ports as net::ERR_CONNECTION_REFUSED; 90 chrome-extension:// loads ignored
Read cost ✅ 22 mm events/logs/network calls: mean 96 ms wall-clock (CLI start included), max 130 ms
Filtered read with an open request ✅ unit: pending → fails → next --errors read returns it once; a record that settles after the read (even one older than the cursor) is returned once; limit: 1 pages late failures; unfiltered reads unchanged. Live: late-settle check above
Transcript (MV3 test extension)
$ mm events mark          → {"cursor":1,"summary":{"startedAt":1795010960760,"seq":1,...}}
$ mm navigate chrome-extension://…/trigger.html?tag=A
$ mm network --cursor 1 --limit 3     (2 pages)
  2 page GET http://127.0.0.1:47123/ok?from=page 200 122ms
  5 sw GET http://127.0.0.1:47123/ok?tag=A&apiKey=[REDACTED] 200 123ms
  6 sw GET http://127.0.0.1:47123/missing?tag=A 404 0ms
  7 sw POST http://127.0.0.1:47123/v3/[REDACTED] 200 41ms rpc=eth_call
  8 sw POST http://127.0.0.1:47123/v3/[REDACTED] 200 43ms rpc=eth_chainId,eth_blockNumber
  9 sw GET http://127.0.0.1:47123/slow - 60ms net::ERR_ABORTED
$ mm network configure --rpc-errors
$ mm network --cursor 12 --errors
  15 sw GET …/missing?tag=B 404 1ms
  16 sw POST …/v3/[REDACTED] 200 43ms rpc=eth_call rpcError=-32000:execution reverted
  18 sw GET …/slow - 58ms net::ERR_ABORTED

{"seq":16,"t":1795010963306,"target":{"kind":"sw","id":"D5F965B86CF62F7BC00A489B9B488D12","url":"chrome-extension://ocffhifhkanmalpagbcoojhomhpifmpp/bg.js"},
 "method":"POST","url":"http://127.0.0.1:47123/v3/[REDACTED]","type":"fetch","rpcMethods":["eth_call"],"status":200,"ms":43,
 "rpcError":{"code":-32000,"message":"execution reverted"}}
MetaMask test build (first screen, no wallet)
mark 0 (records already in the buffer since launch: {"console":0,"network":0})
network after mark: 5 records {"sw":4,"page":1}; stats {"pending":0,"ignored":90,"rpcErrorsSkipped":0}
  1 sw GET http://localhost:12345/state.json - 245ms net::ERR_CONNECTION_REFUSED
  2 sw GET https://chainid.network/chains.json 200 140ms
  3 sw GET https://chainid.network/chains.json 200 136ms
  4 sw GET https://accounts.api.cx.metamask.io/v2/supportedNetworks 200 60ms
  6 page GET http://localhost:9999/ - 1ms net::ERR_CONNECTION_REFUSED
logs: 5 sw error SocketBackgroundToMocha WebSocket error: Event

The e2e build's onboarding screen sends little: wallet traffic (gas, Blockaid, RPC) needs an unlocked wallet, which this proof does not create.

Unit tests
 Test Files  74 passed (74)
      Tests  2689 passed (2689)
Type Errors  no errors
All files          |   98.07 |    95.24 |   97.66 |   98.18 |
  event-buffer.ts / json-rpc.ts / network.ts: 100 / 100 (lines)
  network-source.ts: 98.51 lines
  cdp-manager.ts   |   99.47 |    96.02 |   98.88 |   99.46 |
yarn lint: exit 0
yarn build: exit 0

New: src/events/network-source.test.ts (join, in-place update, failure, redirect hops (each hop settles), per-target keys, ignored URLs, postData/postDataEntries, pending cap, rpcError on/off (settles after the read, including skipped and failed reads), base64, decoded vs encoded size, size/concurrency/failure skips, dataReceived, targetDetached finalization, CORS reason, settle on finish/fail/redirect/start), src/events/event-buffer.test.ts (settle order, paging by settle position, gap (a dropped unsettled record, since), dropped/cleared records never settle, stale cursor, since on t, one cursor across kinds), src/events/json-rpc.test.ts (method cap), src/tools/network.test.ts (late settle after a read, older record, limit: 1 paging, unfiltered reads unchanged, settled: true end-to-end), src/events/network-source.test.ts (abandoned, target closed on restart), CdpManager sendToEventTarget and targetId (timeout, race), CLI routing and schemas.

… worker

Joins Network.requestWillBeSent / responseReceived / loadingFinished /
loadingFailed from every page and the Extension service worker into one
record per request, keyed by (target kind, target id, request id) and
updated in place: method, redacted URL, type, status, ms, failure and
rpcMethods (always an array; single and batch JSON-RPC bodies). Only
http(s) requests are recorded; others are counted as ignored. A redirect
gives one record per hop.

'mm network configure --rpc-errors' reads JSON-RPC response bodies with
Network.getResponseBody (64 KiB, 8 at a time, 2 s timeout; skipped reads
are counted) and records { code, message } as rpcError. Off by default.
Adds CdpManager.sendToEventTarget for commands on the target of an event.

'mm network' reads with --cursor/--since/--limit/--url/--errors.
@cryptotavares
cryptotavares force-pushed the cryptotavares/mmcli-p2-17-network branch from 48c7db6 to ce385a3 Compare November 18, 2026 18:51
@cryptotavares
cryptotavares marked this pull request as ready for review November 18, 2026 18:58
@cryptotavares
cryptotavares requested a review from a team as a code owner November 18, 2026 18:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant