Skip to content

Phase 5: ADR 0050 and a Python MCP client verification - #1330

Merged
neuromechanist merged 4 commits into
feature/issue-1065-epic-mcp-serverfrom
feature/issue-1297-phase5-docs-release
Sep 9, 2026
Merged

neuromechanist merged 4 commits into
feature/issue-1065-epic-mcp-serverfrom
feature/issue-1297-phase5-docs-release

Conversation

@neuromechanist

Copy link
Copy Markdown
Contributor

Summary

Part A of phase 5, the in-repo half. The rest of the phase (docs page, OSA runtime, release) is blocked on deployment and follows in order.

Closes nothing yet; #1297 stays open until the docs and OSA halves land.
Part of epic #1065.

ADR 0050: no WebAssembly in the Worker bundle

Formalizes the rule two phases learned the hard way. numcodecs instantiated WASM from a runtime fetch(); hyparquet-compressors compiled hysnappy's WASM at MODULE LOAD and so crashed isolate startup for every request, not just get_events calls. Both were invisible to bun test, which is why backend/scripts/mcp-smoke.sh under real workerd is the gate rather than a nicety.

It could not be filed earlier: the ADR index test enforces gapless numbering and 0049 was still open in #1292. That merged, so the content staged in the design doc is now a pointer to the real ADR.

The ADR is honest about the gate's own gap: the smoke script drives a throwaway entry module because wrangler dev --local cannot start the real index.ts (#1324), so it proves the MCP sub-app's dependency graph starts under workerd, not that the whole worker does.

scripts/mcp/verify_streamable_http.py

Drives a target host with the official Python MCP SDK: the negotiated protocol revision, all six tools with the tools/list cache hint, then a chain from search_datasets through describe_dataset and list_recordings into get_events, render_overview and both read_window modes, against a recording it discovers rather than one hardcoded here.

Three reasons it exists, none covered by the TypeScript tests:

  1. mcp-smoke.sh drives our own framing with curl. It says nothing about whether a real MCP SDK, doing its own handshake, can talk to us. OSA speaks this SDK, so this is the client that has to work.
  2. The docs page copies its examples from a real session captured here, not from the design doc.
  3. It is the standing check for a deploy, run against staging after the epic reaches dev and against production after the release.

Its SDK usage was validated against a local MCPServer harness before first use, rather than discovered during a cutover. That caught two real problems:

  • Any check failure inside the client's anyio task group came back as ERROR ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) with none of the message explaining what failed. Now unwrapped, so a failure reads FAIL render_overview: no image content block on the result.
  • An over-cap read_window refusal can arrive either as a tool error (is_error) or as a raised protocol error, depending on whether the input schema or the tool rejects it. The check accepts both, since both are refusals and asserting one would report a false failure against the other.

Verified API surface (mcp 2.2.0, the current release, checked by introspection rather than recalled): high-level Client("<url>") speaks Streamable HTTP and negotiated 2026-07-28; FastMCP was renamed MCPServer in 2.x; Python-side result fields are snake_case (structured_content) while the wire keeps structuredContent.

Two assumptions in #1297 that turned out to be wrong

Both recorded in the design doc, both change what the remaining parts do:

  • There is no "For-agents guide" to add an MCP section to, in either nemarOrg/docs or nemarOrg/website. The docs work is a new page, modeled on docs's own develop/zarr-contract.md.
  • OSA cannot consume an MCP server at all. No mcp dependency, no client code, nothing reads mcp_servers outside the config model whose docstring still says "(Phase 2)". And its two existing NEMAR tools call nemar.org/api/dataexplorer/datapipeline/..., which returns 404 since the legacy site was retired. So this server replaces them rather than supplementing them. Per the owner's decision, that runtime gets built in this phase.

Why the rest is not in this PR

mcp.nemar.org and mcp-test.nemar.org are custom_domain = true routes, so neither hostname exists until a deploy provisions it. Neither resolves today. Verification is impossible until the epic reaches dev; production waits on the release. Writing docs or OSA wiring against an unverified server would mean describing behavior nobody has seen.

Test plan

  • bun test test/adr-index.unit.test.ts 8 of 8, including the gapless check that previously blocked filing 0050.
  • MCP cluster plus contract and schema tests: 376 pass, 0 fail.
  • bun run typecheck clean.
  • Verification script: SDK usage exercised end to end against a local MCPServer harness over Streamable HTTP; the full chain runs against a live host in Part C.

ADR 0050 formalizes the rule two phases learned the hard way: no WebAssembly
in the Worker bundle, ever, for any package this server depends on. Both
failures (numcodecs instantiating WASM from a runtime fetch, and
hyparquet-compressors compiling hysnappy's WASM at module load and so crashing
isolate startup for every request) were invisible to bun test and caught only
under real workerd, which is why the smoke script is the gate rather than an
optimization. It could not be filed until ADR 0049 landed on dev, since the
index test enforces gapless numbering; the content staged in the design doc is
now a pointer to the ADR.

scripts/mcp/verify_streamable_http.py drives a target host with the official
Python MCP SDK: the negotiated revision, all six tools with their cache hint,
then a chain from search through describe and list_recordings into the three
recording tools and both read_window modes, against a recording it discovers
rather than one hardcoded here. mcp-smoke.sh checks our own framing with curl
and says nothing about whether a real SDK can talk to us, and OSA speaks this
SDK, so this is the client that has to work. It is also where the docs page
will copy its examples from.

Its SDK usage was validated against a local MCPServer harness rather than
discovered during a cutover, which caught two things worth having: any check
failure inside the client's task group came back as a bare "ExceptionGroup:
unhandled errors" with none of the message that explains it, and an over-cap
refusal can arrive either as a tool error or as a raised protocol error, so the
check accepts both.

Design doc records the verified SDK version and the two assumptions in #1297
that turned out to be wrong: there is no For-agents guide to extend, and OSA
has no MCP runtime at all while its two existing NEMAR tools call an endpoint
that now 404s.
test/adr-index.unit.test.ts validates gapless numbering, README index
agreement, and superseded-target existence. Nothing in the path filters
matched .context/**, so it never ran when those files changed: #1292 and #1330
both showed a fully green check list with the one test that could have failed
skipped, and both were verified by hand instead.

Same shape as the shared/** gap this file already documents, same fix. Wired
into unit-pure, with a self-reference so editing the filter re-runs it.
@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

📦 Package Preview

Your package is ready to test!

bun install -g nemar-cli@PR1330

Version: 0.10.1-PR1330.e2dcff8
Commit: e2dcff8


🤖 Updated on each push.

Review of scripts/mcp/verify_streamable_http.py found the script could not
connect at all and could report a pass it had not earned. Fixed:

- The default URL omitted the /mcp transport path. GET / is a descriptor and
  POST / answers Not Found, which during a cutover reads like an
  unprovisioned hostname. A bare host is now normalized and reported.
- The over-cap probe wrapped its call in `except Exception`, so a 429 or a
  502 became a PASS. Removed: both a tool error and a schema rejection
  arrive as is_error, so an exception there is a transport failure.
- The probe sat exactly AT all three caps, which the schema accepts (`>`).
  Now one second over the duration cap with one channel, so exactly one cap
  is crossed, and the refusal text must name it.
- The envelope check tested 3 of 14 required fields. Now all 14, plus
  non-null where the contract forbids null, and required only where the
  contract makes it required: it is .optional() on three tools, so failing
  hard everywhere reported missing provenance against a valid legacy index.
- The subject was `results[0]`, now scanned for a v3 index with a usable
  group. has_zarr means converted, not v3, so a legacy hit is legitimate.
- Read `results` only; `datasets` is catalog.json's field.
- Notes are counted and repeated in the closing line, so a degraded run
  cannot look clean.

Adds the wire-level test the script's cap assertion depends on: an over-cap
taste answers isError with a text block naming the cap and the recipe
remedy. Validated against a local MCPServer stand-in with twelve fault
injections; all ten defects fail with the right message and the two honest
degradations exit 0 with a note.
An accuracy review of ADR 0050 and the design doc's phase 5 entry found four
wrong claims, two of which would misdirect future work. All verified before
changing, two by re-running the experiment.

ADR 0050 said there is no flag permitting dynamic WASM compilation, so the
restriction was the platform's and not configurable. Inverted. Measured under
wrangler dev --local: compiling at MODULE LOAD is permitted by
allow_eval_during_startup, which is the default from compatibility_date
2025-06-01, so the phase 3 shape is blocked only because this worker pins
2024-12-01. Compiling inside a request stays disallowed under every
combination tested. So the decision survives but gains a real consequence: a
routine compatibility_date bump silently removes the guard that caught the
phase 3 failure, which is why the rule is written down and smoke-gated rather
than left to the runtime. The flag's docs never mention WebAssembly, so the
ADR now says the behavior is measured, not documented.

ADR 0050 also framed both failures as dependencies that broke the server with
a green suite. Neither was ever in backend/package.json: numcodecs was the
spike's path (a), and hyparquet-compressors was removed before merge. The
lesson holds, the history does not, and an ADR outranks the design doc.

Design doc: the For-agents guide #1297 names does exist, published at
docs platform/for-agents.md since 2026-09-03, so the docs work is a section
on that page and #1297's premise was right. develop/zarr-contract.md is gone,
split into six pages under platform/zarr/. Both errors came from reading a
stale local checkout instead of origin/main.

Also: #1324 has three non-handler exports, not two strings; the fixtures are
real bytes where a real store exercises the path and synthetic only where the
catalog has no such shape; the subpath-import option is named; and the
custom-domain zone-edit caveat is moved next to the sequencing it gates.
@neuromechanist

Copy link
Copy Markdown
Contributor Author

Review rounds addressed

Two reviews ran on this PR. Both found real defects; everything is fixed in
294cc6b and e2dcff8.

Round 1, the verification script. The headline finding is that the script
could not connect at all: the default URL omitted the /mcp transport path, and
POST / answers Not Found, which during a cutover is indistinguishable from
an unprovisioned hostname. My own validation missed it because I passed /mcp
explicitly. Also fixed: an except Exception around the over-cap probe that
turned a 429 or a 502 into a PASS; a probe that sat exactly AT all three caps,
which the schema accepts; an envelope check covering 3 of 14 required fields;
and results[0] as the subject, which fails against a legacy-index dataset that
has_zarr legitimately matches.

Round 2, factual accuracy of ADR 0050 and the design doc. Four wrong claims,
two of which would have misdirected future work.

The ADR's central platform claim was inverted. It said no flag permits dynamic
WASM compilation. Measured under wrangler dev --local, twice, independently:

compatibility_date flags compile at result
2024-12-01 (ours) nodejs_compat module load worker never starts
2024-12-01 + allow_eval_during_startup module load HTTP 200
2025-11-09 nodejs_compat (flag default-on) inside fetch() HTTP 500, still disallowed

So the phase 3 shape is configurable and blocked only because we pin
2024-12-01; the phase 1 shape is genuinely impossible. The decision survives,
and gains a consequence worth more than the original reasoning: a routine
compatibility_date bump silently removes the guard that caught the phase 3
failure.
That is now why the rule is written down and smoke-gated rather than
left to the runtime to police. The flag's own docs never mention WebAssembly, so
the ADR states the behavior is measured, not documented.

The ADR also framed both incidents as dependencies that broke the server with a
green suite. Neither was ever in backend/package.json: numcodecs was the
spike's path (a), hyparquet-compressors was removed before merge. Corrected,
because an ADR outranks the design doc by house rule.

The design doc aimed the docs work at a page that does not exist and away from
one that does. platform/for-agents.md has been published since 2026-09-03, so
#1297's premise was right and my "assumption was wrong" note was the wrong
thing
; and develop/zarr-contract.md is gone, split into six pages under
platform/zarr/. Both errors came from reading a stale local checkout instead
of origin/main.

What was verified rather than asserted

  • The refusal text the script asserts on now has a test behind it:
    backend/test/mcp-route.test.ts pins the wire shape (isError plus a text
    block naming the cap and the recipe remedy). Mutation-checked, and the real
    string is taste duration_s (61) exceeds the 60 s cap; omit taste for a recipe instead.
  • The script itself was driven against a local MCPServer stand-in with twelve
    fault injections, entering through a bare host. Ten defects fail with a message
    naming the cause; the two that are the server being honest rather than broken
    exit 0 with a note. Method recorded in the script's docstring.

Gates: 254 MCP tests, ADR index 8/8, typecheck and biome clean, unit-pure
confirmed RUN rather than skipped.

@neuromechanist
neuromechanist merged commit 1de3452 into feature/issue-1065-epic-mcp-server Sep 9, 2026
12 checks passed
@neuromechanist
neuromechanist deleted the feature/issue-1297-phase5-docs-release branch September 9, 2026 12:20

This branch was previously deployed

1 inactive deployment
npm — e2dcff86 Deployed Sep 9, 2026 by neuromechanist via publish #2425
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant