Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
465134f
Phase 1: MCP design, contracts, and spike (#1314)
neuromechanist Sep 8, 2026
eeef010
Merge remote-tracking branch 'origin/dev' into feature/issue-1065-epi…
neuromechanist Sep 8, 2026
cfb3162
chore: sync dev with main after v0.10.0
Sep 8, 2026
bf91e7c
chore: bump version to 0.10.1-dev0
Sep 8, 2026
d14a9d5
Phase 2: MCP host fork and discovery tools (#1323)
neuromechanist Sep 8, 2026
2106474
Merge remote-tracking branch 'origin/dev' into feature/issue-1065-epi…
neuromechanist Sep 8, 2026
e6480dc
Phase 3: MCP recording tools (list_recordings, get_events, render_ove…
neuromechanist Sep 8, 2026
629e9ab
Phase 4: read_window recipe and capped taste (#1327)
neuromechanist Sep 9, 2026
441376b
docs: ADR 0049 supersedes 0025 on compute locality (#1292)
neuromechanist Sep 9, 2026
4a51174
chore: bump version to 0.10.1-dev1
Sep 9, 2026
023691b
Merge remote-tracking branch 'origin/dev' into feature/issue-1065-epi…
neuromechanist Sep 9, 2026
1de3452
Phase 5: ADR 0050 and a Python MCP client verification (#1330)
neuromechanist Sep 9, 2026
3980068
fix: padded view chunks and the MCP rate-limit bucket
neuromechanist Sep 9, 2026
f21df49
fix: key the projection cache on the conversion, not the commit
neuromechanist Sep 9, 2026
749d292
fix: bound the two unbounded anonymous reads
neuromechanist Sep 9, 2026
d5a9e0b
fix: make the parity guard bite, and dtype mean one thing
neuromechanist Sep 9, 2026
6d0135f
fix: the remaining review findings on the MCP tools
neuromechanist Sep 9, 2026
3757be7
Merge pull request #1332 from nemarOrg/feature/issue-1065-epic-mcp-se…
neuromechanist Sep 9, 2026
60f835b
chore: bump version to 0.10.1-dev2
Sep 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ADR 0025: Inference compute runs on the user's device; the MCP is a stateless broker

**Status:** accepted
**Date:** 2026-08-16
**Status:** superseded by ADR-0049
**Date:** 2026-08-16 (superseded 2026-09-08)
**Owner:** Yahya

## Context
Expand All @@ -17,7 +17,7 @@ or a local `uvx` process with no server at all.
Meanwhile eegprep depends on `oct2py` (Octave), so it cannot run in a browser,
and `osc/osa` already ships a persistent LangGraph/FastAPI assistant platform with a NEMAR assistant.

## Decision
## Decision (at the time)

NEMAR serves bytes and understanding; it does not run the science.
The MCP server is a **stateless, recipe-first broker on Cloudflare Workers**:
Expand All @@ -30,6 +30,20 @@ and MNE/numpy under Pyodide for the browser lane.
Stateful conversational surfaces (the first-party chatbot) are OSA assistants
hosted on nemarring.ucsd.edu, not Workers and not new products.

## Why it was superseded

- The eegprep premise was wrong.
eegprep 0.3.0 is a pure-Python wheel whose only `oct2py` import is a lazy import inside the EEGLAB comparison bridge;
the browser blocker is the dependency declaration, which the owner controls.
ADR 0049 makes eegprep the default engine in every lane and drops the accepted drift.
- The chatbot hosting statement was wrong. OSA runs at `api.osc.earth`, not nemarring,
and OSA now owns a browser-side execution runtime that is generic across communities.
- The product vision became three stages with a login boundary only at HPC submission.
0025 said nothing about where assisted execution runs or which stage needs identity;
0049 does.
- What 0025 got right is carried forward verbatim in 0049:
the MCP is a stateless, recipe-first broker on Workers, and bulk bytes go direct to S3.

## Consequences

- The Worker cost and memory concerns dissolve by construction:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# ADR 0049: Compute runs in the browser by default, OSA owns the execution runtime, and only HPC submission is gated

**Status:** accepted
**Date:** 2026-09-08
**Owner:** Seyed Yahya Shirazi

Supersedes ADR 0025.
The parts of 0025 that still hold are restated here so this document is the single current statement;
0025 is kept for its history.

## Context

ADR 0025 settled three things at once:
the MCP is a stateless, recipe-first broker on Cloudflare Workers;
inference compute runs on the user's device;
and the first-party chatbot is an OSA assistant hosted on nemarring.ucsd.edu.
It also recorded that eegprep depends on `oct2py`, so it "cannot run in a browser",
made MNE the browser engine, and accepted engine drift between the browser and desktop lanes.

Two of those premises did not survive contact.
eegprep 0.3.0 ships as a pure-Python wheel;
its only `oct2py` import is a lazy import inside the EEGLAB comparison bridge (`eegprep/eeglabcompat.py`),
and `psutil` is imported the same way.
What blocks a browser install is the dependency declaration in `pyproject.toml`
(`oct2py`, `pyedflib`, and `psutil`; the ICA dependency `python-picard` is pure Python and its `numexpr` use is optional), not the algorithms.
The owner maintains eegprep, so that is a packaging fix, not a constraint.
Separately, the OSA NEMAR assistant is deployed at `api.osc.earth`, not nemarring,
and OSA is gaining a browser-side code execution runtime.

The product vision has also sharpened into three stages:
the assistant writes code and runs it against streamed data;
the user tinkers with the same script and results in a notebook;
the user submits the work to a supercomputer.
The open question was where compute runs at each stage and where the login boundary sits.

## Decision

**Three stages, one engine, one login boundary.**

1. **Stage 1, assisted execution, runs in the user's browser.**
The assistant's `execute_code` tool is executed by the widget in a Pyodide worker,
driven from OSA's server-side LangGraph through client-executed tool calls.
It is anonymous.
Nothing of NEMAR's is used beyond the public data plane and the MCP recipes.
2. **Stage 2, tinkering, is the same browser runtime behind a notebook surface.**
Scripts and results move between stage 1 and stage 2 through browser storage.
It is anonymous.
3. **Stage 3, HPC submission, goes through a NEMAR gateway** and is the only stage that requires
ORCID identity, the service tier, and the export-control review that website ADR 0010 reserves for compute.

**eegprep is the default engine in every lane**, browser included,
once its pure-Python core installs under Pyodide.
MNE is the fallback until then, and the assistant names the engine it used.

**OSA owns the execution runtime, and it is community-agnostic.**
NEMAR contributes a runtime configuration block, a pure-Python recipe reader, and the MCP.
The chatbot runs wherever OSA runs; nemarring is not a requirement.

**Carried forward from 0025, unchanged:**
the MCP is a stateless, recipe-first broker on Workers;
the Worker never decodes signal data beyond a capped taste;
bulk bytes go direct to S3.
NEMAR serves bytes and understanding.
It now also hands the user a place to run the science, in their own browser, but it does not run it for them.

## Consequences

- **eegprep packaging is on the critical path.**
Move `oct2py`, `pyedflib`, and `psutil` into extras, keep the lazy imports,
and add a CI job that installs eegprep under Pyodide and runs the EEGLAB-parity tests there.
ICA needs no new dependency: `runica` and `eeg_picard` are pure numpy and scipy, and `python-picard`
falls back to numpy when `numexpr` is absent. Browser ICA speed depends on routing matrix products
through scipy's OpenBLAS, because Pyodide builds numpy without BLAS; ICLabel needs a torch-free
inference path before it is available in the browser.
Until it lands, the browser lane uses MNE; that is a tracked gap, not accepted drift.
- **Browser reads are HTTPS only.** WebAssembly has no sockets,
so `boto3`-based clients (eegdash, and nemar-py as packaged today) are not browser clients.
Recipes must carry the HTTPS `data_base`, which index v3 does,
and a small pure-Python reader that turns a recipe into a numpy array is needed.
- **Every executing origin must be allowed to read the data plane.**
The S3 bucket CORS policy and the `zarr.nemar.org` browser-origin allowlist must cover
`nemar.org` for the embed and `osc.earth` if the widget executes there.
Allowlisting an origin routes its bytes through the Worker; widening S3 CORS does not.
Decide per origin.
- **Memory is the binding constraint, not CPU.** wasm32 tops out between 2 and 4 GB,
so the assistant asks the MCP for a slice or the view pyramid, never a whole recording.
The read cap and recipe design in #1065 is what makes the browser lane feasible.
- **The backend's compute-adjacent roles are exactly three:**
recipes through the MCP, the origin allowlist, and the stage-3 gateway
(website #6's NSG connector sketch is the prior art).
No kernels, no sandboxes, no job runners in the Worker, and no per-user compute on the OSA host.
- **History lives in browser storage.** Syncing it for signed-in users is a later option, never a requirement.
- **Costs accepted:** a first load of roughly 50 to 60 MB for the scientific stack, cached afterward;
no threads or GPU in the browser;
two OSA design items, interrupt-and-resume tool calls and a checkpointer,
precede any NEMAR-visible feature.

## Alternatives considered

- **Server-side execution as a tool on the OSA host.**
The host is a thin CPU that orchestrates LLM calls; per-user numpy saturates it at a handful of sessions,
model-written code on a shared host needs real sandboxing, and egress would scale with users. Lost.
- **Hosted per-user kernels behind the service tier.**
A coherent continuum, but it makes NEMAR a compute service, adds an ops surface,
and gates stages 1 and 2 behind login. Rejected by the owner; only stage 3 is gated.
- **Server for stage 1, Pyodide for stage 2 only.**
Two engines, two package sets, and results that do not carry over at the handoff that matters. Lost.
- **Client-side agent loop (the QP pattern).**
Moves the whole loop into the browser and loses server-side retrieval tools, budget enforcement, and threads.
Kept as the fallback if interrupt-and-resume proves unworkable.
- **MNE as the permanent browser engine.**
Accepts the drift 0025 accepted. Lost, because the `oct2py` premise was wrong and the owner controls eegprep.

## Receipts

- ADR 0025 (superseded); `.context/draft-zarr-inference-ecosystem-plan.md` sections 2 and 6.
- eegprep 0.3.0 on PyPI is `py3-none-any`; lazy `oct2py` import at `src/eegprep/eeglabcompat.py`;
hard dependencies in `pyproject.toml` (sccn/eegprep, checked 2026-09-08).
- Pyodide's package list includes numpy, scipy, matplotlib, h5py, sympy, zarr, numcodecs, pyarrow, fsspec, sqlalchemy;
MNE 1.12.1 and mne-bids 0.19.0 are pure Python (PyPI `requires_dist`, checked 2026-09-08).
- OSA design note: `.context/browser-execution-tool-design.md` in OpenScience-Collective/osa,
and the client-executed `execute_python_code` prior art in its `.context/tool-system-guide.md`.
- website ADR 0010 (service tier is upload plus compute, export-control review); website #6 (NSG connector).
- nemarOrg/nemar-cli#1065 (MCP), #1063 (User-Agent convention).
Verified 2026-09-08: `api.nemar.org` and `zarr.nemar.org` return 403 to the `Python-urllib` User-Agent only.
- Catalog on 2026-09-08: 623 datasets serve Zarr, 42 are fidelity-verified,
so `has_zarr` is the availability filter and the verify status rides in the provenance envelope.
149 changes: 149 additions & 0 deletions .context/decisions/0050-no-wasm-in-the-worker-bundle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
# ADR 0050: No WebAssembly in the MCP Worker bundle, and real workerd is the only gate that proves it

**Status:** accepted
**Date:** 2026-09-09
**Owner:** Seyed Yahya Shirazi

## Context

The NEMAR Model Context Protocol (MCP) server (epic #1065) runs inside the same Cloudflare Worker
as the rest of the backend. Cloudflare's runtime restricts dynamic code generation, and
WebAssembly (WASM) compilation falls under that restriction, but NOT uniformly, and the difference
turns out to matter:

- **Compiling WASM inside a request is disallowed unconditionally.** No compatibility date or flag
changes it.
- **Compiling WASM at module load is disallowed for THIS worker specifically**, because
`backend/wrangler-sccn.toml` pins `compatibility_date = "2024-12-01"`. The
`allow_eval_during_startup` flag permits it, and became the DEFAULT as of `2025-06-01` (the
runtime says so in as many words when you specify it past that date: "became the default as of
2025-06-01 so does not need to be specified anymore").

Either way, a Worker that tries does not degrade; it fails.

Twice, in two different phases, a plausible npm package turned out to be unusable this exact way,
and in neither case could `bun test` have told us:

- **Phase 1, `numcodecs`, evaluated in the transport spike and never adopted.** Its Blosc codec
loads its WASM module through a runtime `fetch()` followed by `WebAssembly.instantiate()` on the
fetched bytes, and the published tarball ships no `.wasm` file to import instead. Under real
workerd: `RuntimeError: Aborted(CompileError: WebAssembly.instantiate(): Wasm code generation
disallowed by embedder)`, on EVERY `decode_chunk` call. It also cost about 604 KiB of bundle for
no benefit. It was only ever in `backend/spike/mcp-transport/package.json`, never in
`backend/package.json`: the spike existed to answer this question and did.
- **Phase 3, `hyparquet-compressors`, installed during development and removed before merge.** Its
`compressors` export eagerly constructs `hysnappy`'s `snappyUncompressor()` at MODULE LOAD, which
synchronously compiles a WASM module: `WebAssembly.Module(): Wasm code generation disallowed by
embedder`. This is the worse shape of the two, because it crashed isolate startup for EVERY
request rather than only a `get_events` call, and regardless of whether any dataset's parquet
actually used SNAPPY. It was caught by the smoke script while `bun test` stayed fully green
(PR #1326), and never reached `backend/package.json`.

Neither was caught by `bun test`, and neither could be. Bun compiles WASM happily, so the suite
stayed green throughout the window `hyparquet-compressors` was installed, and nothing in the suite
exercised `numcodecs` at all. Phase 3's shape would have shipped a worker that could not start;
phase 1's would have failed every decode call. Both were found by running the real runtime, which
is the only thing that can find them.

## Decision

**No package in this server's dependency graph may compile or instantiate WebAssembly, at import or
at runtime.** When a dependency needs WASM for a codec we need, we implement the codec in pure
JavaScript or use a pure-JS package instead, and we say so where the substitution lives.

**A phase that adds or upgrades a dependency is not done until `backend/scripts/mcp-smoke.sh`
has passed under real workerd.** `bun test` is necessary and not sufficient; it cannot observe this
class of failure at all.

The two substitutions this rule has already produced:

- `backend/src/services/blosc-decode.ts` decodes blosc/zstd/shuffle int16 chunks in pure
JavaScript (a blosc header parse plus `fzstd` plus an unshuffle), replacing `numcodecs`.
- `backend/src/mcp/tools/get-events.ts` builds a one-entry `{ ZSTD }` compressors map over
`fzstd` instead of importing `hyparquet-compressors`. That the archive is entirely ZSTD is
measured, not assumed: `bun run zarr:geometry-check` reads every published `events.parquet`
footer and reports the codec of every column chunk (row group x column).

## Consequences

Easier: the bundle stays small, isolate startup stays fast, and a dependency bump cannot silently
take the server down in a way tests bless.

Harder: we own codec code we would otherwise get for free, and we must keep it correct. The
mitigation is that these decoders are narrow, pure, and tested against real captured production
bytes wherever a real store exercises the path, with synthetic chunks only for shapes the catalog
does not currently contain (a short trailing view chunk, for instance).

**The platform is only half-enforcing this rule for us, and a date bump would stop it enforcing
the other half.** Request-time compilation stays impossible no matter what. But the module-load
half, the shape that kills isolate startup, is currently blocked only because our
`compatibility_date` predates `2025-06-01`. Measured under `wrangler dev --local`: with
`compatibility_date = "2024-12-01"` plus `allow_eval_during_startup`, or with any date from
`2025-06-01` on, the phase 3 entry module compiles its WASM and the worker starts normally. So
**a routine `compatibility_date` bump silently removes the guard that caught the phase 3 failure**,
which is precisely why this rule is written down and gated by the smoke script rather than left to
the runtime to police.

New obligations: every dependency addition or upgrade in this subsystem runs the workerd smoke
before merge, and a reviewer who sees a new dependency should ask what it does at module load.
Reading a package's import-time side effects is now part of the review, not an optimization.

The smoke script currently drives a throwaway entry module
(`backend/scripts/mcp-smoke-entry.ts`) rather than `backend/src/index.ts`, because
`wrangler dev --local` cannot start the real entry in this environment (issue #1324, unrelated to
the MCP work). `index.ts` carries three non-handler named exports (`NON_PROD_SANDBOX_CLEANUP_QUERY`
and `PROD_SANDBOX_CLEANUP_QUERY`, both strings, and `DEV_CRON_ALLOWLIST`, a `readonly string[]`),
and this workerd build rejects any named export on an entry module that is not a function or an
`ExportedHandler`; fixing only the strings would not fix the issue. That is a real gap in the gate:
it proves the MCP sub-app and its dependency graph start under workerd, not that the whole worker
does. Closing #1324 closes the gap.

## Alternatives considered

- **Allow WASM by widening the compatibility settings.** This works, for exactly one of the two
shapes. Measured under `wrangler dev --local` on the phase 3 entry module: adding
`allow_eval_during_startup` to `compatibility_flags` (or moving `compatibility_date` to
`2025-06-01` or later, where it is the default) lets `hysnappy` compile at load and the worker
starts and answers 200; without it the worker never starts. Rejected anyway, on three grounds:
it grants dynamic code generation to the whole worker at startup to buy one codec we already
have in 15 lines of pure JavaScript; it does nothing for the phase 1 shape, since request-time
compilation stays disallowed under every combination tested; and it would make a
`compatibility_date` bump a load-bearing security-adjacent decision rather than routine
maintenance. Note that the flag's documented text names only `eval()` and `new Function(text)`
and never mentions WebAssembly, so the WASM behavior above is MEASURED, not documented; a future
reader who consults only the docs page will reach the wrong conclusion.
- **A statically imported `.wasm` module**, which Cloudflare does support (Wrangler bundles
`.wasm` as a `CompiledWasm` module). Neither offending package ships a `.wasm` file to import,
so adopting this would mean vendoring and maintaining a fork of each.
- **Keep the WASM packages and lazily load them only on the paths that need them.** Lazy USE does
not help the phase 3 case at all: `hyparquet-compressors/src/index.js` statically re-exports
`compressors.js`, so importing anything from the package root runs the eager compile, and the
failure is at module load taking down every request. A subpath import that never touches
`compressors.js` was technically available (the package declares no `exports` map), but
depending on an undeclared internal path is more coupling than a one-entry map over `fzstd`,
which we already pin. Lazy loading would also leave a landmine for whoever later imports the
module from a hot path.
- **Trust `bun test` and catch it in staging.** This is what would have happened without the
smoke gate, and it is why the rule exists: the phase 3 failure crashed isolate startup for
every request, so the first symptom would have been a fully dead deployment.
- **A dependency allowlist enforced in CI.** Considered and deferred as premature. The check
that actually catches the failure is running the runtime; a list of forbidden package names
would need maintaining and would not catch a transitive dependency that adds WASM in a patch
release.

## Receipts

- `.context/mcp-server-design.md` section 9 (dependency table, both offending rows, including that
`numcodecs` was the spike's path (a) and never a dependency of the real server), section 10
(phase 1 decode-path measurements and the bundle figures), section 10.3 (the phase 3 failure).
- `backend/src/services/blosc-decode.ts`, `backend/src/mcp/tools/get-events.ts`,
`backend/scripts/mcp-smoke.sh`, `backend/scripts/mcp-smoke-entry.ts`.
- PR #1326 (the phase 3 failure, caught by the smoke script with `bun test` green).
- Issue #1324 (`wrangler dev --local` cannot start `backend/src/index.ts`).
- The compatibility-flag measurements in this ADR were taken on 2026-09-09 with
`wrangler dev --local` against a two-line entry module importing `hysnappy@1.0.0`, at module
load and inside `fetch()`, across `compatibility_date` `2024-12-01` and `2025-11-09` with and
without `allow_eval_during_startup`.
- ADR 0049 for the posture this server inherits: a stateless, recipe-first broker whose job is
to hand a client the information to fetch bytes itself, which is what keeps the Worker's
dependency surface small enough for this rule to be practical.
Loading
Loading