Repository navigation
Phase 8: zarr fidelity verification sweep - #1203
Merged
neuromechanist merged 15 commits intoSep 2, 2026
Merged
neuromechanist merged 15 commits into
neuromechanist merged 15 commits into
Conversation
📦 Package PreviewYour package is ready to test! bun install -g nemar-cli@PR1203Version: 🤖 Updated on each push. |
neuromechanist
force-pushed
the
feature/issue-1068-phase8-fidelity-sweep
branch
from
September 2, 2026 18:38
b731361 to
eda78be
Compare
neuromechanist
merged commit Sep 2, 2026
360c2c7
into
feature/issue-1181-epic-zarr-serving
12 checks passed
This was referenced Sep 2, 2026
Closed
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
backend/src/services/zarr-fidelity-sweep.ts,runZarrFidelitySweep): re-derives ground truth per sampled recording from the dataset's ownchannels.tsvand modality sidecar (*_eeg.json/*_ieeg.json/*_meg.json/*_emg.json), fetched via the public, credential-freeraw.githubusercontent.comcontent host (never the GitHub API/App/PAT, which is what keeps this dev-cron-safe), and compares it against the publishedindex.json.verified/failed/unverifiable) plus a bounded mismatch-examples list into thesweep_stampsJSON column only (ADR 0034/0035 -- no new column), and re-arms verification on a re-conversion (zarr_source_commitchange).has_zarrkeeps its existing meaning (converted); the new, separatehas_zarr_verifiedfilter (backend + CLI--has-zarr-verified) is "converted AND verified".zarr_verify_status/zarr_verified_atare additive on the catalog/detail contract and on everyzarr-catalog.jsonentry, with a.superRefine()guardingzarr_verified_atagainst a nullzarr_verify_status.POST /admin/datasets/zarr-fidelity-sweepand CLInemar admin zarr-fidelity-sweep [--limit N], both sharing the one sweep implementation with the daily cron (added toDEV_CRON_ALLOWLIST-- it only reads S3 + the public content host and writes this env's own D1).failedverdict, writes anaudit_logrow (zarr_fidelity_failed) and includes the dataset in the sweep's response/cron log line for the observability dashboard;#1068is not closed (the re-measurement after the engine bump remains).Verdict rules
channel_count_mismatch: total store channels (summed across a store's groups, matchinggenerate_zarr.py'sstore_total_channels) fall short ofchannels.tsv's data-row count.duration_mismatch: the store's longest groupduration_sdisagrees with the sidecar'sRecordingDurationby more than 1 second.rate_mismatch: a group's servedratedisagrees withmin(SamplingFrequency, MODALITY_RATES[modality])(EEG/MEG 250 Hz, IEEG/EMG 1000 Hz).sampleEvenly) plus every store with an_channels === 1group. Verdict isunverifiablewhen no ground truth was reachable for any sampled store,failedwhen any mismatch was found, elseverified.Test plan
backend/test/zarr-fidelity-sweep.test.ts: pure-function unit tests (bidsSidecarCandidates,zarrFidelityStoreChannelTotal/Duration,zarrFidelitySelectSample,pushZarrFidelityExample) plus real-engine tests (freshDb/realD1 + a real localBun.serve()answering both the signed S3 index GET and the raw-content sidecar GET) covering verified / each mismatch code / unverifiable / v1-v3 index tolerance / the 20-entry-4KB example cap / re-conversion re-arming / an absent index / an invalidsource_commit/ the limit clamp. Mutation-proved (flipped the channel-count comparison; the two dependent tests failed as expected, then reverted).backend/test/zarr-fidelity-sweep-route.test.ts: real admin route tests (auth 401/403, a thrown dependency becomes 500, and a real success path against the same local receiver with the stamp verified in D1).backend/test/cron-sweep-wiring.test.ts: the sweep is registeredall-envs, wired outside the prod-only block, inDEV_CRON_ALLOWLIST, wrapped inctx.waitUntil, and its chain has a.catch.backend/test/catalog-has-zarr.test.ts:has_zarr=1is unchanged (still every converted row regardless of verify status);has_zarr_verified=1returns onlyverifiedrows; both fields present on list and detail rows.test/contract-schemas.test.ts: the two new fields parse; thezarr_verified_at-without-zarr_verify_statuspair is rejected on bothcatalogItemSchemaanddatasetDetailSchema.backend/test/zarr-catalog.test.ts: the candidate SQL projects the two fields fromsweep_stamps;buildZarrCatalogcarries them per entry without filtering the population.test/zarr-fidelity-sweep-cli.test.tsandtest/has-zarr-cli-flag.test.ts: real subprocess CLI invocations (bun run src/index.ts admin zarr-fidelity-sweep,... dataset list/search --has-zarr-verified) against a local capture server, covering auth, request shape, output, and exit codes.cd backend && bun test): 1495 pass, 0 fail.tsc --noEmitclean on every touched/new file.Addresses #1068 (verification sweep and has_zarr semantics)Part of epic #1181Docs pointer (for nemarOrg/docs)
has_zarris unchanged; documenthas_zarr_verifiedas the strict "converted AND verified" filter, on both the REST API (GET /datasets?has_zarr_verified=1) and the CLI (--has-zarr-verified).zarr_verify_status(verified/failed/unverifiable/null) andzarr_verified_aton the dataset catalog/detail response and onzarr-catalog.jsonentries -- null until the daily sweep (orPOST /admin/datasets/zarr-fidelity-sweep) reaches a freshly-converted dataset.nemar admin zarr-fidelity-sweep [--limit N].