You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The website "Has HED annotation" filter (FilterSidebarhas_hed) is a dead control: filterStateToAPIQuery() never sends it and applyClientFilters() has no predicate, so checking it narrows nothing and non-HED datasets (e.g. on007058) still appear. There is no HED field in D1 or the /datasets API today.
Goal
Detect Hierarchical Event Descriptors (HED) annotations per dataset VERSION, store them first-class, and make datasets filterable by HED presence from both the website and the CLI.
Detection rule (decided)
has_hed = true only when BOTH hold:
dataset_description.json has a HEDVersion key, AND
at least one real HED annotation exists: an *_events.json column with a "HED" key, OR an *_events.tsv with a HED column.
hed_version = the HEDVersion string (e.g. "8.3.0").
This high-precision AND is what eliminates the false positives.
Storage (per-version, denormalized latest)
Per-version truth on dataset_versions (has_hed, hed_version).
Denormalized latest-version value on datasets (has_hed, hed_version, hed_checked_at) so list filtering stays a single-table query (mirrors modality/license_tier).
Per-version matters: e.g. nm000132 v1.0.0 lacks HED but v1.1.1 has it.
Template
Follow the channel/montage epic #854 end-to-end: migration (0054/0055 style) -> probeHed() in getBidsTreeStats (github.ts) next to probeChannelMontage -> computeDatasetMetadataColumns/writeDatasetMetadataColumns (dataset-metadata-columns.ts, both enrich + version-DOI callers) -> admin backfill sweep -> /datasets projection + filter -> CLI flag.
Phases
Schema: D1 columns on datasets + dataset_versions, indexes, type/projection plumbing (NULL until populated).
Detection/compute: probeHed() + thread through compute/write for both callers; write per-version row + denormalized latest.
Backfill sweep: POST /admin/datasets/hed-sweep with hed_checked_at resumability + nemar admin subcommand.
API filter + CLI: /datasets?has_hed=1 server filter + projection; nemar dataset list/search --hed + HED badge.
Dependent follow-up (separate repo)
nemarOrg/website: send has_hed in filterStateToAPIQuery, add has_hed to the Dataset type, make the checkbox live. Tracked separately.
Epic complete. All 4 phases (#870-#873) merged into the epic branch; epic merged into dev via PR #878 (regular merge, preserving phase history). Cross-phase finalize review fixed 2 seam bugs. Follow-up: nemarOrg/website#139 wires the has_hed filter on the website.
Activity
Merge pull request #878 from nemarOrg/feature/issue-869-epic-hed-dete…
neuromechanist commentedon Jun 30, 2026
Epic complete. All 4 phases (#870-#873) merged into the epic branch; epic merged into dev via PR #878 (regular merge, preserving phase history). Cross-phase finalize review fixed 2 seam bugs. Follow-up: nemarOrg/website#139 wires the has_hed filter on the website.
Merge pull request #879 from nemarOrg/dev