Skip to content

Phase 1: HED schema + projection plumbing - #874

Merged
neuromechanist merged 2 commits into
feature/issue-869-epic-hed-detectionfrom
feature/issue-870-phase1-hed-schema
Jun 30, 2026
Merged

neuromechanist merged 2 commits into
feature/issue-869-epic-hed-detectionfrom
feature/issue-870-phase1-hed-schema

Conversation

@neuromechanist

Copy link
Copy Markdown
Contributor

Summary

Lays the schema + read-path groundwork for per-version HED detection. Adds first-class HED columns and exposes them through the /datasets API, leaving them NULL until phase 2's probeHed() populates them. Mirrors channel/montage phase 1 (#857).

  • Migration 0056_hed_columns.sql: datasets ADD has_hed INTEGER, hed_version TEXT, hed_checked_at TEXT + idx_datasets_has_hed; dataset_versions ADD has_hed INTEGER, hed_version TEXT. Per-version truth on dataset_versions; latest-version denormalized onto datasets for single-table list filtering.
  • /datasets projection: project d.has_hed, d.hed_version in both the --mine browse query and the main list query, next to n_channels/electrode_system. Raw (not COALESCE'd) so NULL = "not classified yet".

No detection, writes, filter, or CLI here — those are phases 2-4. Columns stay NULL.

Closes #870
Part of epic #869

Test plan

  • New backend/test/hed-columns-migration.test.ts (6 tests, real bun:sqlite, no mocks): asserts the new columns + index exist, round-trips a populated row, NULL semantics for unpopulated rows, and the per-version case (v1.0.0 null, v1.1.1 set).
  • bun test test/migrations.test.ts still green (no regression).
  • bunx biome check clean on changed files.
  • bun run typecheck (backend) clean.

@neuromechanist

Copy link
Copy Markdown
Contributor Author

Review summary (3 agents: code, tests, comments — Sonnet)

No critical or important issues in the code itself. All actionable findings addressed in 301b5f3:

Addressed

  • has_hed CHECK constraint (tests/code): added CHECK (has_hed IN (0,1)) to both datasets.has_hed and dataset_versions.has_hed (NULL still passes), plus a test that an out-of-domain value (2) is rejected.
  • has_hed = 0 vs NULL (tests): added a round-trip test on both tables proving checked-but-no-HED (0) is distinct from not-classified-yet (NULL) — the load-bearing sweep-skip distinction.
  • hed_checked_at IS NULL predicate (tests): added a candidate-selection test (unswept selected, stamped drops out) so the schema phase stands on its own ahead of phase 3.
  • HEDVersion array (comments): documented in the migration that HEDVersion may be a scalar or array, and that phase 2 stores the array form comma-joined.
  • Stale wire-shape counts (code/comments): corrected ?mine 24→25 and list 29→30 (both were already off by one pre-PR) and added the #869 HED enumeration entry.
  • Test docstring (comments): items 4/5 clarified (include hed_checked_at; "without setting the new columns" wording).

Skipped (with reason)

  • Route-level projection test: no such template exists in the repo — n_channels/electrode_system (migration 0054) have no GET /datasets wire-shape test either. The projection is a pure additive column pair with no conditional logic. If a wire-shape snapshot test is ever added, has_hed/hed_version should be included.
  • Phase-2 function-name forward references in the migration comment: intentional spec for phase 2; will be updated in the phase-2 PR if names change.

Local gates: 31 tests pass (hed-columns-migration + migrations), Biome clean, backend typecheck clean.

@github-actions

Copy link
Copy Markdown

📦 Package Preview

Your package is ready to test!

bun install -g nemar-cli@PR874

Version: 0.8.73-PR874.301b5f3
Commit: 301b5f3


🤖 Updated on each push.

@neuromechanist
neuromechanist merged commit 558bf43 into feature/issue-869-epic-hed-detection Jun 30, 2026
1 check passed
@neuromechanist
neuromechanist deleted the feature/issue-870-phase1-hed-schema branch June 30, 2026 01:03

This branch was previously deployed

1 inactive deployment
npm — 301b5f3a Deployed Jun 30, 2026 by neuromechanist via publish #1631
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant