Skip to content

Rebuild directory recordings on sidecar edits - #1123

Merged
neuromechanist merged 2 commits into
devfrom
fix/issue-1106-events-rebuild-dir-recordings
Aug 22, 2026
Merged

neuromechanist merged 2 commits into
devfrom
fix/issue-1106-events-rebuild-dir-recordings

Conversation

@neuromechanist

Copy link
Copy Markdown
Contributor

Summary

compute_worklist built its by_dir index from file heads only, so affected_primaries received an empty bucket for a CTF .ds, MEF3 .mefd, or 4D/BTi recording. Editing the _events.tsv beside one queued nothing — the recording kept serving stale annotations until the next --clean full rebuild happened to pick it up. That papering-over is why the bug survived: the hourly cron runs --clean, so the incremental path was wrong without ever being visibly wrong.

Reproduced before fixing, across all three formats:

Format Before After
CTF .ds [] ['sub-01/meg/sub-01_task-x_meg.ds']
MEF3 .mefd [] ['sub-01/ieeg/sub-01_task-x_ieeg.mefd']
4D/BTi [] ['sub-01/meg/sub-01_task-x_meg']

Why this fix

A directory recording sits in the same parent directory as its sidecars (sub-01/meg/..._meg.ds next to sub-01/meg/..._events.tsv), so keying by_dir on all buildable primaries makes the existing entities-base matching resolve it with no change to affected_primaries itself.

Changes inside a recording directory never reach this map — dir_recording_of and the bti_dirs check resolve those earlier and continue. The map is consulted only for siblings alongside the recording, which is precisely the sidecar case, so widening it does not affect any other path.

Test plan

Ten tests in TestSidecarRebuildsDirectoryRecordings:

  • events edit rebuilds CTF .ds, MEF3 .mefd, and BTi
  • events added rebuilds
  • events deleted still rebuilds, so the recording is regenerated without the stale events block rather than left carrying them
  • a file primary and a directory recording sharing one directory both rebuild
  • four negative guards: different task entity, sibling recording with different entities, sidecar in another subject's directory, and a change inside the recording directory still resolving through the earlier path

Verified the tests are not tautological: reverting the one-line by_dir change fails 7 of 10. The 3 that pass either way are the negative guards, by design.

  • Full suite green: 275 tests in test_generate_zarr.py (was 265), plus test_zarr_queue.py, test_purge_non_raw_stores.py, test_patch_power_line_frequency.py

Closes #1106. Part of epic #1095.

compute_worklist keyed by_dir on file heads only, so affected_primaries
got an empty bucket for a CTF .ds, MEF3 .mefd, or 4D-BTi recording and an
events or companion edit beside one queued nothing at all. The recording
kept serving stale annotations until the next --clean full rebuild, which
is why this stayed invisible.

Key by_dir on all buildable primaries instead. Changes inside a recording
directory never reach this map (dir_recording_of and bti_dirs resolve them
first), so it is consulted only for siblings alongside the recording,
which is exactly the sidecar case.

Ten tests: all three directory formats, add/modify/delete of the events
file, and four negative guards against over-matching. Seven fail without
the fix; the three guards pass either way by design.

Closes #1106
@github-actions

github-actions Bot commented Aug 22, 2026 •

Copy link
Copy Markdown

📦 Package Preview

Your package is ready to test!

bun install -g nemar-cli@PR1123

Version: 0.9.9-PR1123.4df274b
Commit: 4df274b


🤖 Updated on each push.

Three findings from review of #1123.

The fix comment said an events OR COMPANION edit rebuilt nothing. Only
events is right: companion extensions are EEGLAB/BrainVision-specific, so
reaching a directory recording that way would need a same-stem .fdt/.eeg/
.vmrk beside a .ds/.mefd/BTi dir, which is not valid BIDS. Overstated the
bug's scope.

affected_primaries' docstring still said primaries_by_dir holds only
buildable heads. This PR is what made that false, so it is corrected here
rather than left to drift, including why a directory recording reaches
only the companion and events branches.

Added a session/run/acq test. The other fixtures are flat sub-01, which is
not how CTF/MEF3 data arrives; these are exactly the multi-session,
multi-run formats and that matching had never been exercised on a
directory recording.
@neuromechanist
neuromechanist merged commit 1580f72 into dev Aug 22, 2026
12 checks passed
@neuromechanist
neuromechanist deleted the fix/issue-1106-events-rebuild-dir-recordings branch August 22, 2026 17:11

This branch was previously deployed

1 inactive deployment
npm — 4df274bf Deployed Aug 22, 2026 by neuromechanist via publish #2039
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant