Skip to content

Widen the BTi dispatch gate so it stays a superset of the converter - #1105

Merged
neuromechanist merged 2 commits into
devfrom
fix/bti-gate-superset
Aug 21, 2026
Merged

neuromechanist merged 2 commits into
devfrom
fix/bti-gate-superset

Conversation

@neuromechanist

Copy link
Copy Markdown
Contributor

Summary

Fixes a superset violation I introduced in #1101. The 4D/BTi dispatch gate was narrower than the
converter, which is the same silent-staleness failure mode that the missing KIT extensions caused, and
which #1101 existed to fix in the first place.

The divergence

Side Rule
Converter (bti_recordings, generate_zarr.py) c,rf* file plus a sibling config, no constraint on the directory name
Gate (isBtiMember, before this PR) c,rf* basename plus parent segment ending in _meg

So a 4D/BTi recording directory not named ..._meg/ would be discovered and converted, but a push
touching it would never re-dispatch, leaving its serving copy stale with no error anywhere. Exactly
what happened with .con/.sqd/.kdf.

The fix

Match c,rf* at any depth. The gate cannot replicate the converter's sibling-config rule, because
it evaluates one changed path at a time rather than the whole tree, so it takes the permissive side of
the asymmetry: a false positive costs one no-op workflow run; a false negative costs correctness.

A top-level c,rfDC still does not match, since it sits inside no directory and therefore cannot be a
recording member. config and hs_file are still deliberately unmatched, so .datalad/config cannot
false-positive.

How this happened, since it is worth recording

The _meg narrowing came from a code-review suggestion on #1101 to close a theoretical any-depth
false positive. That was locally reasonable and I applied it, but it was globally wrong: it traded a
harmless false positive for a silent false negative across a cross-repo boundary the reviewer could
not see. The converter-side rule was written afterwards, in nemarDatasets/.github#100, and its author
flagged this exact risk class as unverified in their report, which is what prompted the check.

This is the second time this invariant has been violated and nothing but a comment guards it.
That is tracked in #1103.

Test plan

  • bun test test/webhook-zarr-dispatch.test.ts — 31 pass, 0 fail
  • Biome clean on both changed files
  • tsc --noEmit clean at root and in backend/
  • The test that pinned the narrow behaviour is rewritten to pin the superset requirement instead,
    with the reasoning inline so a future reviewer does not re-narrow it

Part of epic #1095

The converter's bti_recordings keys a 4D/BTi directory on a c,rf* file plus a
sibling config, with NO constraint on the directory name. The gate required the
parent segment to end in _meg, making it NARROWER than the converter: a BTi
directory named anything else would be converted but would never re-dispatch on
a push, so its serving copy would go stale silently.

That is the same failure mode the missing KIT extensions caused, which is what
this whole gate fix started from. The gate cannot evaluate the sibling-config
rule because it sees one changed path at a time rather than the tree, so it takes
the permissive side: a false positive costs one no-op workflow run, a false
negative costs correctness.

The _meg narrowing came from a review suggestion that was locally reasonable but
violated the cross-repo invariant. See #1103.
Review found the gate still was not a complete superset: the converter rebuilds
a BTi recording on ANY touched file inside a qualifying directory, and has tests
for a config-only edit and an hs_file deletion both rebuilding. The gate matched
only c,rf*, so those pushes would not re-dispatch.

Match config/hs_file when the parent segment ends in _meg. That is the narrowest
rule covering the BIDS-conformant case, and matching them at any depth is not an
option because .datalad/config exists in essentially every dataset repo and would
fire on nearly every metadata-only push across ~785 repos. Purely additive to the
name-independent c,rf* match, so it cannot re-narrow the gate.

Residual accepted gap, now documented in the code: a config/hs_file-only change
in a BTi directory NOT named _meg still will not re-dispatch. It self-heals on
the next dispatching push, since the workflow diffs last-converted against HEAD
rather than trusting the event payload.
@github-actions

Copy link
Copy Markdown

📦 Package Preview

Your package is ready to test!

bun install -g nemar-cli@PR1105

Version: 0.9.9-PR1105.a779c13
Commit: a779c13


🤖 Updated on each push.

This branch was previously deployed

1 inactive deployment
npm — a779c13a Deployed Aug 21, 2026 by neuromechanist via publish #2002
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant