Repository navigation
Widen the BTi dispatch gate so it stays a superset of the converter - #1105
Merged
Merged
Conversation
The converter's bti_recordings keys a 4D/BTi directory on a c,rf* file plus a sibling config, with NO constraint on the directory name. The gate required the parent segment to end in _meg, making it NARROWER than the converter: a BTi directory named anything else would be converted but would never re-dispatch on a push, so its serving copy would go stale silently. That is the same failure mode the missing KIT extensions caused, which is what this whole gate fix started from. The gate cannot evaluate the sibling-config rule because it sees one changed path at a time rather than the tree, so it takes the permissive side: a false positive costs one no-op workflow run, a false negative costs correctness. The _meg narrowing came from a review suggestion that was locally reasonable but violated the cross-repo invariant. See #1103.
Review found the gate still was not a complete superset: the converter rebuilds a BTi recording on ANY touched file inside a qualifying directory, and has tests for a config-only edit and an hs_file deletion both rebuilding. The gate matched only c,rf*, so those pushes would not re-dispatch. Match config/hs_file when the parent segment ends in _meg. That is the narrowest rule covering the BIDS-conformant case, and matching them at any depth is not an option because .datalad/config exists in essentially every dataset repo and would fire on nearly every metadata-only push across ~785 repos. Purely additive to the name-independent c,rf* match, so it cannot re-narrow the gate. Residual accepted gap, now documented in the code: a config/hs_file-only change in a BTi directory NOT named _meg still will not re-dispatch. It self-heals on the next dispatching push, since the workflow diffs last-converted against HEAD rather than trusting the event payload.
📦 Package PreviewYour package is ready to test! bun install -g nemar-cli@PR1105Version: 🤖 Updated on each push. |
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a superset violation I introduced in #1101. The 4D/BTi dispatch gate was narrower than the
converter, which is the same silent-staleness failure mode that the missing KIT extensions caused, and
which #1101 existed to fix in the first place.
The divergence
bti_recordings,generate_zarr.py)c,rf*file plus a siblingconfig, no constraint on the directory nameisBtiMember, before this PR)c,rf*basename plus parent segment ending in_megSo a 4D/BTi recording directory not named
..._meg/would be discovered and converted, but a pushtouching it would never re-dispatch, leaving its serving copy stale with no error anywhere. Exactly
what happened with
.con/.sqd/.kdf.The fix
Match
c,rf*at any depth. The gate cannot replicate the converter's sibling-configrule, becauseit evaluates one changed path at a time rather than the whole tree, so it takes the permissive side of
the asymmetry: a false positive costs one no-op workflow run; a false negative costs correctness.
A top-level
c,rfDCstill does not match, since it sits inside no directory and therefore cannot be arecording member.
configandhs_fileare still deliberately unmatched, so.datalad/configcannotfalse-positive.
How this happened, since it is worth recording
The
_megnarrowing came from a code-review suggestion on #1101 to close a theoretical any-depthfalse positive. That was locally reasonable and I applied it, but it was globally wrong: it traded a
harmless false positive for a silent false negative across a cross-repo boundary the reviewer could
not see. The converter-side rule was written afterwards, in
nemarDatasets/.github#100, and its authorflagged this exact risk class as unverified in their report, which is what prompted the check.
This is the second time this invariant has been violated and nothing but a comment guards it.
That is tracked in #1103.
Test plan
bun test test/webhook-zarr-dispatch.test.ts— 31 pass, 0 failtsc --noEmitclean at root and inbackend/with the reasoning inline so a future reviewer does not re-narrow it
Part of epic #1095