Repository navigation
Description
Symptom
For on* datasets (and any dataset whose root BIDS metadata is small), data.nemar.org/<id>/<v>/manifest.json returns ZERO root files even though those files exist in the git tree at the tagged commit.
Example: `on005262` v1.0.0.
GitHub tree at refs/tags/v1.0.0 (verified via gh api repos/nemarDatasets/on005262/git/trees/c4ddc459...):
blob 100644 427 .gitattributes
blob 100644 66 CHANGES
blob 100644 432 README.md
blob 100644 461 dataset_description.json
data.nemar.org/on005262/v1.0.0/manifest.json contains 559 entries — all under sub-*/ or .nemar/. None of the root files appear.
For comparison, `nm000104` v2.0.0 has 5 root files in its manifest (README.md is 9763 bytes there).
Root cause
backend/src/services/manifest.ts lines 85-150 (generateManifest).
Walkthrough for README.md (432 bytes):
- Line 100-107 — file lands in
pointerCandidatesbecause20 < 432 < 500and path doesn't start with.git. - Line 115-133 — pointer-resolution loop fetches the blob, calls
parseAnnexPointer(content). Returnsnullbecause it's a real README, not an annex pointer. Nothing is added tofiles. - Line 136-141 —
regularFilesfilter excludes anything inpointerCandidates. So README.md is excluded from this loop too.
Result: README.md is silently dropped from the manifest. Same path for CHANGES (66B), dataset_description.json (461B).
nm000104 doesn't hit this because its root files are larger:
- README.md = 9763 bytes (> 500, skips the pointer-candidate filter, lands cleanly in
regularFiles) - dataset_description.json = 1353 bytes (same)
- participants.tsv = 2291 bytes (same)
So the bug only surfaces for datasets with small BIDS docs at root — common for OpenNeuro mirrors.
Secondary bug
!entry.path.startsWith(".git") on line 138 also catches .gitattributes because of the substring match. The intent was clearly .git/ (with trailing slash) since .github/ is excluded separately. As written, .gitattributes is excluded too.
Fix sketch
After the pointer-resolution loop, retain candidates whose parseAnnexPointer returned null and add them to regularFiles. Two-line change:
// Inside the inner loop, track the misses:
const nonAnnexCandidates: Array<typeof entry> = [];
for (const { entry, key } of results) {
if (key) {
files[entry.path] = { key, size: ..., checksum: ... };
} else {
nonAnnexCandidates.push(entry); // <-- new
}
}
// Then in regularFiles filter, also include the misses:
const regularFiles = [
...nonAnnexCandidates,
...entries.filter(
(entry) =>
!entry.path.startsWith(".git/") && // <-- trailing slash
entry.path !== ".gitattributes" && // explicit allow (or just keep this in)
!entry.path.startsWith(".github/") &&
!pointerCandidates.some((p) => p.path === entry.path),
),
];Or cleaner: do a single pass over entries, decide per-entry whether it's annex or git-tree based on the content check result, and skip nothing.
Impact
Affects every on* dataset's detail page on the new nemar.org (nemarOrg/website Phase 2 just shipped). The website now falls back to fetching the README from the dataset's GitHub repo when it isn't in the manifest, but that loses the per-version snapshot guarantee — fixing this server-side restores it.
Also affects rclone / aria2c users syncing via data.nemar.org who won't see README/CHANGES/dataset_description.json in the file list.
Verification
After the fix, curl -s https://data.nemar.org/on005262/v1.0.0/manifest.json | jq '[.[] | select(.path | contains("/") | not)]' should return 4 entries (.gitattributes, CHANGES, README.md, dataset_description.json).
Related: #449 (the data.nemar.org epic), nemarOrg/website#1 (detail page).
Activity
fix(manifest): include small regular git-tree files at root
Fix: manifest generator drops small regular git-tree files at root (#510
neuromechanist commentedon May 15, 2026
Fixed in #510 (squash-merged to dev as 99f21e2). Will reach main on the next dev→main release. Verified end-to-end:
.gitattributesno longer accidentally excluded by the.gitsubstring match.backend/test/manifest-small-root-files.test.ts(all green in CI).Step toward #448 (OpenNeuro mirror DOI mapping), not a complete fix for that epic.