Repository navigation
Description
on006502 publishes no Zarr index: all 380 CTF .ds recordings fail with file_read_error, reason "Some of the mandatory HPI device-coordinate info was not there."
Found while resolving #1100. The data is intact and the head-position-indicator coordinates are present; only the labels naming them differ from what MNE accepts.
Root cause
MNE's _read_hc (mne/io/ctf/hc.py) classifies each coil point by substring-matching the descriptor line against a literal, case-sensitive table:
_kind_dict = {
"nasion": CTF.CTFV_COIL_NAS,
"left ear": CTF.CTFV_COIL_LPA,
"right ear": CTF.CTFV_COIL_RPA,
"spare": CTF.CTFV_COIL_SPARE,
}This dataset's .hc files use different vocabulary:
measured Nasion coil position relative to head (cm):
measured LPA coil position relative to head (cm):
measured RPA coil position relative to head (cm):
Nasion fails on case; LPA and RPA are not in the table at all. Every point falls through to kind = -1, so _make_ctf_coord_trans_set (mne/io/ctf/trans.py) finds no LPA, RPA, or NAS and raises.
Verified by running MNE's own parser against a recording from this dataset:
points parsed: 6
kind=-1 frame=CTF_DEVICE valid=True (x3)
kind=-1 frame=CTF_HEAD valid=True (x3)
Both coordinate frames are present and every point is valid. The information MNE says is missing is right there under a different name.
Confirmed by patching only the labels
Copied one recording, rewrote only the three descriptor labels in its .hc (Nasion -> nasion, LPA -> left ear, RPA -> right ear), changed no coordinates and no other file, and reconverted through the same Recording.from_file -> to_zarr call the converter uses:
original -> FAILED file_read_error (mandatory HPI device-coordinate info)
labels-only patch -> CONVERTED groups=['events','meg_250hz','misc_600hz'] 60.3 MiB
Scope
380 recordings in on006502, the whole dataset. No other Track 5 dataset is CTF, so nothing else is currently known to be affected, but any CTF dataset written with this naming will hit it.
Options
- Upstream in MNE: make
_kind_dictmatching case-insensitive and addlpa/rpaas aliases. Correct place for it, slowest path, and it would need an MNE release before it reaches us. - In biosigIO: normalize the descriptors when opening a CTF directory, before handing off to MNE. Keeps NEMAR out of it and fixes every biosigIO consumer.
- In the converter: rewrite the
.hcin the per-recording scratch copy before reading.
Option 3 is the fastest and the one I would be most careful about: it means rewriting acquisition metadata before reading it. It is contained (the scratch copy is discarded after conversion, the archived file is never touched) but it sets a precedent for the converter silently correcting source data, which is worth an explicit decision rather than a quiet patch. Options 1 or 2 are cleaner if the wait is acceptable.
Recommend deciding between these before implementing.
Activity
neuromechanist commentedon Aug 22, 2026
Decision: fix this in biosigIO, not upstream in MNE and not by normalizing in the converter.
That settles the three options in the description. Reasoning, for the record:
.hcin the converter's scratch copy would work and is contained, but it means the converter silently rewrites acquisition metadata before reading it. That is a precedent worth not setting for a vendor naming variant.importers/meg.pycallsread_raw_fif/ the CTF reader directly), so normalizing the coil descriptors there fixes every biosigIO consumer at once and keeps NEMAR out of the business of editing source data.Scope for the biosigIO change: MNE's
_kind_dictmatches the literal lowercase stringsnasion,left ear,right ear. Accept theNasion/LPA/RPAspelling this dataset uses, case-insensitively, when opening a CTF.ds. The coordinates are already correct and in both frames; only the labels differ, which is why rewriting the three descriptor lines and nothing else made the byte-identical recording convert.Tracking this issue against that work rather than closing it: it stays open until a biosigIO release carries the fix, the pin here is bumped, and
on006502converts.neuromechanist commentedon Aug 22, 2026
Filed upstream: mne-tools/mne-python#14190. There was no existing MNE issue for this; I searched their tracker for the error text,
_read_hc, and the CTF/HPI phrasings before opening one.This does not change the plan here. We fix it in biosigIO regardless and do not wait on MNE.
What the upstream report adds
Two things I did not have when writing this issue.
A public reproducer that needs no NEMAR access.
on006502is OpenNeurods006502(CC0), and the failure is raised before the.meg4is ever opened, so a single recording's metadata is enough:3.2 MB, then
mne.io.read_raw_ctf(...)raises. Worth keeping for our own regression test, since it means we can cover this without a MEG-sized fixture.CTF's own reader is more permissive than MNE, which is the argument for fixing it upstream rather than normalising on our side.
readHcinsidereadCTFds.m(bundled in FieldTrip atexternal/ctf/readCTFds.m) does not match coil names against a vocabulary at all; it reads the descriptor positionally and stores whatever name is there.MEGCoilLabel = strvcat('nasion','left ear','right ear')is declared at line 528 and never used. The same reader carries an explicit workaround for astadardtypo emitted by some Acq releases, so descriptor variation is clearly expected on the CTF side.Provenance, in case it recurs: NIMH MEG Core Facility, DSQ-2010,
Acq 5.2.1-linux-20060809, each dataset laterRewritten by writeCTFdsper its.hist.writeCTFds'swriteHcwritesds.hc.namesverbatim, so the rewrite preserves the spelling rather than introducing it. Which upstream step producedNasion/LPA/RPAis still unknown. Sampledsub-01,sub-03,sub-15,sub-20: all the same.Correction to my root-cause note above
I described this as a case-sensitivity problem. Case is only half of it.
Nasionmissesnasionon case, butLPAandRPAare a different vocabulary entirely and would missleft ear/right earat any casing. A biosigIO fix that only lowercases will not work.The mapping that does work, verified against both the variant file and a canonically spelled one (
ds000247), is to lowercase the descriptor and add the aliases:With that, the unmodified
.hcparses to[NAS, LPA, RPA, NAS, LPA, RPA]across both coordinate frames and reading proceeds to the data stage; a canonically spelled.hcparses exactly as before.Where the fix goes
Still biosigIO, not here, and not a
.hcrewrite in the converter. If MNE takes the upstream patch, biosigIO's workaround becomes a version-gated no-op rather than something we have to unpick.neuromechanist commentedon Aug 22, 2026
Upstream PR is open: mne-tools/mne-python#14191, fixing mne-tools/mne-python#14190.
Two changes to
mne/io/ctf/hc.py: match the.hcdescriptor line case-insensitively, and addlpa/rpaas aliases forleft ear/right ear. The test copies MNE's CTF testing dataset, rewrites only the three descriptor words, and asserts the resultingdev_head_tand every digitization point match a read of the untouched original; it is parametrized overNasion/LPA/RPAand a mixed-case spelling. Both parametrizations fail onmainwith exactly the error we see onon006502and pass with the change, and the rest oftest_ctf.pyis unaffected (same 5 pre-existing failures before and after, all from testing-data files I did not download locally).We are not waiting on this. The biosigIO fix still goes in; if MNE merges, it becomes a version-gated no-op.
neuromechanist commentedon Sep 3, 2026
Closing. Fixed upstream in biosigio 1.2.7 (pinned in scripts/zarr/requirements.txt); zarr.nemar.org/on006502/zarr/index.json shows no coil-naming failures. on006502's remaining failures are memory-class and are being re-converted by the engine-3 wave under the v0.9.14 memory fixes.