Skip to content

fix(eyetracking): read eyelink calibration files as UTF-8 (#14000) - #14002

Merged
larsoner merged 3 commits into
mne-tools:mainfrom
CedricConday:fix/eyelink-calibration-utf8
Jun 30, 2026
Merged

fix(eyetracking): read eyelink calibration files as UTF-8 (#14000)#14002
larsoner merged 3 commits into
mne-tools:mainfrom
CedricConday:fix/eyelink-calibration-utf8

Conversation

@CedricConday

Copy link
Copy Markdown
Contributor

Reference

Closes #14000

What was wrong

mne.preprocessing.eyetracking.read_eyelink_calibration() decoded the .asc file as ASCII:

lines = fname.read_text(encoding="ASCII").splitlines()

When a MSG line contained a non-ASCII character (e.g. a zero-width no-break space, U+FEFF, which encodes to the 0xef byte from the traceback), this raised UnicodeDecodeError and the whole read failed. As the reporter notes, MSG lines are not guaranteed to be ASCII — users may put UTF-8 text in their messages — and mne.io.read_raw_eyelink() already reads the same files without issue.

Fix

Decode as UTF-8 instead. ASCII is a subset of UTF-8, so existing all-ASCII files are unaffected; this only widens what can be read. This matches the maintainer's suggestion in the issue thread ("I think we should decode as UTF8. If later we need to add a param to control this we can").

Test

Adds test_calibration_non_ascii, mirroring the existing test_calibration_newlines: it injects a MSG line containing non-ASCII content (including U+FEFF, the exact character that triggered the original crash) into the testing .asc, then asserts the calibration reads without error and the parsed values are unchanged.

…14000)

read_eyelink_calibration() decoded .asc files as ASCII, which raised a
UnicodeDecodeError when a "MSG" line contained non-ASCII characters
(e.g. a zero-width no-break space, U+FEFF). MSG lines are not guaranteed
to be ASCII since users may include UTF-8 text in their messages, and
read_raw_eyelink() already reads these files without issue.

Decode as UTF-8 instead (ASCII is a subset, so existing files are
unaffected). Adds a regression test that injects non-ASCII content into a
MSG line, plus a changelog entry.
@larsoner

Copy link
Copy Markdown
Member

Pre-commit CI failed

codespell............................................................................Failed
- hook id: codespell
- exit code: 65

mne/preprocessing/eyetracking/tests/test_calibration.py:302: caf ==> calf

can you either modify the line or add caf to ignore_words.txt?

@CedricConday

Copy link
Copy Markdown
Contributor Author

Thanks! Fixed in 2565238 — reworded the test message to drop the caf token (kept the U+FEFF character that triggers the original crash). pre-commit is green now.

@larsoner larsoner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marking for merge when green, thanks in advance @CedricConday !

@larsoner
larsoner merged commit 90b8cee into mne-tools:main Jun 30, 2026
32 checks passed
@welcome

welcome Bot commented Jun 30, 2026

Copy link
Copy Markdown

🎉 Congrats on merging your first pull request! 🥳 Looking forward to seeing more from you in the future! 💪

CedricConday added a commit to CedricConday/mne-python that referenced this pull request Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reading UTF characters in read_eyelink_calibration()

2 participants