Skip to content

mne is a base dependency, but eegprep is its peer rather than its consumer #395

Closed

Description

@neuromechanist

Follows epic #324. The base install pulls mne, and mne is used only as an I/O and interchange bridge: eegprep implements the same ground, so a user who never crosses to MNE still downloads it.

Where mne is actually used

Five modules on epic/324-pyodide-browser import it, all bridges:

  • src/eegprep/functions/popfunc/_file_io.py
  • src/eegprep/functions/popfunc/pop_fileio.py
  • src/eegprep/functions/popfunc/pop_writeeeg.py
  • src/eegprep/functions/miscfunc/eeg_eeg2mne.py
  • src/eegprep/functions/miscfunc/eeg_mne2eeg.py

_file_io.py is the one that matters. It defines 18 top-level functions and exactly two of them touch mne (mne_raw_to_eeg, eeg_to_mne_raw), while the other sixteen are format-agnostic helpers: write_json, events_to_records, records_to_events, channel_labels, eeg_from_data, infer_dataformat, load_data_array, normalize_icachansind, read_table_records, and so on. Those helpers are imported by roughly ten modules across popfunc/ and plugins/ (firfilt, EEG_BIDS, pop_importdata, pop_loadbci, pop_chanevent, pop_importevent, pop_expevents, ...). The module's top-level import mne at line 11 therefore makes mne a hard requirement of nearly every path, in service of two functions nobody on those paths calls.

What it costs in a browser

Resolved against the Pyodide 0.29.5 distribution lock plus PyPI metadata, with every wheel measured (Content-Range on a one-byte range request, since the CDN answers HEAD without a length):

base install packages download
as declared on the epic branch 61 62.6 MB
without mne 48 54.1 MB
without mne and pybids 26 42.1 MB
without mne, pybids, matplotlib, sympy 14 27.7 MB

mne's marginal cost inside the base is 8.5 MB and 13 packages rather than its standalone 37.6 MB, because scipy and matplotlib are shared with the rest.

Dropping pybids is worth more than its bytes. It is the only reason docopt is in the closure (pybids -> num2words -> docopt), and docopt publishes an sdist and no wheel at any version, which is why tools/check_pyodide_base_resolution.py carries LOCAL_WHEEL_PACKAGES = {"docopt"}. Without pybids in the base, the closure has no package that needs a locally built wheel at all, and the live harness gets simpler rather than merely smaller.

Note on what laziness can and cannot buy

eegprep/__init__.py already resolves its public names through _LAZY_EXPORTS and importlib, so import eegprep is cheap. That saves startup time, not download bytes: micropip installs the declared dependency closure whether or not a lazy import is ever reached. Bytes only move when a dependency stops being declared as required. So the lazy imports are the precondition and the extras are the payoff, in that order.

Suggestion

  • Import mne inside the two functions in _file_io.py that need it, and inside the bridge modules, the way the epic already handles oct2py and psutil.
  • Move mne to an extra (eegprep[mne], or fold it into an I/O extra) and do the same assessment for pybids, which is a BIDS layout concern rather than a preprocessing one.
  • Extend the existing gate: install the base with no extras, import the public surface, and assert "mne" not in sys.modules. A resolution check alone will not notice a lazy import that was quietly made eager again.

Measured in the context of the browser execution epic, where the same runtime wants to read a window and plot it before it ever preprocesses: the reader path alone (zarr + numpy) is 12 packages and 4.2 MB. Related: OpenScience-Collective/osa#375, nemarOrg/nemar-cli#1457.

Activity

  1. neuromechanist commented on Sep 20, 2026

    @neuromechanist
    MemberAuthor

    Every base dependency, by what it is actually for

    Counting import sites under src/eegprep on epic/324-pyodide-browser, separating module-level imports from ones inside a function:

    dependency top-level lazy what it serves
    numpy 272 0 the arrays. Not going anywhere.
    scipy 52 3 see below, it is not one thing
    matplotlib 28 0 plotting
    mne 5 0 I/O and interchange bridges only
    h5py 3 0 MATLAB v7.3 .set reading, HDF5 export
    pyyaml 3 0 CLI batch and migrate
    colorama 1 0 log coloring
    packaging 1 0 extension version checks
    python-picard 1 0 ICA, in eeg_picard.py. Brings scikit-learn, 5.6 MB.
    pybids 1 2 plugins/EEG_BIDS only
    sympy 1 0 one call, see #396
    threadpoolctl 1 1 utils/math_backend.py
    neo 0 1 one lazy import in plugins/EEG_BIDS/raw.py
    eeglabio 0 0 nothing. Declared at pyproject.toml:27 and imported nowhere in src/.
    scikit-learn 0 0 not imported directly; arrives through python-picard

    Two of those are free to remove. eeglabio is declared and unused. sympy is 4.1 MB for a single sp.nsimplify() at pop_resample.py:209, which #396 shows is also giving wrong answers for non-integer sampling rates where fractions.Fraction is exact. The lightest option and the correct one are the same one.

    On scipy, which is the interesting case

    The instinct that the whole package is not needed is right about the usage and blocked by the packaging. Usage by submodule:

    scipy.io 36 references, scipy.signal 27, scipy.linalg 13, scipy.special 6, scipy.optimize 5, scipy.interpolate 3, scipy.stats 1, scipy.fft 1.

    The largest single consumer is scipy.io, across 13 modules, and all of it is MATLAB .mat traffic (loadmat, savemat, whosmat) in service of EEGLAB .set compatibility. A browser session that reads its data from Zarr never touches any of it.

    That does not shrink the install, though, because Pyodide ships scipy as one 16.4 MB wheel; there is no way to install scipy.signal alone. So scipy is not a trimming problem like mne or pybids, it is a timing problem: the only lever is not loading it until preprocessing actually starts.

    What that buys, measured

    Against the Pyodide 0.29.5 distribution, closure resolved from its lock plus PyPI, every wheel measured:

    tier packages download
    read a window (zarr, numpy) 12 4.2 MB
    read and plot (+ matplotlib) 22 14.0 MB
    preprocess (+ scipy) 23 30.4 MB
    ICA (+ python-picard, which brings scikit-learn) 27 36.3 MB
    eegprep's base install as declared today 61 62.6 MB

    So a browser-lean eegprep lands near 36 MB against 62.6 MB, and more importantly the first thing a person does, look at their data, costs 14 MB instead of 62.6 MB. The rest arrives when it is first needed, which is what Pyodide's loadPackagesFromImports and micropip are for.

    Suggested split

    pybids is worth singling out again: it is the only reason docopt is in the closure, and docopt publishes no wheel at any version, which is why the Pyodide gate carries LOCAL_WHEEL_PACKAGES = {"docopt"}. Out of the base, nothing in the closure needs a hand-built wheel at all.

    Whether matplotlib belongs in base or in an extra is the one real judgment call here, since some plotting is wanted early. It is 6.7 MB and 10 packages, and at 28 module-level imports it is the one that would take the most work to make lazy.

  2. neuromechanist commented on Sep 20, 2026

    @neuromechanist
    MemberAuthor

    Correction: eeglabio is used, and my audit was wrong about it

    The table above lists eeglabio as "declared at pyproject.toml:27 and imported nowhere". That is true of the string eeglabio in src/, and it is the wrong test. Removing it fails 12 tests on every Python and platform:

    FAILED tests/test_eeg_mne2eeg.py::TestEEGMNE2EEG::test_eeg_mne2eeg_raw_object
      RuntimeError: For exporting to EEGLab to work, the module eeglabio is needed,
      but it could not be imported.
    

    eeg_mne2eeg exports through mne, and mne imports eeglabio when asked to write an EEGLAB file. So the dependency is real, it is simply consumed through another library's optional-extra mechanism rather than by an import in this repository. Grepping for import sites cannot see that, and nothing in the audit method above would have caught it.

    Two things follow.

    For the dependency plan: eeglabio belongs wherever mne ends up. It is only reachable through the mne bridge, so if mne moves to an extra, eeglabio moves with it and neither is in the base install; if mne stays, so does eeglabio. It should not be listed separately.

    For the rest of the table: the counts for mne, pybids, h5py, neo, sympy, matplotlib and the others are import-site counts and stand on their own, because each of those IS imported by name in src/. But the same blind spot applies in the other direction: a package with import sites may also be required by a second path the count does not show, and a package with none may still be required, as this one was. Before anything is actually removed, the test suite has to run in an environment that genuinely lacks it. That is the check that caught this, and it is cheap.

    #397 now removes sympy only, which is verified the right way: the suite runs with sympy absent from the environment rather than merely absent from pyproject.toml.

  3. neuromechanist commented on Sep 20, 2026

    @neuromechanist
    MemberAuthor

    Decision: eegprep keeps mne, and the browser runtime becomes a separate package

    Recording the outcome so this issue stops reading like an open question.

    eegprep stays whole. Its default install keeps every dependency it has today, pip install eegprep is unchanged, and no existing user gets less than they had. The browser runtime is built as a separate package to a download budget instead, and the seam between the two is owned and declared: where both implement the same operation the contract says so and says how they differ, and where the lean path cannot do something it says that too. Divergence is allowed, undeclared divergence is not.

    What changed the answer was the eeglabio correction above. An import-site audit is not evidence that a dependency is removable, because a package can be reached through another library's optional-extra mechanism where no grep of this repository can see it. Every remaining candidate here, mne, pybids, h5py and neo, would need the same proof run against an environment that genuinely lacks it, and the payoff for that work is a smaller default install, which is a change to what existing users receive rather than a pure gain.

    So the measurements in this thread keep their value, as the budget the separate package is built to, and stop being a plan to shrink this one.

    What stands from this issue:

    • sympy removed, in Fix the resampling ratio for non-integer sampling rates, and drop sympy #397, the one dependency with genuinely no caller, along with the resampling defect that finding exposed.
    • phase/374-extras-split, already in epic/324-pyodide-browser, which moved oct2py, psutil and pyedflib out of the base install. That work is good and this decision does not reverse it; it marks where the line stops.
    • mne, pybids, h5py, neo: staying. Not a deferral.

    The full reasoning, the alternatives and the honest cost of the seam are in nemarOrg/nemar-cli ADR 0069 (nemarOrg/nemar-cli#1458).

    Closing. Reopen if the browser package turns out to need something from here that the contract cannot express.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions