Skip to content

Phase C4: nemar-ica recipe #5

Open

Description

@neuromechanist

Goal

The nemar-ica recipe exists: the floor, then ASR, an average reference, a 2,000-iteration pamica decomposition rank-capped at 64 (GPU float64 in production, the Fortran amica15 binary on CPU as the fallback) and ICLabel. The ICA weights, sphere and ASR state are saved as files beside the signalJourney record, which names them with checksums. A CUDA image flavor exists for V100 nodes. CI runs a tiny CPU-only decomposition; GPU validation is a later manual Expanse step.

Routing

  • Readiness: expand (see Decide in plan)
  • Implementer: Sonnet
  • Opus checkpoint: review before merge (the eegprep pamica backend PR, determinism per device, what the record claims)
  • Depends on: Phase C3: nemar-floor recipe #4, Phase C2: nemar-python runtime image #3 ; Unblocks: Epic A's pipeline registry stamps (recipe nemar-ica)
  • Repository and base branch: nemarOrg/nemar-process, epic branch feature/issue-#1-epic-process

Scope

In: recipes/nemar_process/nemar_ica/, a CUDA image flavor, the CPU fallback, determinism tests, upstream issues. Out: running on Expanse GPUs (manual later step), fit-window caps (the design found them unsafe; the recipe fits the full recording), other modalities, publication.

Design references

First-party design v2 B.5 (rank cap, average reference, what is stored), B.6a items 1, 4, 5, 6 (2,000 iterations, thread pinning, did_converge from stop_reason_, no LLt), B.6c (block 131,072, float64, V100 sm_70, torch 2.8.0 cu128), B.6d (the published Linux binary fails on glibc 2.28 until sccn/pAMICA PR #371, merged to dev 2026-10-13, is released; pin the binary by digest); C.8, C.10; v1 B.4.

Changes

  • Steps after the floor (Phase C3: nemar-floor recipe #4 step table): ASR (eegprep.plugins.clean_rawdata.asr_calibrate then asr_process, state {M, T, B, A, sos} plus sampling rate and channel order saved; eegprep.clean_asr discards the state, an upstream gap); average reference (eegprep.pop_reref(EEG, []), good-channel set recorded); ICA via eegprep.pop_runica(EEG, icatype='pamica', ...), absent from eegprep 0.5.0, so this phase adds it upstream (next bullet); ICLabel (eegprep.pop_iclabel).
  • pamica: n_models=1, n_mix=3, max_iter=2000, fixed seed, block_size=131072, dtype=float64, rank min(N_good - 1, 64) (pcakeep), threads pinned and recorded. Record stop_reason, iterations, LL_final, LL_best, best_iter, device, and did_converge derived from stop_reason_, never converged_.
  • Upstream first, no shim: this phase opens a pull request on sccn/eegprep that adds icatype='pamica' to pop_runica (calling pamica.AMICA(...).fit(...) or AMICANative, mapping onto EEG.icaweights, icasphere, icawinv) and the ASR state export that clean_asr discards, with eegprep's own tests. The owner maintains eegprep and reviews it. The recipe pins the eegprep release that contains it, or that commit until a release ships. Nothing numerical is written in this repository.
  • Files per recording (design C.8, C.10 names, desc-nemarica): unmixing.tsv, mixing.tsv, pca.tsv, asr.json, asrmix.tsv, ica.json (engine, versions, device, dtype, block, seed, threads, rank, reference, good-channel set, convergence), density.json, components.tsv (variance, ICLabel probabilities). Default format: text with %.17g, so float64 round-trips. LLt is never stored. Each file's sha256, size, shape, dtype and labels go in extensions.nemar.outputs.
  • CPU fallback: AMICANative with amica15 at /opt/amica/amica15, installed from a digest-checked asset built for glibc 2.28 or lower (PR #371 CI or a later release); 16 or more threads; the record names engine amica15 and its digest.
  • CUDA flavor: images/nemar-python/Dockerfile.cuda with extra cuda, torch==2.8.0 from the cu128 index (sm_70) and pamica installed --no-deps (its metadata wants a newer torch); tag <ver>-cuda; the Phase C2: nemar-python runtime image #3 workflow builds, signs and records it; CI has no GPU, so it builds the image and runs import torch, pamica.
  • Tiny CI run: the Phase C3: nemar-floor recipe #4 fixture cut to 8 channels and 60 seconds, rank 6, 50 iterations, CPU float64; it asserts records and files, not quality.

Decide in plan

Weight file format (default above; whether a .npy twin is worth it); whether ICA fits the ASR-processed copy (default yes, the stored ASR state lets a consumer reproduce it) or the floor copy; the pop_runica keywords for pamica; the pamica release the Fortran binary comes from.

Tests (write these first)

  1. Same seed, threads and device: weights, sphere and ica.json byte-identical across two runs. Mutate: omit the seed.
  2. Weight files load back to the arrays in memory exactly, and their sha256 in the record matches. Mutate: change %.17g to %.9g.
  3. did_converge is false for a 5-iteration fit that stops on max_iter even when pamica's converged_ is true. Mutate: read converged_.
  4. Rank is min(N_good - 1, 64) and the good-channel set is recorded. Mutate: skip the average reference.
  5. Records validate; the Phase C3: nemar-floor recipe #4 guard test (no ICA, ASR or filter code under recipes/) still passes.
  6. CPU fallback: the record names amica15 and its digest; with the binary absent the run fails with a typed error, never a silent switch. Mutate: remove the check.
  7. The CUDA image builds and imports torch and pamica.

Acceptance criteria

  • The plan decides the items above and files upstream issues (pamica backend switch, ASR state export, LLt=False) on sccn/eegprep and sccn/pAMICA.
  • Tests pass; both image flavors build, are signed, digests recorded.
  • docs/operator.md lists the manual GPU validation steps, with no host or account details.

Gates

uv run ruff check . && uv run ruff format --check . && uv run ty check && uv run pytest tests/recipes tests/images --cov=recipes --cov-fail-under=85

Rules that bite here

  • No MATLAB, and no numerics in this repository: a missing step is added to eegprep upstream (owner ruling, 2026-11-03).
  • Every pinned parameter, including block size, dtype, seed, threads and device, is in the record.
  • Determinism is per device, dtype and thread count; across devices it is tolerance (matched components at least 0.999).
  • Digest-pin the Fortran binary: pamica's resolver never rechecks a cached latest.

Escalate to the owner if

  • The eegprep pull request is not merged within a week of being opened, or its review asks for a design change.
  • No glibc-2.28-safe amica15 build is obtainable by digest.

When merged

Update the first-party design status; give Epic A the recipe id, version and engine string for its stamps.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions