Repository navigation
Description
Goal
The nemar-python runtime image exists as code and as a CI product: Python 3.12 with eegprep, MNE-Python, NumPy, SciPy, pamica (CPU) and the signalJourney validator, built from a committed lock file, pushed to GHCR, turned into a SIF from its digest, signed with cosign keyless, and smoke-tested both as a container image and as a SIF under Apptainer. Every consumer refers to it by digest.
Routing
- Readiness: ready
- Implementer: Sonnet
- Opus checkpoint: none
- Depends on: Phase C1: Repository scaffold and the nemar-launch launcher #2 (repository, CI skeleton, Apptainer step) ; Unblocks: Phase C3: nemar-floor recipe #4, Phase C4: nemar-ica recipe #5, Phase C5: Tapis definitions and the observer #6
- Repository and base branch: nemarOrg/nemar-process, epic branch
feature/issue-#1-epic-process
Scope
In: images/nemar-python/ (CPU flavor only), the dependency declaration and lock, the build and signing workflow, the digest record, the smoke test. Out: the CUDA flavor, the Fortran amica15 binary and the ICLabel weights check (#5); recipes; registering anything with Tapis.
Design references
Design page section 7 (runtimes); first-party design v1 F.2 (base, pinning, GHCR, cosign, SIF via oras://) and v2 B.6 (two flavors from one lock).
Changes
- Root
pyproject.toml: dependency groupruntime(and extracpu) pinning exact versions:eegprep[mne,pybids,picard,torch]==0.5.0(do not use theeeglabextra: it installs an Octave bridge, which the no-MATLAB rule forbids),mne==1.13.2,pamica==0.4.0,mne-icalabelonly if Phase C4: nemar-ica recipe #5 needs it, NumPy and SciPy as resolved, torch from the CPU index via[tool.uv.sources]withextra = "cpu". One rootuv.lockserves image, recipes and tests. Versions were current on PyPI on 2026-11-03; re-check. - signalJourney validator: it is not on PyPI and resolves its schema relative to its source tree, so a plain install cannot find the schema. Add it as
signaljourney-validator @ git+https://github.com/signalJourney/signalJourney@<commit>(commit169c3261d044on 2026-11-03, or newer), and vendor that commit'sschema/directory into the Python packagenemar_process/_vendor/signalJourney/schema/with aSOURCEfile naming the commit. Providenemar_process.sj.validator()returningValidator(schema_dir=<vendored dir>). File an upstream issue onsignalJourney/signalJourneyto publish the validator to PyPI with the schema packaged; do not edit that repository from here. images/nemar-python/Dockerfile:python:3.12-slimpinned by digest,uvcopied from a version-pinned image,uv sync --frozen --no-dev --group runtime --extra cpuinto/opt/venv,OPENBLAS_NUM_THREADS=1,OMP_NUM_THREADS=1,PYTHONHASHSEED=0and a non-root user, labels with source commit and lock hash. Phase C3: nemar-floor recipe #4 installs the recipes package; this phase installs a placeholder.images/nemar-python/smoke.py: imports every pinned package and prints versions as JSON; runs one tiny eegprep step (load the smoke fixture,pop_eegfiltnew,clean_flatlines) and validates a hand-written signalJourney record withnemar_process.sj.validator().tests/fixtures/smoke/plusscripts/make_smoke_fixture.py: a deterministic synthetic 8-channel, 20-second.set(seeded NumPy), documented as synthetic and as a smoke input only; scientific fixtures are Phase C3: nemar-floor recipe #4's..github/workflows/image-nemar-python.yml: on pull request build without pushing; on tagimage-nemar-python-v*or manual dispatch:docker/build-push-actiontoghcr.io/nemarorg/nemar-python:<ver>-cpu, capture the digest output,cosign signkeyless onghcr.io/nemarorg/nemar-python@sha256:<digest>(id-token: write),apptainer buildthe SIF from that digest reference,apptainer pushit tooras://ghcr.io/nemarorg/nemar-python-sif:sha256-<hex>, cosign-sign the SIF artifact digest, run the smoke test in both forms, and attachnemar-python-<ver>-cpu-digests.json(image digest, SIF digest, lock sha256, eegprep, pamica, torch versions, run URL) to the release.docs/images.md: how consumers reference a digest, how to verify (cosign verify --certificate-identity-regexp ... --certificate-oidc-issuer https://token.actions.githubusercontent.com), and that SIF compatibility with the cluster's Apptainer variant is unverified until the operator tests it.
Tests (write these first)
uv lock --checkpasses and the lock contains no package named in ano-matlabdenylist (oct2py,matlab,matlabengine). Mutate: add theeeglabextra.smoke.pyin the lock's venv prints versions equal to the pins. Mutate: change one pin.- The vendored schema validates a signalJourney example (copied to
tests/fixtures/sj/) and rejects a record missingpipelineInfo. Mutate: emptyschema_dir. - CI job: build the image on the pull request, run
smoke.pyinside it, then build a SIF from the local image and run it under Apptainer with--containall --cleanenv --no-home. Mutate: removescipyfrom the lock; the job must fail. actionlintpasses and the Dockerfile base is a digest. Mutate: use a tag.
Acceptance criteria
- The pull-request workflow builds and smoke-tests the image and the SIF.
- A manual dispatch pushes, signs, builds the SIF from the digest, signs it, and attaches the digest file;
cosign verifysucceeds in the same run. - No MATLAB, Octave or
eeglabextra in the image or the lock. - The upstream signalJourney packaging issue is filed and linked.
Gates
uv lock --check && uv run ruff check . && uv run ruff format --check . && uv run ty check && uv run pytest tests/images --cov and, on a machine with Docker and Apptainer, docker build -f images/nemar-python/Dockerfile . then docker run --rm <image> python /opt/smoke.py.
Rules that bite here
- Digest everywhere: a tag moves, a digest does not; the Dockerfile base, the uv image and every consumer reference are digests.
- No MATLAB path of any kind, including a transitive one: check the lock.
- GHCR names are lowercase (
nemarorg), and signing needsid-token: writein the job only.
Escalate to the owner if
- GHCR publishing is not enabled for the repository.
- The resolver cannot satisfy pamica's torch floor with the CPU index (record the exact error).
When merged
Record the first digest in docs/images.md; tell #6 and Epic A the image reference format; add the signalJourney and eegprep upstream issue links to docs/upstream.md.
Activity
neuromechanist commentedon Nov 4, 2026
Driver amendment, 2026-11-04, from the review and Opus checkpoint of the launcher (#7). This comment is part of the phase brief; where it differs from the body above, this comment wins. The launcher-side facts it relies on are in #7 and in the nemar-cli follow-up phase A1b (launch-specification and launcher-detail contract).
The SIF this phase builds is named by the image digest:
<64 lowercase hex characters of the OCI digest>.sif, placed under the directory the Tapis application sets asNEMAR_SIF_ROOT. The launcher refuses a specification whoseimage.sifis not that name forimage.digest.neuromechanist commentedon Nov 4, 2026
Merged into the epic branch by #8 (squash). Rounds 1 and 2 converged; CI green on the final head (container and SIF smoke test, images, both launcher jobs). The publish, sign and release chain has NOT been run: it runs only from a tag or a dispatch on a commit of main, after the epic merges. Owner items before the first publish are in docs/images.md (GHCR publishing enabled, both packages linked to the repository with write access, a tag ruleset on image-nemar-python-v* and protection on main or an environment with reviewers). Upstream issues: signalJourney/signalJourney#6 and sccn/eegprep#533.