Skip to content

Failure reporters survive a log with no marker - #1366

Merged
neuromechanist merged 1 commit into
devfrom
fix/workflow-reporter-fallback
Sep 10, 2026
Merged

neuromechanist merged 1 commit into
devfrom
fix/workflow-reporter-fallback

Conversation

@neuromechanist

Copy link
Copy Markdown
Contributor

Fixes #1364, found by the release review of main...dev. Live in production since nemarDatasets/.github@91bebcc0 this morning, so this needs the paired copy (below).

What was broken

Actions runs a run: step under bash -e {0}, and these steps add set -o pipefail. So when the tee'd log holds no ✖:

line="$(grep -nF '✖' "$log" 2>/dev/null | tail -n 1 | cut -d: -f1)"

grep exits 1, pipefail promotes it to the pipeline's status, and -e kills the step at that assignment — before the named fallback three lines later. Nothing is posted, and the report job's terminal: prepare=failure copy=failure finalize=failure roll-up is what lands: exactly the pre-change behavior, for exactly the failure modes the fallback was written for (an OOM, an evicted runner, a cancellation, or a failure early enough that the log was never written).

prepare was also the only reporter that never initialized msg, so under set -u it died at its own [ -z "$msg" ] test with msg: unbound variable.

So #1351's stated decision — "An empty extraction becomes a named fallback, never an empty error_message" — did not hold in the deployed shell.

Why the test suite was green

backend/test/import-capture-workflow.test.ts asserted the fallback like this:

expect(workflow()).toContain(fallback)

The string was in the file. The code path was dead. That is the third instance today of a test that exercises text or a callee while the caller is wrong, so the reporters now run: the script is extracted from the YAML and executed under bash -e, cut at the point where msg is final, so nothing needs jq and nothing can reach api.nemar.org. Nine new cases across the three stages: no marker, missing log, a real CLI failure line still winning, and prepare's upstream-inaccessible branch.

The paired deploy

.github/dataset-workflows/onboard-openneuro.yml is deployed by copying it into nemarDatasets/.github, and #1358's parity test enforces byte-identity. So unit-pure is red on this PR until the org copy is updated, which is the correct order: confining the failure here rather than to every other open PR. The org-side copy goes live for every dataset import the moment it merges there.

Test plan

  • bun test backend/test/import-capture-workflow.test.ts: 22 pass (13 before)
  • Mutation-proved: removing || true from the three greps fails 6; removing prepare's msg="" fails 2
  • bun run typecheck clean, Biome clean
  • The parity test will pass once the org copy matches; I will link the org-side commit here

Actions runs a `run:` step under `bash -e {0}`, and these steps add
pipefail, so when the tee'd log holds no `✖` the grep exits 1, pipefail
promotes it to the pipeline's status, and `-e` kills the step at the
assignment -- before the named fallback. Nothing was posted at all, and the
stage roll-up this epic replaced is what landed, for exactly the failures
the fallback exists for: an OOM, an evicted runner, a cancellation, or a
failure early enough that the log was never written. `prepare` also ran
under `set -u` without initialising `msg`, so even without `-e` it died at
its own fallback test with `msg: unbound variable`.

The reporters now run in the test rather than being asserted as text, which
is why this shipped: the fallback assertion was
`expect(workflow()).toContain(fallback)`, and it passed over dead code. The
script is extracted from the file and executed under `bash -e` up to the
point where `msg` is final, so nothing needs jq and nothing can reach
api.nemar.org.

Needs the same file copied into nemarDatasets/.github; the parity test is
red until it is.

Tests: 22 in import-capture-workflow.test.ts. Removing `|| true` fails 6,
removing prepare's initialiser fails 2.
@neuromechanist

Copy link
Copy Markdown
Contributor Author

Paired org-side change is merged: nemarDatasets/.github#116, commit bc599a9e. The deployed copy now matches this branch byte for byte, so the parity check should pass on a re-run.

@neuromechanist
neuromechanist merged commit 9299623 into dev Sep 10, 2026
22 of 23 checks passed
@neuromechanist
neuromechanist deleted the fix/workflow-reporter-fallback branch September 10, 2026 14:51

This branch was previously deployed

1 inactive deployment
npm — 48f52d65 Deployed Sep 10, 2026 by neuromechanist via publish #2494
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant