Skip to content

CLI: view alias, sweep commands, recover fix - #1223

Merged
neuromechanist merged 9 commits into
devfrom
fix/cli-small
Sep 3, 2026
Merged

neuromechanist merged 9 commits into
devfrom
fix/cli-small

Conversation

@neuromechanist

Copy link
Copy Markdown
Contributor

Summary

Five small, independently reviewable CLI-side fixes, one commit each.

nemar dataset view is now an alias for nemar dataset status, registered via Commander's .alias() and mentioned in the command description and the AGENTS.md CLI table.

nemar admin recording-stats-sweep and nemar admin signal-defaults-sweep are new commands mirroring hedSweepCommand's batch-loop/--reset/--verbose/--json shape, calling the existing POST /datasets/recording-stats-sweep and POST /datasets/signal-defaults-sweep backend routes via new client functions in src/lib/api/admin.ts.

Three afterAll hooks restored process.env.X with an unguarded assignment (process.env.X = previous), which coerces undefined into the literal string "undefined" instead of deleting the key; since root bun test runs test/ and backend/test/ in one process, this leaked into later test files. Fixed test/maintenance-client.test.ts and test/curl-stream-copy.unit.test.ts to match test/publish-progress.test.ts's existing guarded pattern, and trimmed test/search-color.unit.test.ts's now-stale workaround comment.

nemar admin recover was broken in every published npm build: scripts/recover-datasets.json was never shipped (missing from package.json's files), and the default path resolution hardcoded a source-checkout-only depth that lands outside the installed package from the single-file dist/index.js bundle. Now ships the file and tries the source-tree path first, falling back to the bundled depth.

uploadDataToS3's git-state reconcile (listTrackedPaths / computeAddTargets) lived entirely inside the s3_upload-incomplete gate, so a data file added to a dataset after a prior run completed was silently never picked up on a resumed upload. The reconcile now runs unconditionally (cheap, no content access) and reopens the s3_upload gate when it finds real work; the expensive upload itself stays gated exactly as before.

Closes #1212
Closes #1194
Closes #1175
Closes #1049
Closes #1070

Test plan

  • bun test test/cli.test.ts -t "Dataset Status" — 6 pass (alias resolves identically to status, including --help text and dataset --help's status|view listing).
  • bun test test/recording-signal-sweeps-cli.test.ts — 19 pass, real subprocess CLI against a local capture server (no mocks); confirmed 16/19 assertions fail when the two new addCommand() registrations are removed.
  • bun test test/maintenance-client.test.ts test/curl-stream-copy.unit.test.ts test/search-color.unit.test.ts — 17 pass; reproduced the TEST_API_URL poisoning with a throwaway probe (checkHealth() failed with "Could not connect to undefined" when the guard was reverted) and confirmed the guarded restore fixes it.
  • bun test test/recover-datasets-packaging.test.ts — 5 pass; asserts the real bun pm pack tarball includes the file, and runs the real nemar admin recover --all --json from both bun run src/index.ts and a real bun build dist/index.js bundle placed at the npm-installed layout. Confirmed 3/5 fail when the fix is reverted.
  • bun test test/upload-s3-resume.unit.test.ts — 2 pass, real git-annex repo on disk; confirmed the new assertions fail (result stays "ok", new file stays untracked) when the fix is reverted. Sibling suites (annex-add-targets, upload-progress, upload-steps, recover-datasets) still pass: 159 tests total across all touched + sibling files.
  • bun run lint — clean (biome check src/).
  • bun run typecheck — clean (tsc --noEmit for both root and backend/).
  • bun run build (the project's real build script, --target bun) — succeeds; verified nemar admin recover works from the built bundle.
  • bun test test/cli.test.ts has 13 pre-existing failures unrelated to this PR (auth/admin/upload/collaborator tests requiring a live backend + TEST_ADMIN_API_KEY/TEST_USER_API_KEY from test/.env.test, which is not present in this environment); none of them touch code this PR changes.

nemar dataset view now resolves to the same status command.
Description text and AGENTS.md CLI table mention the alias.
Tested: bun test test/cli.test.ts -t 'Dataset Status' (6 pass); confirmed the 3 new assertions fail when .alias("view") is reverted.
nemar admin recording-stats-sweep and nemar admin signal-defaults-sweep
call the existing backend admin routes, mirroring hedSweepCommand's
batch-loop/--reset/--verbose/--json shape. Client calls added to
src/lib/api/admin.ts; both commands added to the AGENTS.md CLI table.
Tested: bun test test/recording-signal-sweeps-cli.test.ts (19 pass),
real subprocess CLI against a local capture server (no mocks); confirmed
16/19 assertions fail when the two addCommand() registrations are
removed.
process.env.X = previous poisons X to the literal string "undefined"
when it was never set, instead of deleting it; root bun test runs test/
and backend/test/ in one process, so this leaked into later files.
Fixed the unguarded restores in test/maintenance-client.test.ts and
test/curl-stream-copy.unit.test.ts to match test/publish-progress.test.ts's
guarded pattern, and trimmed search-color's now-stale workaround comment
(the TEST_API_URL override there is still needed, just no longer as a
bug workaround).
Tested: bun test on all three files (17 pass); reproduced the poisoning
with a throwaway probe (checkHealth() failed with "Could not connect to
undefined" when the guard was reverted) and confirmed the guarded
restore fixes it.
scripts/recover-datasets.json was never shipped (package.json files
omitted it) and defaultRecoverDatasetsPath() hardcoded a source-tree-only
depth (two levels up from src/commands/), which lands outside the
installed package entirely from the single-file dist/index.js bundle.
Now ships the file via files and tries the source-tree path first,
falling back to the bundled (one level up) path.
Tested: bun test test/recover-datasets-packaging.test.ts (5 pass) --
asserts the real bun pm pack tarball includes the file, and runs the
real nemar admin recover --all --json from both bun run src/index.ts
and a real bun build dist/index.js bundle placed at the npm-installed
layout. Confirmed 3/5 assertions fail when the fix is reverted (the
packed-file-list check and the dist-bundle run; the source-mode run
was never broken, as expected).
The git-state reconcile (listTrackedPaths / computeAddTargets) lived
entirely inside the s3_upload-incomplete gate, so once a prior run
stamped s3_upload complete, a data file added afterward was silently
never picked up again on a resumed upload.
Moved the cheap reconcile above the gate so it always runs; when it
finds real work despite the stale completion stamp, it clears
s3_upload so the existing gate naturally reopens. The expensive upload
itself stays gated exactly as before.
Tested: bun test test/upload-s3-resume.unit.test.ts (2 pass) -- drives
the real uploadDataToS3 entry point against a real git-annex repo on
disk; confirms a new file gets git-annex tracked and the stale
s3_upload stamp clears even though a real local credentials server
always errors (proving the reconcile path, not a full S3 round-trip,
which needs live AWS creds and belongs to the e2e suite). Confirmed the
new assertions fail (result stays "ok", file stays untracked) when the
fix is reverted. Sibling suites (annex-add-targets, upload-progress,
upload-steps) still pass: 97 tests.
Critical review finding on PR #1223: reopening s3_upload for new
content (issue #1070's fix) left dataset_save/github_push stamped
complete from the prior run, so a resumed upload could re-upload a new
file to S3 but never commit or push its git-annex pointer -- and
printUploadSuccess would still clear progress and report success.

clearStepCompleted now also clears dataset_save/github_push whenever
the reconcile reopens s3_upload, and the two/three writeUploadProgress
calls collapse into one combined write (review suggestion 4, same fix).

Tested: bun test test/upload-s3-resume.unit.test.ts (3 pass) -- the new
test pre-stamps all four steps complete, adds a file, drives the real
uploadDataToS3 then the real saveDatasetStep, and asserts dataset_save/
github_push are reopened and the new file actually gets committed.
Confirmed it fails (dataset_save stays true) when the fix is reverted.
Important review finding on PR #1223: the same packaging bug the
recover fix addressed for scripts/recover-datasets.json also applies
to scripts/withdrawn-datasets.json (defaultWithdrawnDatasetsPath) --
confirmed missing from the tarball with bun pm pack --dry-run.

Same fix as recover: added to package.json's files, and
defaultWithdrawnDatasetsPath tries the source-tree depth first,
falling back to the bundled depth.

Tested: bun test test/recover-datasets-packaging.test.ts (10 pass,
extends the existing recover coverage) -- asserts the real bun pm pack
tarball includes the file, and runs the real nemar admin withdraw
--all --json (via a local stub answering the withdraw endpoint, since
even withdraw's dry run calls the backend once per target) from both
bun run src/index.ts and a real bun build dist/index.js bundle.
Confirmed 3/10 fail when the fix is reverted.
Suggestion 5 on PR #1223 review: hed-sweep, recording-stats-sweep, and
signal-defaults-sweep duplicated the same do-while batch loop, progress
guard, GitHub-pacing sleep, and try/catch three ways. runSweepBatchLoop
now owns that shared control flow; each command's totals accumulation
and verbose/summary text stay inline via a callback, so no command's
actual output changes.

Also closes a pre-existing gap: hed-sweep had no CLI-level test
asserting its exact output before this. test/hed-sweep-cli.test.ts adds
that coverage and doubles as the regression guard for the extraction.

Tested: bun test test/hed-sweep-cli.test.ts (10 pass) and
test/recording-signal-sweeps-cli.test.ts (19 pass, unchanged assertions
on exact stdout still hold). Verified byte-identical --help output for
all three commands before/after, and confirmed
test/hed-sweep-cli.test.ts passes identically against the pre-extraction
(triplicated) code, proving the refactor is behavior-preserving rather
than merely matching new code.

Found and deliberately left alone (out of scope, pre-existing, filed as
a follow-up candidate): the root CLI's global --verbose option
(src/index.ts:54) shadows every subcommand's own local --verbose
(including these three, plus admin.ts:3372 and :4750), so
options.verbose is always undefined in these action handlers and
--verbose is currently a silent no-op on all of them.
@github-actions

github-actions Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

📦 Package Preview

Your package is ready to test!

bun install -g nemar-cli@PR1223

Version: 0.9.15-PR1223.3bcc20f
Commit: 3bcc20f


🤖 Updated on each push.

unit-pure failed on #1223 because test/api-export-surface.unit.test.ts pins lib/api/admin's runtime exports; the two sweep commands added recordingStatsSweep, recordingStatsSweepReset, signalDefaultsSweep and signalDefaultsSweepReset. Added to the admin pin and to POST_SPLIT_ADDITIONS with their issue reference. Tested: bun test test/api-export-surface.unit.test.ts (10 pass).

This branch was previously deployed

1 inactive deployment
npm — 3bcc20fd Deployed Sep 3, 2026 by neuromechanist via publish #2252
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant