Skip to content

Surface dataset withdrawal in the CLI - #1134

Merged
neuromechanist merged 1 commit into
devfrom
feature/issue-1048-surface-withdrawal
Aug 23, 2026
Merged

neuromechanist merged 1 commit into
devfrom
feature/issue-1048-surface-withdrawal

Conversation

@neuromechanist

Copy link
Copy Markdown
Contributor

Summary

nemar dataset status on008014 printed Status: active, a live DOI link, and a download hint, with nothing saying the dataset had been withdrawn. It also printed Data: incomplete (some files failed to upload; see \nemar admin data-integrity-sweep`)` -- pointing an operator at an integrity sweep for content that was removed on purpose.

That combination is not a cosmetic gap: it is what made a withdrawal cohort look like a data-loss incident and sent #1048 off after the wrong defect. Every surface described these datasets as broken rather than as gone.

Closes #1048.

Why status could not just say "withdrawn"

status is the lifecycle column (active/archived/deleted) and withdrawal is a separate axis recorded in withdrawn_at/withdrawn_reason (migration 0060). A withdrawn dataset is legitimately still active. So the fix is to print the second axis, not to overload the first.

No backend change was needed for the dataset half: GET /datasets/:id selects d.*, so both columns were already in the response and simply never rendered. Visibility rules are untouched -- an anonymous caller still gets 404 for these, as before.

What changed

nemar dataset status, when withdrawn_at is set:

  • a Withdrawn: line directly under Status:, with the reason, because that is the line a reader takes their answer from
  • the DOI is dimmed and marked (tombstoned), since it resolves to a registrar tombstone rather than the dataset
  • Data: reads removed as part of the withdrawal instead of sending the reader to the integrity sweep
  • the download hint is replaced with a line saying the content is gone

nemar admin import status gains a withdrawn N line. GET /admin/imports now returns that count, computed by joining datasets.withdrawn_at. It is reported alongside by_status, not inside it: a withdrawn dataset's import genuinely completed and its import_jobs row is correctly complete, so moving the count would misreport the import lifecycle. The CLI line says so explicitly (included in complete). The field is optional client-side, so an older backend just omits the line.

Test plan

Five real-D1 tests (backend/test/imports-withdrawn-count.test.ts, no mocks): only withdrawn_at rows count; withdrawn rows stay inside by_status.complete; zero when nothing is withdrawn; the count stays fleet-wide under a ?status= filter; and a withdrawn dataset with no import job is excluded.

Verified end to end against the live API rather than only in tests:

$ nemar dataset status on008014
  Status:      active
  Withdrawn:   7/21/2026 - upstream_403
  DOI:         https://doi.org/10.82901/nemar.on008014 (tombstoned)
  Data:        removed as part of the withdrawal
  This dataset was withdrawn; its content is no longer available.

$ nemar dataset status on008082      # healthy control, unchanged
  Status:      active
  DOI:         https://doi.org/10.82901/nemar.on008082
  To download this dataset: ...

The Data: line above is the one the first pass of this change got wrong -- it still read incomplete ... see nemar admin data-integrity-sweep until running it against real data showed it. Also confirmed admin import status degrades cleanly against production, which does not yet return withdrawn: the line is simply absent.

tsc --noEmit (both projects) and biome check clean.

@neuromechanist
neuromechanist merged commit 2b14906 into dev Aug 23, 2026
12 checks passed
@neuromechanist
neuromechanist deleted the feature/issue-1048-surface-withdrawal branch August 23, 2026 06:48

This branch was previously deployed

1 inactive deployment
npm — 91290bbc Deployed Aug 23, 2026 by neuromechanist via publish #2059
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant