Repository navigation
Phase 4: legacy DB purge + ds-row deletion (closes #748) - #845
Merged
neuromechanist merged 1 commit intoJun 27, 2026
Conversation
Epic #837 Phase 4 tooling (guarded run-once scripts, not CLI/endpoints): - purge-legacy-datapipeline.ts: delete our nm/on records from the 4 legacy dataexplorer tables via the extracted legacy-purge.ts client (dry-run default, --execute needs NEMAR_USERNAME/PASSWORD, idempotent). Dry-run: 608 datasets. - delete-ds-rows.ts: delete the ds shadow rows from D1 (children-first, datasets last -> FTS/FK cascade) + Vectorize, with a pre-delete snapshot. Closes #748. Dry-run confirms ds rows live ONLY in datasets (173), 0 in dependent tables. PRECONDITION: catalog-sync cron must be OFF first (else 4h re-fold). The --execute runs are gated prod ops.
📦 Package PreviewYour package is ready to test! bun install -g nemar-cli@PR845Version: 🤖 Updated on each push. |
neuromechanist
merged commit Jun 27, 2026
65ffc52
into
feature/issue-837-epic-legacy-separation
1 check passed
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 4 of epic #837 — the data-purge half of "complete separation". Two guarded run-once scripts (same pattern as the Phase 2 re-target; not CLI/endpoints):
scripts/purge-legacy-datapipeline.ts— deletes NEMAR's ownnm/onrecords from the 4 legacydataexplorerdatapipeline tables so the legacy site stops showing our datasets. Reuses the slimlegacy-purge.tsclient extracted in Phase 3. Dry-run default;--executeneedsNEMAR_USERNAME/NEMAR_PASSWORD; idempotent. Dry-run: 608 nm+on datasets.scripts/delete-ds-rows.ts— deletes the legacyds######shadow rows (folded OpenNeuro catalog, owner=SYSTEM) from D1 + Vectorize, closing Search returns non existent dataset #748 (search returning backing-less shadows). Children-first thendatasets(FTS trigger + FK cascade); pre-delete snapshot; Vectorizedelete-vectors. Dry-run confirms the shadows live only indatasets(173 rows), 0 in every dependent table.Ordering precondition: the incoming catalog-sync (GitHub "Catalog Sync" workflow +
/admin/catalog/sync) must be stopped before deletingdsrows, or the 4h cron re-folds them. The runs happen post-deploy / after the cron is disabled, gated on owner go.Closes #840? no — Closes #793; fixes #748. Part of epic #837.
Test plan
bun test test/delete-ds-rows.test.ts(FK-safe delete order, ds-only targeting, toCountSql).--executeruns performed out-of-band (cron OFF first), gated on owner go.