Skip to content

Phase 3: Kill both legacy nemar.org couplings - #843

Merged
neuromechanist merged 2 commits into
feature/issue-837-epic-legacy-separationfrom
feature/issue-840-phase3-kill-legacy-couplings
Jun 27, 2026
Merged

neuromechanist merged 2 commits into
feature/issue-837-epic-legacy-separationfrom
feature/issue-840-phase3-kill-legacy-couplings

Conversation

@neuromechanist

Copy link
Copy Markdown
Contributor

Summary

Phase 3 of epic #837. Removes both directions of the legacy nemar.org/dataexplorer coupling in code.

Incoming (catalog pull): deletes catalog-sync.yml (4h cron), catalog-sync.ts, and the POST /admin/catalog/sync + GET /admin/catalog/status endpoints. This is the mechanism that folded the 183 ds shadow rows (#748); after this they stop regenerating.

Outgoing (datapipeline push): deletes nemar-sync.ts + nemar-sync-reference.md. The file is split first:

  • 4 BIDS-tree helpers (parseParticipantsTsv, countSessionDirs, countSubjectDirs, extractTasks) -> new bids-tree.ts (still used by the metadata pipeline).
  • the auth + delete client -> new legacy-purge.ts (the one-time purge client for Phase 4).

Callers:

  • publish sync_nemar step -> logged no-op (kept in allSteps/PublicationStep so old DB records stay valid; mirrors the disabled upload_to_zenodo).
  • webhook post-version-DOI sync -> refreshDatasetMetadata (preserves the metadata-columns refresh; drops only the nemar.org push) + removes the shouldSyncToNemarAfterVersionDoi gate.
  • reindex runDatasetSync renamed to refreshDatasetMetadata (metadata-only); 2 callers updated.
  • removed POST /admin/datasets/:id/sync, GET /admin/sync/status, the nemar admin sync run/status CLI + syncDataset/getSyncStatus api methods, and NEMAR_USERNAME/NEMAR_PASSWORD bindings.

No prod mutation here (all code). nemar_sync_* columns + catalog_sync_log table + CF secrets are dropped in Phase 5; the legacy DB purge + ds deletion run in Phase 4.

Closes #840
Part of epic #837

Test plan

  • Backend tsc --noEmit clean; CLI tsc --noEmit clean.
  • Biome clean on all changed code files (1 pre-existing unrelated test-format nit left untouched).
  • Deleted 5 legacy-sync test files; relocated helper-test imports to bids-tree.
  • bun test on affected suites (dataset-metadata-columns, dataset-reindex, manifest-callback) green.

Retire both directions of the legacy nemar.org/dataexplorer coupling
(epic #837 Phase 3):

- Incoming (catalog pull): delete catalog-sync.yml, catalog-sync.ts, and
  the POST /admin/catalog/sync + GET /admin/catalog/status endpoints. This
  is what folded the ds shadow rows (#748).
- Outgoing (datapipeline push): delete nemar-sync.ts + nemar-sync-reference.md.
  Split out the 4 BIDS-tree helpers into bids-tree.ts (still used by the
  metadata pipeline) and the slim delete-only client into legacy-purge.ts
  (for the Phase 4 one-time purge).
- Callers: publish sync_nemar step -> logged no-op (kept in allSteps);
  webhook post-version-DOI sync -> refreshDatasetMetadata (keeps the
  metadata-columns refresh, drops the nemar.org push); reindex runDatasetSync
  -> refreshDatasetMetadata; remove POST /admin/datasets/:id/sync, GET
  /admin/sync/status, the nemar admin sync CLI + api methods, and
  NEMAR_USERNAME/NEMAR_PASSWORD bindings.
- Delete 5 legacy-sync test files; relocate helper-test imports to bids-tree.

nemar_sync_* columns + catalog_sync_log table + CF secrets are dropped in
Phase 5. Tested: backend + CLI typecheck, biome, affected unit tests green.
@github-actions

github-actions Bot commented Jun 27, 2026 •

Copy link
Copy Markdown

📦 Package Preview

Your package is ready to test!

bun install -g nemar-cli@PR843

Version: 0.8.70-PR843.8debf37
Commit: 8debf37


🤖 Updated on each push.

Review fixes (PR #843): delete the orphaned backend/test/version-doi-sync-trigger.test.ts (imported the removed shouldSyncToNemarAfterVersionDoi); wrap the 3 post-version-DOI refresh sites in refreshMetadataAfterVersionDoi so a pre-refresh throw (transient GitHub failure) records metadata_columns_error to D1 instead of only logging.
@neuromechanist
neuromechanist merged commit 77b8f0a into feature/issue-837-epic-legacy-separation Jun 27, 2026
1 check passed
@neuromechanist
neuromechanist deleted the feature/issue-840-phase3-kill-legacy-couplings branch June 27, 2026 00:26

This branch was previously deployed

1 inactive deployment
npm — 8debf372 Deployed Jun 27, 2026 by neuromechanist via publish #1581
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant