Repository navigation
Release 0.8.70: legacy nemar.org separation + DOI re-targeting, ORCID SSO, prescreen fix - #848
Merged
Merged
Conversation
feat: ORCID SSO backend (migration 0050 + /auth/orcid/* routes)
feat: ORCID-canonical name + profile fields (city/country, affiliation) — #835
* doi: canonical /dataset landing target Replace legacy nemar.org/dataexplorer/detail?dataset_id=<id> DOI _target and enrichment links with the canonical https://nemar.org/dataset/<id> (version DOIs: ?v=v<version>), via two shared helpers in shared/datacite-constants.ts. Part of epic #837 (Phase 1, #838). * doi: merge enrich-dataset shared import
* chore: remove legacy nemar.org dataexplorer couplings Retire both directions of the legacy nemar.org/dataexplorer coupling (epic #837 Phase 3): - Incoming (catalog pull): delete catalog-sync.yml, catalog-sync.ts, and the POST /admin/catalog/sync + GET /admin/catalog/status endpoints. This is what folded the ds shadow rows (#748). - Outgoing (datapipeline push): delete nemar-sync.ts + nemar-sync-reference.md. Split out the 4 BIDS-tree helpers into bids-tree.ts (still used by the metadata pipeline) and the slim delete-only client into legacy-purge.ts (for the Phase 4 one-time purge). - Callers: publish sync_nemar step -> logged no-op (kept in allSteps); webhook post-version-DOI sync -> refreshDatasetMetadata (keeps the metadata-columns refresh, drops the nemar.org push); reindex runDatasetSync -> refreshDatasetMetadata; remove POST /admin/datasets/:id/sync, GET /admin/sync/status, the nemar admin sync CLI + api methods, and NEMAR_USERNAME/NEMAR_PASSWORD bindings. - Delete 5 legacy-sync test files; relocate helper-test imports to bids-tree. nemar_sync_* columns + catalog_sync_log table + CF secrets are dropped in Phase 5. Tested: backend + CLI typecheck, biome, affected unit tests green. * webhooks: record metadata-refresh error to D1 on throw Review fixes (PR #843): delete the orphaned backend/test/version-doi-sync-trigger.test.ts (imported the removed shouldSyncToNemarAfterVersionDoi); wrap the 3 post-version-DOI refresh sites in refreshMetadataAfterVersionDoi so a pre-refresh throw (transient GitHub failure) records metadata_columns_error to D1 instead of only logging.
* scripts: one-off DOI re-target to canonical URL Epic #837 Phase 2. Guarded run-once ops script (NOT a CLI/endpoint) that re-points all NEMAR EZID DOI _targets from the legacy dataexplorer URL to the canonical https://nemar.org/dataset/<id> (forwarded to ww2). Dry-run by default; --execute requires EZID creds; idempotent/resumable (skips DOIs already on the canonical target); --only canaries a single dataset. Reuses the real ezid.ts client. Dry-run validated against prod D1: 1,194 DOIs (590 concept + 604 version). The execute run is a gated prod op. * retarget: use project isRetryable (no dataset-id false-match)
Epic #837 Phase 4 tooling (guarded run-once scripts, not CLI/endpoints): - purge-legacy-datapipeline.ts: delete our nm/on records from the 4 legacy dataexplorer tables via the extracted legacy-purge.ts client (dry-run default, --execute needs NEMAR_USERNAME/PASSWORD, idempotent). Dry-run: 608 datasets. - delete-ds-rows.ts: delete the ds shadow rows from D1 (children-first, datasets last -> FTS/FK cascade) + Vectorize, with a pre-delete snapshot. Closes #748. Dry-run confirms ds rows live ONLY in datasets (173), 0 in dependent tables. PRECONDITION: catalog-sync cron must be OFF first (else 4h re-fold). The --execute runs are gated prod ops.
* cleanup: drop dead nemar_sync state + retire sync docs Epic #837 Phase 5. Remove the now-dead legacy-sync remnants: - migration 0053 drops the nemar_sync_status/at/error columns (0014) and the catalog_sync_log table (0018); no index/trigger/FTS references them. - remove the two dead nemar_sync_status reads from the GET /datasets queries. - AGENTS.md: retire the nemar admin sync command + datapipeline-sync docs; describe the legacy site as being retired and DOIs landing on /dataset. Keeps the (nm|on|ds) exact-id regex (ds bookmarks resolve to their on mirror via source_id) and the legacy-purge.ts client + one-off scripts (needed for the post-deploy purge runs). Cross-repo follow-up: nemar-observability LIKE 'on%'. * cli: drop nemar_sync_status list indicator (now dead) Review fix (PR #846): the 0053 column drop + SELECT removal made dataset.nemar_sync_status always undefined, so the 'nemar dataset list' status indicator would have shown a yellow * on every managed dataset. Remove the indicator + the Dataset.nemar_sync_status field + the stale test-fixture column.
Cross-phase epic review (#847): seedFromBids merged existing related_identifiers (which may carry the old nemar.org/dataexplorer/detail IsDescribedBy URL) and appended the canonical /dataset/ URL without dropping the old one, so re-enrichment accumulated BOTH. Filter out the legacy dataexplorer landing URL so re-enrichment truly self-heals to the canonical one; update the LLM validator prompt to vouch for nemar.org/dataset/...; add a regression test.
…eparation Epic #837: Complete separation from legacy nemar.org dataexplorer + DOI re-targeting
The run-prescreen workflow sends issue_url: null when ISSUE_URL="" via jq (if $iu=="" then null else $iu end). The server validation checked `b.issue_url !== undefined` but not `b.issue_url !== null`, so null triggered 'issue_url must be a string' and every prescreen callback returned 400. The DB handler already handles null correctly (`body.issue_url || null`), so accepting null here is safe and consistent with the field being truly optional. Fixes: all prescreen callbacks failing with HTTP 400 since #756 advisory change.
The ORCID feature (#835) made signupSchema require orcid/city/country, but test/api.test.ts's signup bodies were not updated, so the 3 'existing X returns 409' tests got 400 (validation) before the duplicate check -- leaving integration-dev (required on main) red on dev since the ORCID merge, and blocking the 0.8.70 release. Add valid orcid/city/country to all 7 signup bodies so each isolates its intended validation; 409s reach the conflict path. Verified: full api.test.ts is 70/70 green against the dev backend.
neuromechanist
temporarily deployed
to
development
June 27, 2026 05:44 — with
GitHub Actions
Inactive
This was referenced Jun 27, 2026
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 0.8.70 (0.8.69 → 0.8.70)
Three workstreams on
dev:1. Legacy nemar.org dataexplorer separation + DOI re-targeting (epic #837, closes #793, fixes #748)
https://nemar.org/dataset/<id>(forwarder to ww2 live; 1,194 DOIs already re-targeted on prod).catalog-syncand outgoingnemar-sync+ all callers.ds-shadow deletion (run after this deploys).nemar_sync_*columns +catalog_sync_log(validated on the dev D1 mirror).2. ORCID SSO (#835)
3. #825 — prescreen callback accepts null
issue_urlPost-deploy operational sequence (epic #837)
After this deploys to prod: re-run
retarget-dois.ts --execute(interim DOIs) → disable the "Catalog Sync" workflow →purge-legacy-datapipeline.ts --execute→delete-ds-rows.ts --execute→ verify ww2 search has nods.CI / test notes
api-testred is pre-existing + unrelated: the ORCID feature made signup require city/country/ORCID, soapi.test.ts's field-less signup tests get 400 before the duplicate-409. Soft live-backend tier; not introduced by the epic. (Worth a follow-up to update api.test.ts for the new signup contract.)Merge
Regular merge (NOT squash) per repo convention. Triggers auto-tag
v0.8.70→ npm publish → prod deploy + migrations → sync-dev.