Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
f7ff1af
chore: sync dev with main after v0.8.69
Jun 21, 2026
30a78cb
chore: bump version to 0.8.70-dev0
Jun 21, 2026
c398115
migrate: add 0050 ORCID SSO schema
neuromechanist Jun 22, 2026
2f88e62
session: record auth_method on web_sessions
neuromechanist Jun 22, 2026
7d2842a
orcid: add OAuth service helpers
neuromechanist Jun 22, 2026
0fd8dc2
orcid: add /auth/orcid/* routes
neuromechanist Jun 22, 2026
f88b716
orcid: atomic writes + error handling in routes
neuromechanist Jun 22, 2026
18e2ab8
orcid: expand helper + migration test coverage
neuromechanist Jun 22, 2026
ede29e4
orcid: redirect via APP_BASE_URL not marketing apex
neuromechanist Jun 22, 2026
4531378
orcid: hand new signups to /auth/orcid/complete
neuromechanist Jun 22, 2026
da79dc7
Merge pull request #834 from nemarOrg/feature/issue-832-orcid-sso
neuromechanist Jun 22, 2026
ae65e2a
chore: bump version to 0.8.70-dev1
Jun 22, 2026
9226428
migrate: add 0051 user given/family name
neuromechanist Jun 22, 2026
708b73e
chore: bump version to 0.8.70-dev2
Jun 22, 2026
d03e345
migrate: add 0052 user profile fields
neuromechanist Jun 22, 2026
1fb32e3
orcid: fetch canonical name from public record
neuromechanist Jun 22, 2026
019b478
signup: require ORCID + city/country, store profile
neuromechanist Jun 22, 2026
0e485a4
cli: collect city/country/affiliation, ORCID required
neuromechanist Jun 22, 2026
c6728a0
orcid: dynamic name backfill on web login/link/signup
neuromechanist Jun 22, 2026
89af34f
orcid: finalize requires city/country; refresh name post-response
neuromechanist Jun 22, 2026
1bb1ab4
Merge pull request #836 from nemarOrg/feature/issue-835-orcid-profile
neuromechanist Jun 22, 2026
8092087
chore: bump version to 0.8.70-dev3
Jun 22, 2026
f32d2dd
Phase 1: DOI landing target -> canonical /dataset URL (#842)
neuromechanist Jun 26, 2026
77b8f0a
Phase 3: Kill both legacy nemar.org couplings (#843)
neuromechanist Jun 27, 2026
77f308f
Phase 2: one-off DOI re-target script (#844)
neuromechanist Jun 27, 2026
65ffc52
scripts: Phase 4 legacy purge + ds-row deletion (#845)
neuromechanist Jun 27, 2026
4bd802f
Phase 5: drop dead nemar_sync state + retire sync docs (#846)
neuromechanist Jun 27, 2026
4610711
enrich: drop legacy NEMAR landing URL on re-enrichment
neuromechanist Jun 27, 2026
83f84e3
Merge pull request #847 from nemarOrg/feature/issue-837-epic-legacy-s…
neuromechanist Jun 27, 2026
d790bcb
chore: bump version to 0.8.70-dev4
Jun 27, 2026
a2cc1bc
fix: accept null issue_url in prescreen callback validation (#825)
bruAristimunha Jun 27, 2026
f39c650
chore: bump version to 0.8.70-dev5
Jun 27, 2026
8f10f89
test: add ORCID/city/country to signup test bodies (#849)
neuromechanist Jun 27, 2026
b731b8b
chore: bump version to 0.8.70-dev6
Jun 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
159 changes: 0 additions & 159 deletions .github/workflows/catalog-sync.yml

This file was deleted.

13 changes: 5 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,9 @@ The backend code (`ORG_NAME = "nemarDatasets"` in `backend/src/services/github.t
### Public Browser Sites
Two user-facing dataset browsers exist during the cutover; **prefer ww2 for new references**.

- **`https://ww2.nemar.org`** — **current beta browser** (Astro SSR, lives in `nemarOrg/website`). Dataset pages at `ww2.nemar.org/dataset/<id>`. Reads `api.nemar.org` and `data.nemar.org` directly; does not depend on the legacy datapipeline.
- **`https://nemar.org/dataexplorer/...`** — **legacy PHP site**. Backed by the `dataexplorer_*` tables on `nemar.org/api/dataexplorer/datapipeline`, which `nemar admin sync run` populates. Still public, slated for replacement by ww2.
- **Implication for on\* datasets:** `nemar admin sync` intentionally skips on\*-prefix datasets (alternate_id mapping pending, issue #339), so they render on ww2.nemar.org but the legacy `nemar.org/dataexplorer` detail page is empty until that ships. The archive zip download link in the legacy UI is driven by `zip_file_size` on the datapipeline row, so even an existing S3 archive won't surface there for on\* datasets.
- **Default in docs/comments:** when something says "the website," "the browser," or "the UI" without further qualification, mean ww2.nemar.org. Name `nemar.org` explicitly only when referring to the legacy dataexplorer, the datapipeline tables, or the `nemar.org` DOI landing target.
- **`https://ww2.nemar.org`** — **current dataset browser** (Astro SSR, lives in `nemarOrg/website`). Dataset pages at `ww2.nemar.org/dataset/<id>`. Reads `api.nemar.org` and `data.nemar.org` directly. DOIs land on the canonical `https://nemar.org/dataset/<id>` (versions `?v=v<version>`), which a Cloudflare redirect on the `nemar.org` zone forwards to ww2.
- **`https://nemar.org/dataexplorer/...`** — **legacy PHP site, being retired.** As of epic #837 NEMAR no longer syncs to or from it: the outgoing datapipeline push and the incoming 4h catalog pull are removed, our `nm`/`on` records are purged from its `dataexplorer_*` tables, and the legacy `ds######` shadow rows are dropped from our D1. It may still serve its own OpenNeuro `ds` catalog independently.
- **Default in docs/comments:** when something says "the website," "the browser," or "the UI" without further qualification, mean ww2.nemar.org. Name `nemar.org` explicitly only when referring to the legacy dataexplorer or the `nemar.org/dataset/<id>` DOI landing target.

### S3 Bucket Structure
```
Expand Down Expand Up @@ -115,7 +114,7 @@ Datasets are synced to SDSC Hallu (`ssh hallu`) for processing pipelines and dow
- **Zip dir:** `/data/qumulo/openneuro/zip_files/` (downloadable archives from S3)
- **Discovery:** Queries `GET /datasets` API, filters for `nm`-prefix only
- **Manual run:** `ssh hallu /data/qumulo/openneuro/nemar-cli/scripts/hallu-sync.sh --dataset nm000132 --verbose`
- **Note:** This syncs data files. Legacy `nemar.org/dataexplorer` metadata is synced separately via `nemar admin sync run` or automatically on version DOI publish.
- **Note:** This syncs data files only. (The legacy `nemar.org/dataexplorer` metadata sync was retired in epic #837.)

## Environment Setup
```bash
Expand Down Expand Up @@ -412,9 +411,7 @@ nemar admin revert # Revert dataset to previous version
nemar admin make-public # Publish dataset (permanent)
nemar admin delete-dataset # Delete dataset and all resources
nemar admin bulk-delete # Delete multiple phantom/orphaned datasets (owner only)
nemar admin sync run # Sync dataset metadata to nemar.org (legacy dataexplorer)
nemar admin sync status # Show nemar.org sync status
nemar admin reindex <id> # Refresh enrichment + nemar.org sync + D1 metadata columns
nemar admin reindex <id> # Refresh enrichment + D1 metadata columns
nemar admin reindex --all # Bulk; also --missing-metadata, --stale [--older-than N]
nemar admin summary # summary.json coverage across published versions
nemar admin email-preferences show # Show email notification preferences
Expand Down
2 changes: 1 addition & 1 deletion backend/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "nemar-api",
"version": "0.8.69",
"version": "0.8.70-dev6",
"private": true,
"scripts": {
"dev": "wrangler dev -c wrangler-sccn.toml --env dev",
Expand Down
42 changes: 42 additions & 0 deletions backend/src/db/migrations/0050_orcid_sso.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
-- ORCID SSO: OAuth identities + verified-iD flag + session auth method (#832).
--
-- ORCID becomes a login provider alongside the passwordless email-code flow
-- (0026). Accounts are linked by ORCID iD (the OAuth `orcid`/`sub`), never by
-- email: ORCID keeps email private by default and returns no email claim, so
-- email is not a reliable join key and auto-linking on it is an account-takeover
-- vector.
--
-- oauth_identities - one row per linked provider account. UNIQUE
-- (provider, provider_subject) makes "one ORCID ->
-- one NEMAR account" a DB invariant; the FK cascade
-- drops the link when a user is hard-deleted.
-- users.orcid_verified - distinguishes an OAuth-proven iD from the
-- DOI-*discovered* value already in users.orcid
-- (0026:72), which is citation-facing and must not
-- be overwritten by a login. provider_subject in
-- oauth_identities is always the source of truth for
-- the verified iD; users.orcid stays the citation
-- value, and orcid_verified=1 means the two agree.
-- web_sessions.auth_method - 'email_code' | 'orcid'; lets /auth/me and admin
-- tooling tell how a session was established.
--
-- Additive only (CREATE TABLE + ADD COLUMN), so unlike 0026 there is no table
-- rebuild and no FK gymnastics.

CREATE TABLE IF NOT EXISTS oauth_identities (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
provider TEXT NOT NULL CHECK (provider IN ('orcid')),
provider_subject TEXT NOT NULL, -- ORCID iD, e.g. 0000-0001-2345-6789
provider_email TEXT, -- usually NULL (ORCID email private by default)
display_name TEXT,
connected_at TEXT NOT NULL DEFAULT (datetime('now')),
last_login_at TEXT,
UNIQUE (provider, provider_subject)
);

CREATE INDEX IF NOT EXISTS idx_oauth_identities_user ON oauth_identities(user_id);

ALTER TABLE users ADD COLUMN orcid_verified INTEGER NOT NULL DEFAULT 0;

ALTER TABLE web_sessions ADD COLUMN auth_method TEXT; -- 'email_code' | 'orcid'
10 changes: 10 additions & 0 deletions backend/src/db/migrations/0051_user_real_name.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
-- Real researcher name on the user row, for display, uploads, and citations
-- instead of the username (#832 follow-up; supersedes the display_name sketch
-- in the #833 onboarding plan).
--
-- ORCID is the source of truth: the OAuth flow can populate given_name /
-- family_name from the iD's record. Existing accounts are backfilled manually
-- (only a handful of users). Additive + nullable, so no rebuild and no impact
-- on rows that don't have a name yet.
ALTER TABLE users ADD COLUMN given_name TEXT;
ALTER TABLE users ADD COLUMN family_name TEXT;
9 changes: 9 additions & 0 deletions backend/src/db/migrations/0052_user_profile_fields.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
-- Standardized profile fields (#835): institution + export-control screening.
--
-- affiliation is optional; city/country are required at the app/CLI layer for
-- US export-control / sanctions screening. Kept nullable here so existing rows
-- aren't broken and SQLite doesn't need a table rebuild; the "required" check
-- lives in the signup/onboarding validators, not a NOT NULL constraint.
ALTER TABLE users ADD COLUMN affiliation TEXT;
ALTER TABLE users ADD COLUMN city TEXT;
ALTER TABLE users ADD COLUMN country TEXT;
17 changes: 17 additions & 0 deletions backend/src/db/migrations/0053_drop_legacy_sync_state.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
-- Drop the legacy nemar.org datapipeline sync state (epic #837 Phase 5).
--
-- The outgoing datapipeline push (nemar-sync.ts) and the incoming catalog pull
-- (catalog-sync.ts + /admin/catalog/sync) were removed in Phase 3, so the
-- nemar_sync_* columns (migration 0014) and the catalog_sync_log table
-- (migration 0018) are now dead and have no remaining readers/writers.
--
-- DROP COLUMN is safe here: no index, trigger, FK, or the datasets_fts external
-- -content virtual table references these columns (datasets_fts indexes only
-- name/description/authors/tasks/modalities/readme). Validated on the SCCN dev
-- D1 mirror before prod.

ALTER TABLE datasets DROP COLUMN nemar_sync_status;
ALTER TABLE datasets DROP COLUMN nemar_sync_at;
ALTER TABLE datasets DROP COLUMN nemar_sync_error;

DROP TABLE IF EXISTS catalog_sync_log;
3 changes: 3 additions & 0 deletions backend/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import { maintenanceMode } from "./middleware/maintenance";
import { rateLimiter } from "./middleware/rateLimit";
import { adminRoutes } from "./routes/admin";
import { authRoutes } from "./routes/auth";
import { authOrcidRoutes } from "./routes/auth-orcid";
import { authWebRoutes } from "./routes/auth-web";
import { catalogIndexResponse, dataRoutes } from "./routes/data";
import { datasetRoutes } from "./routes/datasets";
Expand Down Expand Up @@ -132,6 +133,8 @@ api.route("/auth", authRoutes);
// CLI flow; no path overlap with authRoutes (existing /signup, /login,
// /verify, etc. vs new /code/request, /code/verify, /logout, /me).
api.route("/auth", authWebRoutes);
// ORCID SSO (#832). Same /auth prefix; new paths under /auth/orcid/*.
api.route("/auth", authOrcidRoutes);
api.route("/users", userRoutes);
api.route("/admin", adminRoutes);
api.route("/datasets", datasetRoutes);
Expand Down
Loading
Loading