diff --git a/.context/decisions/0045-the-cli-and-the-web-say-one-thing-about-an-account.md b/.context/decisions/0045-the-cli-and-the-web-say-one-thing-about-an-account.md new file mode 100644 index 000000000..dc80f7f3f --- /dev/null +++ b/.context/decisions/0045-the-cli-and-the-web-say-one-thing-about-an-account.md @@ -0,0 +1,191 @@ +# ADR 0045: The CLI and the web say one thing about an account + +**Status:** accepted +**Date:** 2026-09-05 +**Owner:** Seyed Yahya Shirazi + +> Numbering note: epic #1250 runs its phases in parallel worktrees off one epic +> branch, so the numbers were claimed out of order — phase 1 took 0040, phase 5 +> took 0041, phase 3 took 0042, phase 4 took 0043 and phase 7 took 0044. This is +> phase 8, the last of them, and it is 0045. The index's gapless check goes green +> once the epic branch holds all six, which it does. + +## Context + +Epic #1250 rebuilt the account model across two surfaces at once. ADR 0040 fixed +the tiers, 0042 built the upload-access request, 0043 made an identity back one +account, 0044 brought identity self-service to the CLI, and +`nemarOrg/website#301`/`#310` built the browser half. By the end of phase 7 the +same account could be described three different ways in one afternoon: + +- `POST /users/me/upload-access/request` refused with + `missing: ["github_username", "city"]`, assembled by six hand-written `blank()` + checks in `services/upload-access.ts`; +- `nemar auth status` said nothing about either field, and + `nemar auth request-upload-access` rendered them through `UPLOAD_ACCESS_FIX`, + a private table in `src/commands/auth.ts` that named commands with argument + placeholders (`nemar auth profile set-username `); +- the dashboard derived its own list client-side, from the account fields + `/auth/me` happened to carry, and rendered it through the website's own + sentences. + +Three implementations of one rule, in two repositories, none of which failed +when another changed. The concrete drift was already visible in the wording: the +website said "Set it in Settings or run `nemar auth profile set-github`" and the +CLI said "run 'nemar auth profile set-github '", for the same gap on the +same account. + +There is a second, smaller version of the same problem. ADR 0042 built an admin +sweep to give the username-less web/ORCID rows a handle. It has not been run: +production still holds 18 of them (the 19 that ADR counted, minus a duplicate +account deleted since), and the first run is scheduled for after this epic +deploys. Even once it has, the sweep closes the accounts that exist and nothing +after them — a web sign-up whose owner abandons onboarding lands straight back +in `username IS NULL` and stays there until an operator remembers to run it +again. It also never revisits: its candidate predicate is +`username IS NULL OR TRIM(username) = ''`, so a row it has named is a row it +cannot see. + +## Decision + +**One matrix, one copy table, one gap function — declared here, transcribed +there.** + +1. `shared/contract/account-copy.ts` holds every user-facing sentence about + tiers, upload access and missing fields, as literal strings under dotted + keys. `nemarOrg/website/src/lib/account-copy.ts` is a transcription of it. + Each repo has a drift test that reads the other's file **as text** and fails + on any shared key whose string differs + (`test/account-copy-parity.test.ts` here, `test/account-copy-drift.test.ts` + there); both skip with a note when the other checkout is absent, because CI + clones one repo. +2. `shared/contract/profile-gaps.ts` holds the matrix as data — for each field, + what it blocks, where it is set on each surface, and whether it is derivable + from an account row — plus the pure `computeProfileGaps(account)`. +3. **That function is the only implementation.** `GET /users/me` and + `GET /auth/me` both report its output as `profile_gaps`, and + `checkUploadAccessRequest` builds its `missing` array from it. The three + answers cannot disagree because there is one answer; + `backend/test/profile-gaps-route.test.ts` asks all three about one row and + compares. +4. **A username is assigned lazily as well as in batches.** At a web sign-in + (`/auth/code/verify`, the ORCID callback sign-in, and `/auth/orcid/finalize`) + an account with `username IS NULL` and a usable name gets the ADR 0042 + suggestion, collision-suffixed. Migration 0079 adds + `users.username_auto_assigned`, set by both paths and cleared when the person + changes the username. +5. **Sandbox training stays CLI-only** and is deliberately outside the shared + matrix. + +## Consequences + +**A sentence now changes in one place.** Editing copy means editing this +contract and transcribing it; the drift tests make forgetting the second half +loud on a maintainer's machine, and the mirroring is asymmetric on purpose — +`cli.`-prefixed keys are the CLI's own and the website reports them as a note. + +**A rule now changes in one place too, and that is the sharper claim.** Adding +a precondition to the upload-access request means adding a row to +`PROFILE_GAP_MATRIX`, and every surface picks it up: the refusal, the CLI's +Profile block, the upload preflight, the dashboard nudge. Nothing has to be +remembered. + +**The wire grew two fields and lost none.** `profile_gaps` is `.optional()` on +the CLI's `userSchema` (an older backend omits it, and absent must not read as +"nothing missing") and REQUIRED on `webUserSchema`, which only ever describes +what this backend sends. + +**`set_on` on the wire names surfaces, not prose.** `["web", "cli"]`, or +`["web"]` for a name owned by a verified ORCID record. "Settings" is a website +noun and `nemar auth profile set-github` a CLI one; neither is something the +backend should spell for two clients that already know their own vocabulary and +read it out of the copy table. + +**The CLI still caches.** `nemar auth status` prints the gaps from the config +cache so it works offline, with the same three-state honesty the upload-access +line already has: cached, "not checked" when nobody has refreshed, and "not +checked" again when a refresh was asked for and failed. Only `nemar auth +profile` and the upload preflight always fetch. + +**The upload preflight is a hard stop with one deliberate exception.** A missing +grant ends the run before validation, because the run cannot succeed; `--dry-run` +continues, because it uploads nothing and the plan is what was asked for. A +preflight that cannot READ the grant — offline, a 5xx, an older backend — warns +and continues, because the server enforces the gate regardless and refusing an +upload over a briefly unavailable status endpoint invents a refusal nobody made. + +**The username finally gets the index ADR 0042 promised it.** That ADR's +Consequences say a case-variant race "can still land two usernames one shift key +apart" and that "phase 4's case-insensitive unique index closes it"; phase 4 +(migration 0077) built that index for `orcid` and `email` and not for +`username`, so the sentence has been describing a file that did not exist. +Migration 0080 is it — the same partial predicate as 0077's two, and clean to +create because the username catalogue holds no case-variant duplicates. + +**Two write paths now name accounts, and both mark what they did.** The sweep is +the batch path and sign-in is the lazy path; between them an account cannot keep +browsing NEMAR without a handle. The mark is a column and not just an audit row +because the copy that offers "we chose this from your name, change it if you +like" renders on every page load and cannot scan an audit log. + +**The lazy assignment is a nudge, not a guarantee.** No name, a name that folds +to nothing in ASCII, or a saturated base leaves the column NULL and onboarding +asks — nothing is ever invented from the email local part (ADR 0042). At +`/auth/code/verify` the claim rides inside the sign-in's own transaction; on the +two ORCID paths it is chained behind the public-record name refresh, which +already runs after the response, so it lands a moment later. + +**What is NOT shared, and should not become shared.** Sandbox training (no web +surface); anchors and hrefs (website routing); the refusal `message` strings, +which the backend builds and both clients prefer verbatim when present. Adding a +sandbox row to the shared matrix would make the table a claim about both +surfaces that is false on one of them. + +## Alternatives considered + +- **Publish `@nemar/contract` to npm and have the website import it.** The + correct end state, and out of scope for a phase that had to ship alongside a + website release. The contract directory is already written to lift verbatim + (zero deps beyond zod); the drift tests are the interim, and they cost a + transcription rather than a release cycle. +- **Let each surface derive its own gaps from the account fields.** What the + website did while waiting for this, and what the CLI would have had to do. + Two derivations that agree today and are not tested against each other are one + bug fix away from disagreeing, and the disagreement is invisible from either + side. +- **Have the backend send rendered sentences.** It would guarantee identical + wording and destroy everything else: a terminal wants a backtick-quoted + command, a browser wants an anchor to `/settings#profile-city`, and the + backend would be spelling both. Sending `field` plus `blocks` and rendering + locally keeps the rule central and the presentation where it belongs. +- **Assign usernames only in the sweep (ADR 0042 as shipped).** It converges on + the rows that exist and diverges on the ones arriving. The sweep stays; it is + now the batch half of a pair. +- **Assign at sign-up rather than at sign-in.** ORCID sign-up has no name yet: + `POST /auth/orcid/finalize` inserts the row and the given/family names arrive + afterwards from the public record. Assigning at the INSERT would skip exactly + the accounts this exists for, which is why the assignment is chained behind + the name refresh on both ORCID paths. +- **Make the upload preflight a warning only.** Kinder in the moment and wrong: + the upload will 403 at the server, after validation and after a repository has + been asked for. The one case where warning is right is the one where we do not + know the answer, and that is the case that warns. + +## Receipts + +- Issue #1268 (this phase), epic #1250, and the website's companion + `nemarOrg/website#309` / `#310`. +- ADR 0040 (tiers, and `service_access` as the single grant), ADR 0041 (the name + a DOI cites), ADR 0042 (the request, the username default, the sweep), + ADR 0043 (one person, one account), ADR 0044 (identity self-service on the + CLI). +- `shared/contract/account-copy.ts`, `shared/contract/profile-gaps.ts`, + `backend/src/services/profile-gaps.ts`, + `backend/src/services/username-assignment.ts`, migrations + `0079_username_auto_assigned.sql` and + `0080_username_case_insensitive_unique.sql`. +- `test/profile-gaps-matrix.test.ts` (the matrix against a fixture copied from + the website's table, and all 2^7 field combinations), + `test/account-copy-parity.test.ts`, `backend/test/profile-gaps-route.test.ts`, + `backend/test/username-auto-assign-route.test.ts`, + `test/auth-gaps-cli.test.ts`. diff --git a/.context/decisions/README.md b/.context/decisions/README.md index 5e9fd57e8..705b2f727 100644 --- a/.context/decisions/README.md +++ b/.context/decisions/README.md @@ -83,6 +83,7 @@ invisible one — which is why it is checked rather than trusted. - [ADR 0042](0042-upload-access-is-requested-once-by-the-person-who-wants-it.md) - Upload access is requested once, by the person who wants it - [ADR 0043](0043-one-person-one-account.md) - One person, one account: ORCID, email and GitHub each back at most one live account - [ADR 0044](0044-identity-self-service-reaches-the-cli.md) - Identity self-service reaches the CLI, and ORCID does it through the browser +- [ADR 0045](0045-the-cli-and-the-web-say-one-thing-about-an-account.md) - The CLI and the web say one thing about an account: one matrix, one copy table, one gap function ## Backfill note (2026-07-31) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 86bedf6bc..131dd34e4 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -317,7 +317,38 @@ jobs: - name: Install backend dependencies run: cd backend && bun install + # The website's transcription of shared/contract/account-copy.ts, so + # test/account-copy-parity.test.ts can actually compare the two here + # (#1268, ADR 0045). Without it that test found no checkout and returned + # early: the parity claim was enforced on developer machines and nowhere + # else, while reporting green as a required check. + # + # nemarOrg/website is PUBLIC, so no token is needed. Sparse, non-cone, one + # file: a table of string constants, not that repo's module graph. It + # lands in website-copy/ at the repo root, which the unit-pure file glob + # below cannot reach (it lists test/*.test.ts and backend/test/*.test.ts + # only) and .gitignore covers. + # + # REF PAIRING: website `staging` is the counterpart of nemar-cli `dev` and + # of every feature branch off it; website `main` pairs with nemar-cli + # `main`. Comparing a dev-side contract against the website's released + # copy would fail every PR that lands a copy change ahead of the website + # release, which is the normal order. + - name: Check out the website's copy table + uses: actions/checkout@v4 + with: + repository: nemarOrg/website + path: website-copy + sparse-checkout: src/lib/account-copy.ts + sparse-checkout-cone-mode: false + ref: ${{ (github.base_ref == 'main' || github.ref_name == 'main') && 'main' || 'staging' }} + - name: Run pure unit tests (no live backend) + env: + # Names the checkout above. The parity test treats this as the + # contract: a path that is not there FAILS rather than skipping, so a + # broken checkout step cannot silently disable the check. + NEMAR_WEBSITE_ACCOUNT_COPY: website-copy/src/lib/account-copy.ts run: | # Deliberately no TEST_API_URL / secrets: these tests don't touch the # backend, so a misclassified file fails loudly here instead of diff --git a/.gitignore b/.gitignore index 6735c0726..c7e515a08 100644 --- a/.gitignore +++ b/.gitignore @@ -69,6 +69,11 @@ __pycache__/ # inside the working tree since the driver moved into this repo (#1109). .zarr-secrets.env +# The unit-pure CI job sparse-checks out nemarOrg/website's account-copy.ts +# here so test/account-copy-parity.test.ts can compare the two (#1268). Ignored +# so a developer who reproduces that checkout locally cannot commit it. +website-copy/ + # Regenerated by the deno-based BIDS validator tooling on local test runs. # Untracked on dev and unrelated to any change here; it has been committed by # accident twice now (dropped once in 745059e), so ignore it rather than rely diff --git a/AGENTS.md b/AGENTS.md index 2943f618f..d6343959b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,7 +48,19 @@ Load-bearing ones to know before touching the relevant area: 0042 (upload access is requested once, by the person who wants it), 0043 (one person, one account: an ORCID iD, an email or a GitHub handle backs at most one live account), 0044 (identity self-service reaches the CLI; ORCID links through a browser handoff whose -signed state names the account). +signed state names the account), +0045 (the CLI and the web say one thing about an account). + +**Account copy and the profile-gap matrix are declared once, in +[`shared/contract/account-copy.ts`](shared/contract/account-copy.ts) and +[`shared/contract/profile-gaps.ts`](shared/contract/profile-gaps.ts)** (ADR 0045). +Every sentence about tiers, upload access or a missing field comes from the first; +`computeProfileGaps` in the second is the ONE rule behind `profile_gaps` on +`GET /users/me`, `profile_gaps` on `GET /auth/me`, and the `missing` array a refused +upload-access request carries. `nemarOrg/website` transcribes both files and a drift +test on each side compares them when both checkouts are present. Change a sentence or a +rule there, never at a call site; `cli.`-prefixed keys are the CLI's own and the website +ignores them. --- diff --git a/backend/src/db/flag.ts b/backend/src/db/flag.ts new file mode 100644 index 000000000..febd55864 --- /dev/null +++ b/backend/src/db/flag.ts @@ -0,0 +1,33 @@ +/** + * One place that turns a D1 boolean column into a JavaScript boolean. + * + * SQLite has no boolean type, so every flag on `users` is an INTEGER with a + * `NOT NULL DEFAULT 0` — `email_verified` (0001), `orcid_verified` (0050), + * `service_access` (0062), `identity_conflict` (0077), + * `username_auto_assigned` (0079). Each one is read by several SELECTs and + * converted at each of them, and the conversions were drifting into three + * spellings for one question: `x === 1`, `!!x`, and `x === true || x === 1` for + * a value that might already have been normalised upstream. + * + * They are not equivalent. `!!x` says true for any non-zero integer AND for a + * non-empty string, so a column that ever held something unexpected reads as + * set; `x === 1` says false for a `true` that a caller had already converted. + * This is the third spelling, which is the correct one for both directions, and + * it lives here so a new flag column has somewhere obvious to be read from + * rather than a fourth spelling to invent. + * + * Lifted out of services/profile-gaps.ts (#1268 review), where it was a private + * helper doing exactly this for the three flags the gap matrix reads. + */ + +/** + * True for the SQLite integer 1 and for a boolean `true`; false for everything + * else, INCLUDING null and undefined. + * + * `null | undefined` is accepted rather than pushed onto callers because the + * common shape is `flag(row?.some_column)` on a row that may not have been + * found — and an absent row's flag is not set, which is the same answer as 0. + */ +export function flag(value: number | boolean | null | undefined): boolean { + return value === true || value === 1; +} diff --git a/backend/src/db/migrations/0079_username_auto_assigned.sql b/backend/src/db/migrations/0079_username_auto_assigned.sql new file mode 100644 index 000000000..5814b503d --- /dev/null +++ b/backend/src/db/migrations/0079_username_auto_assigned.sql @@ -0,0 +1,47 @@ +-- Mark a username the account did not choose (epic #1250 phase #1268, ADR 0045). +-- +-- WHAT THIS IS FOR. ADR 0042 gave the username-less web/ORCID rows a handle +-- through the admin backfill sweep, and phase 8 adds the lazy half: an account +-- that signs in with `username IS NULL` and a usable name gets the same +-- suggestion assigned server-side, in the same transaction as the sign-in, so +-- that abandoning onboarding no longer leaves the column NULL forever. Both +-- paths write a handle nobody typed. +-- +-- That fact has to survive the write, because it changes what the product may +-- say next. Onboarding and Settings offer "we chose this from your name, change +-- it if you like" (`onboarding.username.auto_assigned.*` in +-- shared/contract/account-copy.ts) — an offer that would be nonsense to someone +-- who typed their handle at CLI signup, and the users table has no other way to +-- tell the two apart afterwards. The audit row records the EVENT; this column is +-- the STATE, and only the state can be read cheaply on every page load. +-- +-- CLEARED, NOT STICKY. `PATCH /auth/profile` sets it back to 0 whenever the +-- username actually changes: once the person has picked one, the offer is spent +-- and repeating it would invite them to change a handle they just chose. +-- +-- DEFAULT 0 IS THE HONEST DEFAULT for the back catalogue. Every existing row is +-- one of: a CLI signup that typed its username, or a web row that is still NULL +-- (and so has nothing to have been auto-assigned). Neither is an +-- auto-assignment, and no data migration guesses at history here. +-- +-- The nameless rows are still nameless. ADR 0042 built the sweep; as of +-- 2026-09-06 production holds 18 of them (the 19 that ADR describes, minus a +-- duplicate account since deleted) and the sweep has been run ZERO times — it +-- is scheduled for after this epic deploys. So the FIRST production run of +-- `POST /admin/users/backfill-usernames` after this column exists is what marks +-- those rows; there is nothing here for it to re-mark. +-- +-- And "re-run the sweep" is not a repair for a row it has already named, +-- anywhere: its candidate predicate is `username IS NULL OR TRIM(username) = ''` +-- (REMAINING_SQL, routes/admin/user-usernames.ts), so a row that HAS a username +-- is no longer a candidate and no future run will ever look at it again. The +-- only rows that can be left holding 0 after being auto-named are ones named by +-- a sweep that ran before this migration — dev D1 only, since production has +-- run none — and fixing those would be a one-off UPDATE against a list of ids, +-- not a sweep. +-- +-- One statement, one column, NOT NULL with a default, so it is a rewrite-free +-- ALTER on SQLite and re-running the file is the only thing that fails (which is +-- what d1_migrations already prevents). + +ALTER TABLE users ADD COLUMN username_auto_assigned INTEGER NOT NULL DEFAULT 0; diff --git a/backend/src/db/migrations/0080_username_case_insensitive_unique.sql b/backend/src/db/migrations/0080_username_case_insensitive_unique.sql new file mode 100644 index 000000000..8f776728e --- /dev/null +++ b/backend/src/db/migrations/0080_username_case_insensitive_unique.sql @@ -0,0 +1,80 @@ +-- One handle, however it is cased: the case-insensitive unique index on +-- `users.username` that ADR 0042 promised and phase 4 did not ship +-- (epic #1250 phase #1268, ADR 0045). +-- +-- WHAT WAS MISSING. ADR 0042's Consequences say, of the username: "A +-- case-variant race can still land two usernames one shift key apart. +-- `users.username` is UNIQUE case-SENSITIVELY (migration 0001) while every +-- check around it is `COLLATE NOCASE`, so `Ada` and `ada` arriving together +-- pass both the pre-check and the constraint. Phase 4's case-insensitive +-- unique index closes it." Phase 4 (0077) built exactly that index for `orcid` +-- and for `email` -- and not for `username`, which still carries only the +-- exact-case UNIQUE from 0001. The sentence in the ADR has been describing a +-- file that does not exist since #1254. This is that file. +-- +-- The window is real and narrow: two concurrent `PATCH /auth/profile` calls +-- claiming `Ada` and `ada`. Each one's NOCASE pre-check sees the other's row +-- absent, and 0001's BINARY constraint then admits both, because to it they are +-- different strings. Everything downstream compares handles case-insensitively +-- -- `nemar admin approve `, the suggestion scan +-- (`services/username-assignment.ts`), the sign-in claim -- so a pair like that +-- makes two accounts one shift key apart that half the product cannot tell +-- apart. +-- +-- WHY IT CREATES CLEANLY, unlike 0077's. That migration could not simply add +-- its indexes: production held a real ORCID duplicate (rows 42/43) and a +-- case-variant email pair, so it had to add `identity_conflict`, flag the +-- non-canonical rows, and make the indexes partial around the flag. The +-- username catalogue was checked before this file was written and holds NO +-- case-variant duplicates among live rows, so there is nothing to flag and no +-- flagging pass here -- only the index. Deliberately no pass: flagging a row +-- for a username collision would also strip its claim on its email and its +-- iD, which is a much larger consequence than the problem. +-- +-- If a future catalogue DOES hold one, this file fails at CREATE and the +-- deploy stops before anything is written (it is the only statement). The +-- recovery is to rename the non-canonical handle by hand -- a username is +-- the one identifier a person can simply be asked to change -- and re-run. +-- Do not widen the predicate to hide the pair. +-- +-- WHAT THE PREDICATE EXCLUDES, matching 0077's two indexes clause for clause: +-- * NULL and blank handles. Most web/ORCID rows have none (0026), and +-- `''` is not so forgiving as NULL -- two blanks WOULD collide -- hence +-- the TRIM. +-- * Tombstones. `deleted_at IS NULL` is belt-and-suspenders here rather +-- than load-bearing: `tombstoneUser` already NULLs `username` (db/ +-- user-tombstone.ts), precisely so a deleted person's handle is free +-- again. The clause matches 0077 so the three identity indexes read the +-- same way. +-- * `identity_conflict = 1`. A row that has already lost its claim on an +-- identifier does not get to hold a second one hostage, and keeping the +-- clause identical to 0077's means the three indexes cannot disagree +-- about which rows are live claims. +-- +-- THIS IS ADDED TO, NOT INSTEAD OF, the table-level UNIQUE from 0001. Both are +-- enforced. The older one still covers rows this predicate excludes (a +-- tombstone that somehow kept a handle), and a case-variant collision now +-- fails at THIS index. SQLite reports either as +-- `UNIQUE constraint failed: users.username` -- the COLUMN, not the index name +-- -- which is what `isUsernameUniqueViolation` (services/username.ts) matches +-- on, so `PATCH /auth/profile` answers the new failure with the same typed +-- `username_taken` 409 it already answers the old one with. The route test +-- `refuses a case-variant claim the pre-check cannot see` provokes it for +-- real rather than trusting that sentence. +-- +-- NO PATTERN OF ANY KIND, so D1's 50-character LIKE/GLOB cap (0077's +-- statement 2, and the reason scripts/d1-migration-check.ts exists) cannot +-- apply. One idempotent statement: re-running the file is a no-op. +-- +-- bun run migrations:d1-check +-- +-- replays every migration through `wrangler d1 execute --local` and diffs the +-- catalogue against bun:sqlite's; this file was added to that replay before +-- being committed. + +CREATE UNIQUE INDEX IF NOT EXISTS idx_users_username_live_unique + ON users(username COLLATE NOCASE) + WHERE username IS NOT NULL + AND TRIM(username) <> '' + AND deleted_at IS NULL + AND identity_conflict = 0; diff --git a/backend/src/routes/admin/user-usernames.ts b/backend/src/routes/admin/user-usernames.ts index 42d79e22e..03aa6ca4c 100644 --- a/backend/src/routes/admin/user-usernames.ts +++ b/backend/src/routes/admin/user-usernames.ts @@ -39,7 +39,12 @@ import type { import { auditLogStatement } from "../../db/audit-log"; import { issueEmailVerificationCode } from "../../services/email-verification"; import { fetchOrcidName, orcidPubBase } from "../../services/orcid-auth"; -import { pickAvailableUsername, suggestUsername } from "../../services/username"; +import { suggestUsername } from "../../services/username"; +import { + pickUsernameForName, + recordUsernameAssignment, + usernameClaimStatement, +} from "../../services/username-assignment"; import type { Bindings } from "../../types/bindings"; import type { AdminRouter } from "./shared"; @@ -125,37 +130,6 @@ const REMAINING_SQL = `SELECT COUNT(*) as n WHERE deleted_at IS NULL AND (username IS NULL OR TRIM(username) = '')`; -/** - * Usernames that would collide with `base` or any of its `-N` variants. - * - * Deleted rows are NOT excluded: `users.username` is UNIQUE across the whole - * table, so a row this query cannot see is still a row the write would collide - * with. (The tombstone nulls the column, so in practice this returns live rows - * only -- the predicate is written for the constraint, not for today's data.) - */ -const TAKEN_SQL = `SELECT username FROM users - WHERE username = ? COLLATE NOCASE OR username LIKE ? ESCAPE '\\'`; - -/** - * Claim `username` for `id`, but only while the row still has none and the - * name is still free. - * - * The NOT EXISTS arm makes the check and the write ONE statement, which SQLite - * and D1 execute atomically -- so a name that was free when the batch scanned - * it and is taken by the time it writes produces `changes = 0` (reported as - * `conflict`) rather than a UNIQUE error or, worse, a stolen handle. The - * username predicate on the row itself is the same idea for the other - * direction: a row that gained a username between the scan and here keeps it. - */ -const CLAIM_SQL = `UPDATE users - SET username = ?, updated_at = datetime('now') - WHERE id = ? - AND deleted_at IS NULL - AND (username IS NULL OR TRIM(username) = '') - AND NOT EXISTS ( - SELECT 1 FROM users u2 WHERE u2.username = ? COLLATE NOCASE AND u2.id != ? - )`; - /** * Issue one verify-your-email code and classify the outcome. * @@ -272,6 +246,10 @@ export function registerUserUsernameRoutes(admin: AdminRouter): void { } } + // The scan-and-pick is `pickUsernameForName`, shared with the sign-in path + // (#1268, ADR 0045) so both routes suffix collisions the same way. The + // suggestion is recomputed here only to tell `single_name` from `no_name`, + // which is a distinction this report makes and that one has no use for. const base = suggestUsername(given, family); if (!base) { // Two different facts, kept apart because they ask different things of @@ -292,21 +270,20 @@ export function registerUserUsernameRoutes(admin: AdminRouter): void { continue; } - const takenRows = await db - .prepare(TAKEN_SQL) - .bind(base, `${base.replace(/[%_\\]/g, "\\$&")}-%`) - .all<{ username: string | null }>(); - const taken = (takenRows.results ?? []) - .map((r) => r.username) - .filter((u): u is string => typeof u === "string"); - - const username = pickAvailableUsername(base, [...taken, ...reserved]); - if (!username) { + const pick = await pickUsernameForName(db, given, family, reserved); + if (pick.status !== "ok") { + // `no_base` is unreachable here -- the `if (!base)` arm above returned + // for exactly that -- so this is `exhausted`, and it is reported as + // `exhausted` rather than as `conflict`. The two used to be one value + // and they ask opposite things: a conflict is retry-safe by definition + // (the next run picks the next suffix), while a saturated base gives + // the same answer every run forever and needs a human to choose a + // handle, like `single_name`. results.push({ id: user.id, email: user.email, orcid: user.orcid, - outcome: "conflict", + outcome: "exhausted", given_name: given, family_name: family, verify: "not_attempted", @@ -314,6 +291,7 @@ export function registerUserUsernameRoutes(admin: AdminRouter): void { }); continue; } + const username = pick.username; reserved.add(username.toLowerCase()); if (!apply) { @@ -330,7 +308,7 @@ export function registerUserUsernameRoutes(admin: AdminRouter): void { continue; } - const claimed = await db.prepare(CLAIM_SQL).bind(username, user.id, username, user.id).run(); + const claimed = await usernameClaimStatement(db, user.id, username).run(); if ((claimed.meta?.changes ?? 0) === 0) { results.push({ id: user.id, @@ -360,6 +338,13 @@ export function registerUserUsernameRoutes(admin: AdminRouter): void { // synthetic test target. // Never fatal: the username IS written, and a failed message is picked // up by the verify-retry pass on the next run. + // + // The per-row audit row goes in first, so the sweep's assignments and the + // sign-in path's carry the same `username_auto_assigned` action and are + // told apart by `details.source` rather than by which endpoint ran (the + // batch summary below is a separate, coarser record). + await recordUsernameAssignment(db, user.id, username, "admin_backfill"); + const verify: BackfillVerifyOutcome = user.email_verified === 1 ? "already_verified" @@ -427,6 +412,9 @@ export function registerUserUsernameRoutes(admin: AdminRouter): void { no_name: results.filter((r) => r.outcome === "no_name").length, lookup_failed: results.filter((r) => r.outcome === "lookup_failed").length, conflict: results.filter((r) => r.outcome === "conflict").length, + // Counted apart from `conflict` because it is the one outcome in this + // report that retrying cannot change. + exhausted: results.filter((r) => r.outcome === "exhausted").length, // Counted across BOTH passes: an operator reading the summary wants to // know how many people were mailed and how many were not, not which loop // tried. A failure used to appear in neither the summary nor the exit diff --git a/backend/src/routes/auth-orcid.ts b/backend/src/routes/auth-orcid.ts index 9a3c3f981..31dce0940 100644 --- a/backend/src/routes/auth-orcid.ts +++ b/backend/src/routes/auth-orcid.ts @@ -106,6 +106,7 @@ import { verifyPending, } from "../services/orcid-auth"; import { profileRefusal } from "../services/profile"; +import { type UsernameAssignmentSource, autoAssignUsername } from "../services/username-assignment"; import { buildSessionCookie, isAllowedOrigin, @@ -301,6 +302,30 @@ async function refreshUserName(env: Bindings, userId: number, orcid: string): Pr } } +/** + * Refresh the name from ORCID, then give the account a username if it still has + * none (#1268, ADR 0045). + * + * ORDER IS THE WHOLE POINT. The ADR 0042 suggestion is first initial + family + * name, and on both of these paths the name arrives from ORCID rather than from + * the row: a brand-new sign-up's columns are NULL at INSERT time, and a + * returning sign-in may be the moment a previously name-less record grows one. + * Assigning before the refresh would therefore skip exactly the accounts this + * exists for. + * + * Runs behind the response like the refresh it wraps -- a public-record fetch + * is not something a sign-in should wait on -- and, like it, never throws. + */ +async function refreshNameThenAssignUsername( + env: Bindings, + userId: number, + orcid: string, + source: UsernameAssignmentSource, +): Promise { + await refreshUserName(env, userId, orcid); + await autoAssignUsername(env.DB, userId, source); +} + /** * Whether a write failed because another account claimed the iD in the moment * between the check and the write. @@ -909,7 +934,7 @@ authOrcidRoutes.get("/orcid/callback", webSessionMiddleware, async (c) => { "orcid", ); await touchIdentityLogin(c.env, orcid); - afterResponse(c, refreshUserName(c.env, user.id, orcid)); + afterResponse(c, refreshNameThenAssignUsername(c.env, user.id, orcid, "orcid_signin")); const session = buildSessionCookie(cookieIdRaw, { domain, maxAgeSeconds }); const next = state.next !== "/" ? state.next : "/dashboard"; return redirect(`${frontend}${next}`, [clearState, session]); @@ -1106,8 +1131,12 @@ authOrcidRoutes.post("/orcid/finalize", zValidator("json", finalizeSchema), asyn c.header("Set-Cookie", buildSessionCookie(cookieIdRaw, { domain, maxAgeSeconds }), { append: true, }); - // Canonical name from ORCID, after the response (best-effort; never blocks signup). - afterResponse(c, refreshUserName(c.env, userId, orcid)); + // Canonical name from ORCID, after the response (best-effort; never blocks + // signup), and then the username derived from it (#1268, ADR 0045): the + // INSERT above leaves `username` NULL by design, and onboarding is where + // the person is asked to confirm it -- but abandoning onboarding must not + // leave the account handle-less forever. + afterResponse(c, refreshNameThenAssignUsername(c.env, userId, orcid, "orcid_signup")); // Mail the first verification code to the address just collected. The // account exists and is signed in either way: an undeliverable code diff --git a/backend/src/routes/auth-web.ts b/backend/src/routes/auth-web.ts index 554d218be..a2d44c799 100644 --- a/backend/src/routes/auth-web.ts +++ b/backend/src/routes/auth-web.ts @@ -46,7 +46,9 @@ import { zValidator } from "@hono/zod-validator"; import { Hono } from "hono"; import { z } from "zod"; +import type { AccountStatus } from "../../../shared/contract/user.js"; import { auditLogStatement } from "../db/audit-log"; +import { flag } from "../db/flag"; import { timingSafeEqual } from "../lib/constant-time"; import { resolveActingAccount } from "../middleware/auth"; import { webSessionMiddleware } from "../middleware/webSession"; @@ -88,11 +90,18 @@ import { normalizeProfilePatch, profileRefusal, } from "../services/profile"; +import { profileGapsForRow } from "../services/profile-gaps"; import { isUsernameUniqueViolation, pickAvailableUsername, suggestUsername, } from "../services/username"; +import { + REREAD_USERNAME_SQL, + pickUsernameForName, + recordUsernameAssignment, + usernameClaimStatement, +} from "../services/username-assignment"; import { buildClearedSessionCookie, buildSessionCookie, @@ -152,7 +161,9 @@ function publicUser(row: { id: number; email: string; role: UserRole | null; - status: string; + // The COLUMN's vocabulary; `userStatusForDashboard` below is what turns it + // into the wire's (shared/contract/user.ts). + status: AccountStatus; email_verified: boolean; given_name: string | null; family_name: string | null; @@ -164,6 +175,7 @@ function publicUser(row: { affiliation: string | null; service_access: boolean; username: string | null; + username_auto_assigned: boolean; service_access_granted_at: string | null; upload_access_requested_at: string | null; }) { @@ -201,6 +213,17 @@ function publicUser(row: { // retry logic and the admin queue, not something to render on a profile. service_access_granted_at: row.service_access_granted_at, upload_access_requested_at: row.upload_access_requested_at, + // What this account is still missing, and what each absence blocks (#1268, + // ADR 0045). Computed from the SAME matrix the CLI's /users/me reports and + // the upload-access request refuses against, so the dashboard nudge, the + // terminal and the refusal cannot name three different sets of fields. + // `row.status` is the INTERNAL status, not the two-state value above it: + // "pending" is what the unverified tier is spelled as in the matrix. + profile_gaps: profileGapsForRow(row), + // Whether the username above was derived rather than chosen, so onboarding + // and Settings can offer to change it once without making that offer to + // someone who typed their own (#1268, ADR 0045). + username_auto_assigned: row.username_auto_assigned, }; } @@ -444,7 +467,8 @@ authWebRoutes.post("/code/verify", zValidator("json", verifySchema), async (c) = `SELECT id, email, role, status, email_verified, given_name, family_name, orcid, orcid_verified, github_username, city, country, affiliation, service_access, - username, service_access_granted_at, upload_access_requested_at + username, username_auto_assigned, + service_access_granted_at, upload_access_requested_at FROM users WHERE email = ? COLLATE NOCASE AND deleted_at IS NULL LIMIT 1`, ) .bind(email) @@ -452,7 +476,9 @@ authWebRoutes.post("/code/verify", zValidator("json", verifySchema), async (c) = id: number; email: string; role: string | null; - status: string; + // Closed by migration 0001's CHECK constraint, so this is a claim the + // database enforces rather than an assumption (shared/contract/user.ts). + status: AccountStatus; // NOT NULL DEFAULT 0 in D1 (0001), so plain number. email_verified: number; given_name: string | null; @@ -467,6 +493,8 @@ authWebRoutes.post("/code/verify", zValidator("json", verifySchema), async (c) = // NOT NULL DEFAULT 0 in D1 (0062), so plain number. service_access: number; username: string | null; + // NOT NULL DEFAULT 0 in D1 (0079), so plain number. + username_auto_assigned: number; service_access_granted_at: string | null; upload_access_requested_at: string | null; }>(); @@ -509,11 +537,61 @@ authWebRoutes.post("/code/verify", zValidator("json", verifySchema), async (c) = ip, "email_code", ); + + // A web account that has never been through onboarding holds + // `username = NULL` (migration 0026), and #1268 stops that outliving the + // sign-in: if the row has a usable name, the ADR 0042 suggestion is claimed + // HERE, in the sign-in's own transaction, so the account is never signed in + // without a handle. Planned before the batch because picking one is a read, + // and the claim itself cannot throw (see CLAIM_USERNAME_SQL) -- a lost race + // reports `changes = 0` and leaves the sign-in untouched. + // + // No name, no assignment: onboarding asks, and nothing is invented from the + // email local part (ADR 0042). + // + // THE SCAN IS WRAPPED, and it has to be. `pickUsernameForName` reads D1 + // live, and by the time it runs the code has ALREADY been consumed by the + // conditional UPDATE above -- but the restore-the-code handler is the catch + // on the batch BELOW, not this one. A transient failure escaping here would + // reach the route's outer catch, answer a generic 500, and burn a code the + // user typed correctly. Assigning a username is a nudge and must never + // block a login, which is the contract every docstring in + // services/username-assignment.ts states; so a failed scan signs the user + // in with no assignment, and the sweep (or the next sign-in) picks the row + // up. + let claimed: string | null = null; + if ((userRow.username ?? "").trim() === "") { + try { + const pick = await pickUsernameForName(db, userRow.given_name, userRow.family_name); + if (pick.status === "ok") { + claimed = pick.username; + } else if (pick.status === "exhausted") { + // Nobody would otherwise see this: the account simply stays NULL and + // onboarding asks. Same sentence `autoAssignUsername` logs on the two + // ORCID doors, because it is the same operational fact. + console.warn( + `[username-assignment] every variant of "${pick.base}" is taken; user ${userRow.id} keeps a NULL username`, + ); + } + } catch (pickErr) { + console.error( + `[auth-web] /code/verify: could not pick a username for user id=${userRow.id}; signing in without one`, + pickErr, + ); + } + } + let promoted: boolean; + let usernameLanded = false; try { - ({ promoted } = await applyEmailVerification(db, userRow.id, "code_signin", [ + const applied = await applyEmailVerification(db, userRow.id, "code_signin", [ prepared.statement, - ])); + ...(claimed ? [usernameClaimStatement(db, userRow.id, claimed)] : []), + ]); + promoted = applied.promoted; + // `extra` mirrors the statements above in order: the session insert, then + // the claim when there was one. + usernameLanded = claimed !== null && (applied.extra[1]?.meta?.changes ?? 0) > 0; } catch (writeErr) { // Distinct from the generic 500 below, and distinct in the log: the // operator needs the account id to see what state it is in. Nothing in @@ -534,6 +612,46 @@ authWebRoutes.post("/code/verify", zValidator("json", verifySchema), async (c) = ); } + if (usernameLanded && claimed) { + await recordUsernameAssignment(db, userRow.id, claimed, "code_signin"); + } + + // A claim that did NOT land has two causes, and one of them makes the row + // this request read at the top stale: either somebody else took the handle + // (the row is unchanged, and `userRow` is still right), or this same + // account gained a username concurrently -- a `PATCH /auth/profile` racing + // the sign-in -- in which case `userRow.username` is the pre-race NULL and + // reporting it would tell the dashboard the account has no handle a moment + // after it got one. One extra read, on a path that is already rare. + // + // Non-fatal like everything else about the assignment: the sign-in has + // COMMITTED by here, and letting a failed read reach the outer catch would + // answer 500 and withhold the cookie for a session that exists. + let currentUsername = userRow.username; + let currentAutoAssigned = flag(userRow.username_auto_assigned); + if (claimed && !usernameLanded) { + try { + const fresh = await db + .prepare(REREAD_USERNAME_SQL) + .bind(userRow.id) + .first<{ username: string | null; username_auto_assigned: number }>(); + if (fresh) { + currentUsername = fresh.username; + currentAutoAssigned = flag(fresh.username_auto_assigned); + } + } catch (rereadErr) { + // Keeps the pre-race `userRow` values on the response rather than + // failing the sign-in over a read that is already a nudge's nudge: the + // client's very next `GET /auth/me` reads the row fresh, so a stale + // value here is at most one round trip old, and blocking a sign-in + // over it would be the wrong trade. + console.error( + `[auth-web] /code/verify: could not re-read the username of user id=${userRow.id} after a lost claim; reporting the row as read`, + rereadErr, + ); + } + } + const user = { id: userRow.id, email: userRow.email, @@ -549,7 +667,11 @@ authWebRoutes.post("/code/verify", zValidator("json", verifySchema), async (c) = country: userRow.country, affiliation: userRow.affiliation, service_access: userRow.service_access === 1, - username: userRow.username, + // The row was read before the claim, so neither half is `userRow`'s own + // value: a landed claim reports what it wrote, and a lost one reports the + // re-read above. + username: usernameLanded ? claimed : currentUsername, + username_auto_assigned: usernameLanded || currentAutoAssigned, service_access_granted_at: userRow.service_access_granted_at, upload_access_requested_at: userRow.upload_access_requested_at, }; @@ -897,6 +1019,12 @@ authWebRoutes.patch( if (patch.username !== undefined) { sets.push("username = ?"); binds.push(patch.username); + // The handle is now one the person typed, so the "we chose this from + // your name, change it if you like" offer is spent (#1268, ADR 0045). + // Reached only for a REAL change: re-submitting the current username is + // dropped from the patch above, so a routine full-form save from an + // auto-named account does not silently retire the offer. + sets.push("username_auto_assigned = 0"); } if (patch.given_name !== undefined) { sets.push("given_name = ?"); @@ -1113,7 +1241,8 @@ async function fetchPublicUserById( `SELECT id, email, role, status, email_verified, given_name, family_name, orcid, orcid_verified, github_username, city, country, affiliation, service_access, - username, service_access_granted_at, upload_access_requested_at + username, username_auto_assigned, + service_access_granted_at, upload_access_requested_at FROM users WHERE id = ? AND deleted_at IS NULL LIMIT 1`, ) .bind(userId) @@ -1121,7 +1250,8 @@ async function fetchPublicUserById( id: number; email: string; role: string | null; - status: string; + // Closed by migration 0001's CHECK constraint (shared/contract/user.ts). + status: AccountStatus; // NOT NULL DEFAULT 0 in D1 (0001), so plain number. email_verified: number; given_name: string | null; @@ -1136,6 +1266,8 @@ async function fetchPublicUserById( // NOT NULL DEFAULT 0 in D1 (0062), so plain number. service_access: number; username: string | null; + // NOT NULL DEFAULT 0 in D1 (0079), so plain number. + username_auto_assigned: number; service_access_granted_at: string | null; upload_access_requested_at: string | null; }>(); @@ -1156,6 +1288,7 @@ async function fetchPublicUserById( affiliation: row.affiliation, service_access: row.service_access === 1, username: row.username, + username_auto_assigned: flag(row.username_auto_assigned), service_access_granted_at: row.service_access_granted_at, upload_access_requested_at: row.upload_access_requested_at, }); diff --git a/backend/src/routes/users.ts b/backend/src/routes/users.ts index 9779295f0..d05458030 100644 --- a/backend/src/routes/users.ts +++ b/backend/src/routes/users.ts @@ -14,7 +14,9 @@ import { zValidator } from "@hono/zod-validator"; import { Hono } from "hono"; import type { Context } from "hono"; import { z } from "zod"; +import type { AccountStatus } from "../../../shared/contract/user.js"; import { auditLogStatement } from "../db/audit-log"; +import { flag } from "../db/flag"; import { authMiddleware } from "../middleware/auth"; import { getAdminEmailsForCategory, @@ -23,6 +25,7 @@ import { } from "../services/email"; import { validateGitHubUsername } from "../services/github"; import { getDatasetsToken } from "../services/github-auth"; +import { profileGapsForRow } from "../services/profile-gaps"; import { ALREADY_APPROVED_REFUSAL, GITHUB_UNAVAILABLE_REFUSAL, @@ -56,6 +59,11 @@ userRoutes.get("/me", async (c) => { orcid_verified, given_name, family_name, + username, + github_username, + city, + country, + username_auto_assigned, sandbox_completed, sandbox_completed_at, sandbox_dataset_id, @@ -67,13 +75,23 @@ userRoutes.get("/me", async (c) => { ) .bind(user.id, user.id) .first<{ - status: string; + // Closed by migration 0001's CHECK constraint (shared/contract/user.ts). + status: AccountStatus; created_at: string; approved_at: string; email_verified: number; orcid_verified: number; given_name: string | null; family_name: string | null; + // Read from the ROW rather than reused from the credential (#1268): the + // gap list must describe the account as it is now, and `AuthUser` types + // both of these non-null because a token-bearing CLI account has them -- + // which is exactly the assumption a NULL-username web row breaks. + username: string | null; + github_username: string | null; + city: string | null; + country: string | null; + username_auto_assigned: number; sandbox_completed: number; sandbox_completed_at: string | null; sandbox_dataset_id: string | null; @@ -125,6 +143,13 @@ userRoutes.get("/me", async (c) => { sandbox_dataset_id: userDetails?.sandbox_dataset_id, // Tiered access (website ADR 0010): admin-granted permission to upload/compute. service_access: !!userDetails?.service_access, + // What this account is still missing, computed by the SAME function the + // upload-access preconditions use (#1268, ADR 0045) -- so `nemar auth + // status` can never list a different set of fields from the one a request + // is refused for. An empty array is a real answer; the key is absent only + // when the row itself could not be read. + profile_gaps: userDetails ? profileGapsForRow(userDetails) : undefined, + username_auto_assigned: flag(userDetails?.username_auto_assigned), }, token: tokenInfo ? { diff --git a/backend/src/services/email-verification.ts b/backend/src/services/email-verification.ts index 87973ce56..853285f06 100644 --- a/backend/src/services/email-verification.ts +++ b/backend/src/services/email-verification.ts @@ -149,6 +149,11 @@ export async function issueEmailVerificationCode( * roads to `verified` stay distinguishable after the fact. */ export type EmailVerificationRoute = "verify_endpoint" | "code_signin"; +/** Just the half of a D1 batch result anyone here reads. D1's own types do not + * narrow `batch()` results per statement, so the cast is unavoidable; naming + * it once keeps it from being spelled three different ways. */ +type BatchMeta = { meta?: { changes?: number } }; + /** * Record that an account proved its inbox, promote it out of `pending`, and * land any statements the CALLER must not outlive that promotion (the sign-in @@ -185,7 +190,7 @@ export async function applyEmailVerification( userId: number, via: EmailVerificationRoute, alsoInTransaction: D1PreparedStatement[] = [], -): Promise<{ promoted: boolean }> { +): Promise<{ promoted: boolean; extra: BatchMeta[] }> { const results = await db.batch([ db .prepare( @@ -207,8 +212,12 @@ export async function applyEmailVerification( ...alsoInTransaction, ]); - const promoted = ((results[0] as { meta?: { changes?: number } })?.meta?.changes ?? 0) > 0; - if (!promoted) return { promoted: false }; + // The caller's own statements, in the order it passed them, so it can read + // `changes` off a conditional UPDATE it batched in here (#1268: the sign-in + // path claims a username this way, and has to know whether it landed). + const extra = results.slice(2) as BatchMeta[]; + const promoted = ((results[0] as BatchMeta)?.meta?.changes ?? 0) > 0; + if (!promoted) return { promoted: false, extra }; try { await auditLogStatement(db, { @@ -224,7 +233,7 @@ export async function applyEmailVerification( err, ); } - return { promoted: true }; + return { promoted: true, extra }; } /** diff --git a/backend/src/services/profile-gaps.ts b/backend/src/services/profile-gaps.ts new file mode 100644 index 000000000..4852f9ad9 --- /dev/null +++ b/backend/src/services/profile-gaps.ts @@ -0,0 +1,54 @@ +/** + * The one adapter between a `users` row and the shared gap matrix (#1268, + * ADR 0045). + * + * The rules live in `shared/contract/profile-gaps.ts`, which is transcribed by + * the website and must therefore stay free of anything D1-shaped. What is + * D1-shaped is exactly this: `email_verified` and `orcid_verified` are 0/1 + * INTEGERs on the row and booleans on the wire, and three different SELECTs + * feed the same computation (`GET /users/me`, `GET /auth/me` via `publicUser`, + * and the upload-access precondition row). Converting them at each call site is + * how one of the three ends up passing `0` where a boolean was expected and + * silently reporting every account's inbox as verified. + */ + +import { type ProfileGapEntry, computeProfileGaps } from "../../../shared/contract/profile-gaps.js"; +import type { AccountStatus } from "../../../shared/contract/user.js"; +import { flag } from "../db/flag"; + +/** + * The columns the gap rules read, as SQLite hands them over. + * + * Both flags accept `number | boolean` because `publicUser` has already + * normalised its row by the time it gets here while `/users/me` has not, and + * demanding one spelling would only add a cast at one of the two call sites. + */ +export interface ProfileGapRow { + /** The column's own vocabulary, not the dashboard's collapsed one: this is + * what SQLite hands back, and migration 0001's CHECK constraint is what + * closes the set. */ + status: AccountStatus; + email_verified: number | boolean; + orcid_verified: number | boolean; + username: string | null; + given_name: string | null; + family_name: string | null; + github_username: string | null; + city: string | null; + country: string | null; +} + +/** Compute `profile_gaps` for one account row. */ +export function profileGapsForRow(row: ProfileGapRow): ProfileGapEntry[] { + return computeProfileGaps({ + status: row.status, + email_verified: flag(row.email_verified), + orcid_verified: flag(row.orcid_verified), + username: row.username, + given_name: row.given_name, + family_name: row.family_name, + github_username: row.github_username, + city: row.city, + country: row.country, + }); +} diff --git a/backend/src/services/upload-access.ts b/backend/src/services/upload-access.ts index 298a99949..075be51c6 100644 --- a/backend/src/services/upload-access.ts +++ b/backend/src/services/upload-access.ts @@ -22,6 +22,7 @@ */ import type { z } from "zod"; +import { profileGapFields } from "../../../shared/contract/profile-gaps.js"; import { UPLOAD_ACCESS_WHY_MAX_CHARS, UPLOAD_ACCESS_WHY_MIN_CHARS, @@ -67,12 +68,6 @@ export interface UploadAccessProfile { country: string | null; } -/** Whitespace counts as absent, matching `resolveUploaderIdentity`'s trim - * (services/uploader-identity.ts): a row holding " " has no city. */ -function blank(value: string | null | undefined): boolean { - return (value ?? "").trim().length === 0; -} - /** * Check everything that can be decided from the account row plus the submitted * text. Returns the refusal to send, or null when the request may proceed to @@ -93,6 +88,14 @@ function blank(value: string | null | undefined): boolean { * GitHub existence is checked by the route after this returns null, because it * costs a GitHub API call and there is no point spending one on a request that * is already refused. + * + * WHERE THE FIELD LIST COMES FROM (#1268, ADR 0045). `missing` is no longer + * assembled here: it is `profileGapFields` over the same row, the function + * `/users/me` and `/auth/me` compute their `profile_gaps` with. That is the + * whole point of phase 8 — a refusal naming `city` and a status command that + * never mentioned it were two implementations of one rule, and one of them was + * always going to drift. What stays here is the ORDER OF THE THREE ANSWERS, + * which is a property of this endpoint and of nothing else. */ export function checkUploadAccessRequest( profile: UploadAccessProfile, @@ -107,7 +110,21 @@ export function checkUploadAccessRequest( }; } - if (profile.email_verified !== 1) { + // The gap list is computed once and then split at `email_verified`, rather + // than computed twice: the inbox has its own refusal code and its own + // endpoint, and it comes FIRST in matrix order, so the remaining entries are + // exactly the profile half in exactly the order the clients render. + const gaps = profileGapFields({ + email_verified: profile.email_verified === 1, + username: profile.username, + given_name: profile.given_name, + family_name: profile.family_name, + github_username: profile.github_username, + city: profile.city, + country: profile.country, + }); + + if (gaps.includes("email_verified")) { return { error: "email_not_verified", message: @@ -116,13 +133,7 @@ export function checkUploadAccessRequest( }; } - const missing: string[] = []; - if (blank(profile.username)) missing.push("username"); - if (blank(profile.given_name)) missing.push("given_name"); - if (blank(profile.family_name)) missing.push("family_name"); - if (blank(profile.github_username)) missing.push("github_username"); - if (blank(profile.city)) missing.push("city"); - if (blank(profile.country)) missing.push("country"); + const missing = gaps; if (missing.length > 0) { return { diff --git a/backend/src/services/username-assignment.ts b/backend/src/services/username-assignment.ts new file mode 100644 index 000000000..8cf95b86c --- /dev/null +++ b/backend/src/services/username-assignment.ts @@ -0,0 +1,274 @@ +/** + * Give a username-less account the ADR 0042 suggestion, at sign-in (#1268, + * ADR 0045). + * + * ADR 0042 built the BATCH path: `POST /admin/users/backfill-usernames` walks + * the rows whose `username IS NULL` and names them. It has not been run yet — + * production holds 18 such rows as of 2026-09-06 and the first run is scheduled + * for after this epic deploys — and when it is, it closes the accounts that + * exist at that moment and nothing after them: a web sign-up whose owner + * abandons onboarding lands right back in the same state, and stays there until + * an operator remembers to run the sweep again. This is the LAZY path, and + * between the two there is no way for an account to keep browsing NEMAR with no + * handle: the sweep catches the ones already there, sign-in catches the ones + * arriving. + * + * THE RULE, in full: + * - the row's username is NULL or blank, and + * - the row has a given AND a family name (the ADR 0042 suggestion is first + * initial + family name; a name that folds to nothing in ASCII counts as + * absent), and + * - some variant of that suggestion is free. + * Anything else leaves the column NULL and says why. Nothing is ever invented + * from the email local part — ADR 0042 is explicit that a handle the person + * never chose and cannot recognise as theirs is worse than none. + * + * WHY THE CLAIM CANNOT THROW, which is what makes it safe to put inside a + * sign-in transaction: {@link CLAIM_USERNAME_SQL} is ONE statement whose + * NOT EXISTS arm is evaluated with the write, so a name that was free when this + * module scanned it and is taken by the time it writes produces `changes = 0`, + * not a UNIQUE error. A sign-in that batches it therefore either lands whole or + * lands whole minus the username — never rolls back over a handle. + * + * The audit row carries the event (`username_auto_assigned`); the 0079 column + * carries the state. Both, because the copy that offers "we picked this, change + * it if you like" is rendered on every page load and cannot afford to scan an + * audit log for it. + */ + +import { auditLogStatement } from "../db/audit-log"; +import { pickAvailableUsername, suggestUsername } from "./username"; + +/** + * Where the assignment happened. Written into the audit row's details so an + * operator can tell the sweep's work from a sign-in's without joining anything. + */ +export type UsernameAssignmentSource = + | "code_signin" + | "orcid_signin" + | "orcid_signup" + | "admin_backfill"; + +/** + * Claim `username` for `id`, but only while the row still has none and the name + * is still free. + * + * Lifted out of routes/admin/user-usernames.ts so the sweep and the three + * sign-in paths write the same statement — including + * `username_auto_assigned = 1`, which is the whole reason a caller must not + * hand-roll this UPDATE. + * + * The NOT EXISTS arm makes the check and the write ONE statement, which SQLite + * and D1 execute atomically, so a lost race reports `changes = 0` rather than + * raising or, worse, stealing a handle. The username predicate on the row + * itself is the same idea for the other direction: a row that gained a username + * between the scan and here keeps it. + */ +export const CLAIM_USERNAME_SQL = `UPDATE users + SET username = ?, username_auto_assigned = 1, updated_at = datetime('now') + WHERE id = ? + AND deleted_at IS NULL + AND (username IS NULL OR TRIM(username) = '') + AND NOT EXISTS ( + SELECT 1 FROM users u2 WHERE u2.username = ? COLLATE NOCASE AND u2.id != ? + )`; + +/** + * Usernames that would collide with `base` or any of its `-N` variants. + * + * Deleted rows are NOT excluded: `users.username` is UNIQUE across the whole + * table, so a row this query cannot see is still a row the write would collide + * with. The LIKE arm can over-match (`alovelace-institute` looks like a + * suffixed variant and is not one), which only ever makes the taken set larger + * and the suggestion later in the sequence, never a collision. + * + * EXPORTED so a test can point THIS statement -- and only this statement -- at + * something the database refuses, and prove that a failed scan does not take a + * sign-in down with it. A hand-copied string in the test would drift the moment + * this one is edited (.rules/testing.md). + */ +export const TAKEN_SQL = `SELECT username FROM users + WHERE username = ? COLLATE NOCASE OR username LIKE ? ESCAPE '\\'`; + +/** + * `/auth/code/verify`'s lost-claim re-read (routes/auth-web.ts). + * + * A claim that loses its race leaves the response holding the row it read + * BEFORE the race, which is wrong in exactly the direction of "this account + * has no handle" when the truth just became "it does, from somewhere else." + * This is the one-shot fix for that: re-read the row after the sign-in has + * already committed. + * + * If the re-read itself fails, the caller falls back to the pre-race values + * rather than failing the sign-in -- the client's very next `GET /auth/me` + * reads the row fresh, so a stale value here is at most one round trip old, + * and blocking a sign-in over it would be the wrong trade. + * + * EXPORTED for the same reason as {@link TAKEN_SQL}: a test needs to point + * THIS statement -- and only this statement -- at something the database + * refuses, to prove that fallback actually holds rather than throwing past + * the route's outer catch. + */ +export const REREAD_USERNAME_SQL = + "SELECT username, username_auto_assigned FROM users WHERE id = ? LIMIT 1"; + +/** What a suggestion attempt produced. `no_base` and `exhausted` are kept apart + * because they ask different things: no_base needs a name (or a human), while + * exhausted means a base exists and every variant of it is taken, which is an + * operational fact worth logging. */ +export type UsernamePick = + | { status: "ok"; username: string; base: string } + | { status: "no_base" } + | { status: "exhausted"; base: string }; + +/** + * The ADR 0042 suggestion for a name, suffixed past whatever is taken. + * + * `reserved` is for a caller handing out several usernames before any of them + * is written (the sweep's dry run): without it two rows with the same suggested + * handle would both be promised it. + */ +export async function pickUsernameForName( + db: D1Database, + givenName: string | null | undefined, + familyName: string | null | undefined, + reserved: Iterable = [], +): Promise { + const base = suggestUsername(givenName, familyName); + if (!base) return { status: "no_base" }; + + const rows = await db + .prepare(TAKEN_SQL) + .bind(base, `${base.replace(/[%_\\]/g, "\\$&")}-%`) + .all<{ username: string | null }>(); + const taken = (rows.results ?? []) + .map((r) => r.username) + .filter((u): u is string => typeof u === "string"); + + const username = pickAvailableUsername(base, [...taken, ...reserved]); + return username ? { status: "ok", username, base } : { status: "exhausted", base }; +} + +/** The account columns the decision reads. */ +interface AutoAssignRow { + username: string | null; + given_name: string | null; + family_name: string | null; +} + +/** What {@link autoAssignUsername} did, for the caller's log and for tests. + * Never an exception: this runs alongside a sign-in that has already + * succeeded, and no outcome here is worth failing one over. */ +export type UsernameAssignmentOutcome = + | { status: "assigned"; username: string } + | { status: "already_set" } + | { status: "no_name" } + | { status: "exhausted" } + | { status: "conflict" } + | { status: "unavailable" }; + +/** + * Assign this account the suggestion, if it needs one and one is available. + * + * Used by the two ORCID paths, where the name itself only lands after the + * response (`refreshUserName` is a public-record fetch and is deliberately not + * on the critical path), so this runs chained behind it rather than inside the + * sign-in write. `/auth/code/verify` does NOT call this: it has a transaction + * of its own and batches {@link CLAIM_USERNAME_SQL} into it directly, so the + * handle and the session land together. + * + * Fails soft and says so. A row that could not be read, a name that folds to + * nothing, a saturated base — each is reported, none throws, and the account is + * left exactly as it was for the sweep (or the next sign-in) to pick up. + */ +export async function autoAssignUsername( + db: D1Database, + userId: number, + source: UsernameAssignmentSource, +): Promise { + try { + const row = await db + .prepare( + "SELECT username, given_name, family_name FROM users WHERE id = ? AND deleted_at IS NULL LIMIT 1", + ) + .bind(userId) + .first(); + if (!row) return { status: "unavailable" }; + if ((row.username ?? "").trim() !== "") return { status: "already_set" }; + + const pick = await pickUsernameForName(db, row.given_name, row.family_name); + if (pick.status === "no_base") return { status: "no_name" }; + if (pick.status === "exhausted") { + // Nobody would otherwise see this: the account simply stays NULL and + // onboarding asks. A saturated base is an operational fact. + console.warn( + `[username-assignment] every variant of "${pick.base}" is taken; user ${userId} keeps a NULL username`, + ); + return { status: "exhausted" }; + } + + const claim = await usernameClaimStatement(db, userId, pick.username).run(); + if ((claim.meta?.changes ?? 0) === 0) { + // The row gained a username, or the handle was claimed, between the scan + // and the write. Not an error: the next sign-in picks the next suffix. + return { status: "conflict" }; + } + await recordUsernameAssignment(db, userId, pick.username, source); + return { status: "assigned", username: pick.username }; + } catch (err) { + // Never fatal. The caller is a sign-in that has already succeeded, and an + // account with no username is the state this function exists to improve on, + // not one it may fail a login over. + console.error(`[username-assignment] could not assign a username to user ${userId}`, err); + return { status: "unavailable" }; + } +} + +/** + * The claim on its own, for a caller that wants it inside its own transaction + * (`/auth/code/verify` batches it with the session insert so the handle and the + * session land together). + * + * Bound, not run. The caller reads `changes` to learn whether the handle + * actually landed, and only then writes the audit row with + * {@link recordUsernameAssignment} — the two are deliberately NOT batched + * together, because a claim that loses its race would otherwise commit an audit + * row saying a username was assigned when none was. + */ +export function usernameClaimStatement( + db: D1Database, + userId: number, + username: string, +): D1PreparedStatement { + return db.prepare(CLAIM_USERNAME_SQL).bind(username, userId, username, userId); +} + +/** + * Record that a username was assigned rather than chosen. + * + * Outside the claim's transaction and non-fatal, for the reason + * `upload_access_requested` records: the username HAS landed by the time this + * runs, and rolling a sign-in back over a failed audit insert is the worse + * outcome. Loud in the log instead. + */ +export async function recordUsernameAssignment( + db: D1Database, + userId: number, + username: string, + source: UsernameAssignmentSource, +): Promise { + try { + await auditLogStatement(db, { + userId, + action: "username_auto_assigned", + resourceType: "user", + resourceId: String(userId), + details: JSON.stringify({ username, source }), + }).run(); + } catch (err) { + console.error( + `AUDIT GAP: username_auto_assigned row not written for id=${userId} (the username "${username}" DID land):`, + err, + ); + } +} diff --git a/backend/src/services/username.ts b/backend/src/services/username.ts index bca7ebdc7..93b649d4f 100644 --- a/backend/src/services/username.ts +++ b/backend/src/services/username.ts @@ -127,10 +127,12 @@ export function suggestUsername( * suffix will solve, and an unbounded loop inside a Worker request is worse * than an honest "could not pick one". */ +export const USERNAME_SUFFIX_LIMIT = 50; + export function pickAvailableUsername( base: string, taken: Iterable, - limit = 50, + limit = USERNAME_SUFFIX_LIMIT, ): string | null { const used = new Set(); for (const t of taken) used.add(t.trim().toLowerCase()); diff --git a/backend/src/services/web-session.ts b/backend/src/services/web-session.ts index 0a902a18e..f72757463 100644 --- a/backend/src/services/web-session.ts +++ b/backend/src/services/web-session.ts @@ -18,6 +18,8 @@ * brute-forced offline. */ +import type { AccountStatus } from "../../../shared/contract/user.js"; +import { flag } from "../db/flag"; import { type Bindings, type UserRole, parseRole } from "../types/bindings"; export const COOKIE_NAME = "nemar_session"; @@ -137,7 +139,10 @@ export interface WebSessionUser { * means the role column held an unrecognised value; treat as no * role rather than guess. */ role: UserRole | null; - status: string; + /** The COLUMN's vocabulary (shared/contract/user.ts), closed by migration + * 0001's CHECK constraint -- not the collapsed value `/auth/me` reports, + * which `publicUser` derives from this one. */ + status: AccountStatus; /** Whether the account has proved control of `email` (ADR 0040 phase 2). * Surfaced on /auth/me so the dashboard can render its verify-your-email * step, and read by /auth/email/verify to answer idempotently when there @@ -165,6 +170,12 @@ export interface WebSessionUser { * fetching it from GET /users/me separately, purely because it was absent * here (nemarOrg/website#306). */ username: string | null; + /** True when that username was DERIVED from the name rather than chosen -- + * by the ADR 0042 backfill sweep or at a web sign-in (#1268, ADR 0045) -- + * and has not been changed since. Onboarding and Settings use it to offer + * "we picked this, change it if you like", an offer that would be nonsense + * to someone who typed their own. Converted from the 0/1 column here. */ + username_auto_assigned: boolean; /** The DATES behind the two upload-access states (ADR 0042), so the * dashboard can render "granted"/"requested" as events rather than flags. */ service_access_granted_at: string | null; @@ -189,7 +200,8 @@ export async function findSessionByCookieId( u.email, u.role, u.status, u.email_verified, u.given_name, u.family_name, u.orcid, u.orcid_verified, u.github_username, u.city, u.country, u.affiliation, u.service_access, - u.username, u.service_access_granted_at, u.upload_access_requested_at + u.username, u.username_auto_assigned, + u.service_access_granted_at, u.upload_access_requested_at FROM web_sessions ws JOIN users u ON u.id = ws.user_id WHERE ws.cookie_id_hash = ? @@ -208,7 +220,8 @@ export async function findSessionByCookieId( last_used_at: string; email: string; role: string | null; - status: string; + // Closed by migration 0001's CHECK constraint (shared/contract/user.ts). + status: AccountStatus; // NOT NULL DEFAULT 0 in D1 (0001), so plain number. email_verified: number; given_name: string | null; @@ -224,6 +237,8 @@ export async function findSessionByCookieId( service_access: number; // NULL on every web/ORCID row until onboarding sets one (migration 0026). username: string | null; + // NOT NULL DEFAULT 0 in D1 (0079), so plain number. + username_auto_assigned: number; // The two dates the dashboard needs to render "granted"/"requested" as // events rather than as flags (ADR 0042; nemarOrg/website#306). service_access_granted_at: string | null; @@ -263,6 +278,7 @@ export async function findSessionByCookieId( affiliation: row.affiliation, service_access: row.service_access === 1, username: row.username, + username_auto_assigned: flag(row.username_auto_assigned), service_access_granted_at: row.service_access_granted_at, upload_access_requested_at: row.upload_access_requested_at, }, diff --git a/backend/test/backfill-usernames-route.test.ts b/backend/test/backfill-usernames-route.test.ts index b05b7e7b5..ee0782f79 100644 --- a/backend/test/backfill-usernames-route.test.ts +++ b/backend/test/backfill-usernames-route.test.ts @@ -27,6 +27,7 @@ import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:tes import { Hono } from "hono"; import { adminRoutes } from "../src/routes/admin"; import { hashApiKey } from "../src/services/token"; +import { USERNAME_SUFFIX_LIMIT } from "../src/services/username"; import type { Bindings, Variables } from "../src/types/bindings"; import { freshDb, realD1 } from "./helpers/d1"; import { withFakeResend } from "./helpers/resend"; @@ -104,6 +105,14 @@ interface CandidateOptions { email_verified?: number; } +/** An unrelated account already holding a handle, to take it out of the pool. */ +function seedTaken(username: string): void { + db.query( + `INSERT INTO users (username, email, password_hash, status, role, email_verified) + VALUES (?, ?, 'x', 'approved', 'member', 1)`, + ).run(username, `${username}@example.org`); +} + /** A username-less row, the shape the sweep exists for. */ function seedCandidate(email: string, options: CandidateOptions = {}): number { const row = { @@ -215,11 +224,39 @@ describe("backfill-usernames: apply", () => { expect(usernameOf(id)).toBe("alovelace"); expect(body.remaining).toBe(0); + // The 0079 column, which is the STATE half and the reason the sweep must go + // through `usernameClaimStatement` rather than write its own UPDATE: the + // "we chose this from your name, change it if you like" offer renders on + // every page load and cannot scan an audit log for permission to appear. + expect( + db + .query<{ username_auto_assigned: number }, [number]>( + "SELECT username_auto_assigned FROM users WHERE id = ?", + ) + .get(id)?.username_auto_assigned, + ).toBe(1); + const audit = db .query<{ details: string | null }, [string]>("SELECT details FROM audit_log WHERE action = ?") .all("usernames_backfilled"); expect(audit).toHaveLength(1); expect(JSON.parse(audit[0].details ?? "{}")).toMatchObject({ assigned: 1, user_ids: [id] }); + + // And the PER-ROW event, which is the same action the three sign-in doors + // write and is told apart from them only by `source` (#1268, ADR 0045). The + // batch row above has no `user_id` subject, so without this one an operator + // cannot ask "how did THIS account get its handle". + const perRow = db + .query<{ user_id: number; details: string | null }, [string]>( + "SELECT user_id, details FROM audit_log WHERE action = ?", + ) + .all("username_auto_assigned"); + expect(perRow).toHaveLength(1); + expect(perRow[0].user_id).toBe(id); + expect(JSON.parse(perRow[0].details ?? "{}")).toEqual({ + username: "alovelace", + source: "admin_backfill", + }); }); test("two rows sharing a suggestion get a collision suffix", async () => { @@ -276,6 +313,33 @@ describe("backfill-usernames: apply", () => { }); describe("backfill-usernames: rows it will not guess at", () => { + test("a saturated base is `exhausted`, which is not a retryable conflict", async () => { + // The suffix search is bounded, and exhausting it used to be reported as a + // `conflict` -- a value whose whole meaning is "retry, it will pick the next + // suffix", which this case never will. The bound is + // `pickAvailableUsername`'s own constant, imported rather than written down + // here so the fixture follows it instead of pinning a copy. It also proves + // the search terminates rather than looping inside a Worker request. + seedTaken("alovelace"); + for (let n = 2; n <= USERNAME_SUFFIX_LIMIT; n++) seedTaken(`alovelace-${n}`); + const id = seedCandidate("ada@nemar.test", { email_verified: 1 }); + + const body = await (await withFakeResend(() => run({ apply: true }))).json(); + + expect(body.exhausted).toBe(1); + expect(body.conflict).toBe(0); + expect(body.assigned).toBe(0); + expect(body.results[0]).toMatchObject({ + id, + outcome: "exhausted", + error: "No free variant of 'alovelace' within the suffix limit", + }); + expect(usernameOf(id)).toBeNull(); + // Still a candidate, and it will be one after every future run too. + expect(body.remaining).toBe(1); + }); + + test("a one-part name is reported, not derived from the email", async () => { // 3 of the 19 production rows are exactly this. `pemberly@nemar.test` would // make a perfectly plausible handle and is not this person's name. @@ -326,24 +390,6 @@ describe("backfill-usernames: rows it will not guess at", () => { expect(body.remaining).toBe(1); }); - test("gives up rather than looping when every variant is taken", async () => { - // The suffix search is bounded (50). Exhausting it is a `conflict`, which - // is a thing to retry, not a silent skip -- and not an unbounded loop - // inside a Worker request. - db.query( - "INSERT INTO users (username, email, password_hash, status) VALUES ('alovelace', 'u1@example.org', 'x', 'approved')", - ).run(); - for (let n = 2; n <= 50; n++) { - db.query( - "INSERT INTO users (username, email, password_hash, status) VALUES (?, ?, 'x', 'approved')", - ).run(`alovelace-${n}`, `u${n}@example.org`); - } - const id = seedCandidate("ada@nemar.test", { email_verified: 1 }); - - const body = await (await withFakeResend(() => run({ apply: true }))).json(); - expect(body.conflict).toBe(1); - expect(usernameOf(id)).toBeNull(); - }); }); describe("backfill-usernames: the verify-your-email message", () => { diff --git a/backend/test/identity-conflict-still-usable.test.ts b/backend/test/identity-conflict-still-usable.test.ts index b53e7d310..3a15f6855 100644 --- a/backend/test/identity-conflict-still-usable.test.ts +++ b/backend/test/identity-conflict-still-usable.test.ts @@ -61,8 +61,9 @@ function seed(email: string, identityBacked: boolean): number { } beforeEach(async () => { - // Every migration EXCEPT 0077, so the duplicate can be seeded pre-flag and - // flagged by the real migration rather than by the test. + // Every migration BEFORE 0077, so the duplicate can be seeded pre-flag and + // flagged by the real migration rather than by the test. The rest are + // replayed after the flag lands (below). db = new Database(":memory:"); for (const file of readdirSync(MIGRATIONS_DIR) .filter((f) => f.endsWith(".sql") && f < TARGET) @@ -73,6 +74,18 @@ beforeEach(async () => { flaggedId = seed("orphan@example.org", false); canonicalId = seed("holder@example.org", true); db.exec(readFileSync(join(MIGRATIONS_DIR, TARGET), "utf-8")); + // ...and then everything AFTER it, so the routes below run against the + // CURRENT schema rather than a 2026-09 snapshot of it. Splitting the replay + // around 0077 is what lets the duplicate be seeded pre-flag; stopping there + // was an accident of how that was written, and it broke the moment + // `GET /users/me` first selected a column a later migration adds (#1268's + // `username_auto_assigned`) -- a 500 that says nothing about identity + // conflicts, which is what this file is supposed to be about. + for (const file of readdirSync(MIGRATIONS_DIR) + .filter((f) => f.endsWith(".sql") && f > TARGET) + .sort()) { + db.exec(readFileSync(join(MIGRATIONS_DIR, file), "utf-8")); + } db.run("INSERT INTO tokens (user_id, api_key_hash, api_key_prefix) VALUES (?, ?, 'nemar_te')", [ flaggedId, diff --git a/backend/test/profile-gaps-route.test.ts b/backend/test/profile-gaps-route.test.ts new file mode 100644 index 000000000..3248e04a4 --- /dev/null +++ b/backend/test/profile-gaps-route.test.ts @@ -0,0 +1,356 @@ +/** + * `profile_gaps` on both user payloads, and its equality with the + * upload-access refusal (#1268, ADR 0045). + * + * The property this file exists to hold: for ONE account row, the list + * `GET /users/me` reports, the list `GET /auth/me` reports, and the `missing` + * array `POST /users/me/upload-access/request` refuses with are the same + * fields in the same order. Before phase 8 they were three implementations of + * one rule; they are now one function called three times, and the way to notice + * that changing is to ask all three about the same row and compare. + * + * Real engine throughout: bun:sqlite behind realD1 with every migration + * applied, real Hono dispatch through `authMiddleware` and + * `webSessionMiddleware`, real hashed API keys, real session issuance. The one + * external boundary the request path touches is GitHub's `/users/:login`, + * served by a local `Bun.serve()` through the `NEMAR_GITHUB_API_URL` override — + * and only the tests that get PAST the profile check reach it. No mocks. + */ + +import type { Database } from "bun:sqlite"; +import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:test"; +import { Hono } from "hono"; +import { computeProfileGaps } from "../../shared/contract/profile-gaps.js"; +import { userMeResponseSchema, webUserSchema } from "../../shared/contract/user.js"; +import { authWebRoutes } from "../src/routes/auth-web"; +import { userRoutes } from "../src/routes/users"; +import { hashApiKey } from "../src/services/token"; +import { checkUploadAccessRequest } from "../src/services/upload-access"; +import { issueSession } from "../src/services/web-session"; +import type { Bindings, Variables } from "../src/types/bindings"; +import { freshDb, realD1 } from "./helpers/d1"; + +const USER_KEY = "gaps-user-key-0123456789abcdef0123456789abcd"; +const ENCRYPTION_KEY = "profile-gaps-test-encryption-key-0123456789"; +const USER_EMAIL = "gaps@nemar.test"; +const ORIGIN = "https://nemar.org"; +const WHY = "Depositing our lab's 64-channel EEG study of motor imagery, 40 participants."; + +let db: Database; +let app: Hono<{ Bindings: Bindings; Variables: Variables }>; +let githubServer: ReturnType; + +beforeAll(() => { + githubServer = Bun.serve({ + port: 0, + // Every handle resolves: this suite is about the PROFILE half of the + // refusal, which is answered before the lookup is ever spent. + fetch: (req) => + new Response( + JSON.stringify({ login: new URL(req.url).pathname.split("/").pop(), id: 4242 }), + { status: 200, headers: { "Content-Type": "application/json" } }, + ), + }); + (globalThis as { NEMAR_GITHUB_API_URL?: string }).NEMAR_GITHUB_API_URL = + `http://localhost:${githubServer.port}`; +}); + +afterAll(() => { + (globalThis as { NEMAR_GITHUB_API_URL?: string }).NEMAR_GITHUB_API_URL = undefined; + githubServer.stop(true); +}); + +function env(): Bindings { + return { + DB: realD1(db), + ENVIRONMENT: "test", + ENCRYPTION_KEY, + WEB_SESSION_COOKIE_DOMAIN: "", + GITHUB_ADMIN_PAT: "test-pat-never-used-against-a-real-host", + } as unknown as Bindings; +} + +interface Overrides { + username?: string | null; + given_name?: string | null; + family_name?: string | null; + github_username?: string | null; + city?: string | null; + country?: string | null; + email_verified?: number; + orcid_verified?: number; + status?: string; + username_auto_assigned?: number; +} + +/** A complete account by default; each test blanks the columns it is about. */ +async function seedUser(overrides: Overrides = {}): Promise { + const row = { + username: "arivers", + given_name: "Ada", + family_name: "Rivers", + github_username: "adarivers", + city: "San Diego", + country: "USA", + email_verified: 1, + orcid_verified: 1, + status: "verified", + username_auto_assigned: 0, + ...overrides, + }; + db.query( + `INSERT INTO users (username, email, password_hash, status, role, email_verified, + given_name, family_name, github_username, city, country, affiliation, + orcid, orcid_verified, signup_source, service_access, + username_auto_assigned) + VALUES (?, ?, 'x', ?, 'member', ?, ?, ?, ?, ?, ?, 'Swartz Center', + '0000-0002-1825-0097', ?, 'web', 0, ?)`, + ).run( + row.username, + USER_EMAIL, + row.status, + row.email_verified, + row.given_name, + row.family_name, + row.github_username, + row.city, + row.country, + row.orcid_verified, + row.username_auto_assigned, + ); + const u = db + .query<{ id: number }, [string]>("SELECT id FROM users WHERE email = ?") + .get(USER_EMAIL); + if (!u) throw new Error("seed failed"); + db.query("INSERT INTO tokens (user_id, api_key_hash, api_key_prefix) VALUES (?, ?, ?)").run( + u.id, + await hashApiKey(USER_KEY), + USER_KEY.slice(0, 8), + ); + return u.id; +} + +function usersMe(): Promise { + return app.request("/users/me", { headers: { Authorization: `Bearer ${USER_KEY}` } }, env()); +} + +async function authMe(userId: number): Promise { + const { cookieIdRaw } = await issueSession(env(), userId, false, null, null, "orcid"); + return app.request("/auth/me", { headers: { Cookie: `nemar_session=${cookieIdRaw}` } }, env()); +} + +function requestUploadAccess(): Promise { + return app.request( + "/users/me/upload-access/request", + { + method: "POST", + headers: { Authorization: `Bearer ${USER_KEY}`, "content-type": "application/json" }, + body: JSON.stringify({ why: WHY }), + }, + env(), + ); +} + +/** The three answers about one row, as bare field-name lists. */ +async function allThree(userId: number): Promise<{ + usersMe: string[]; + authMe: string[]; + missing: string[]; +}> { + const cli = userMeResponseSchema.parse(await (await usersMe()).json()); + const web = webUserSchema.parse((await (await authMe(userId)).json()).user); + const refusal = await (await requestUploadAccess()).json(); + return { + usersMe: (cli.user.profile_gaps ?? []).map((g) => g.field), + authMe: web.profile_gaps.map((g) => g.field), + missing: refusal.missing ?? [], + }; +} + +beforeEach(() => { + db = freshDb(); + app = new Hono<{ Bindings: Bindings; Variables: Variables }>(); + app.route("/users", userRoutes); + app.route("/auth", authWebRoutes); +}); + +describe("GET /users/me carries profile_gaps", () => { + test("an incomplete account lists exactly what it is missing, in matrix order", async () => { + await seedUser({ username: null, city: null, github_username: null }); + const body = userMeResponseSchema.parse(await (await usersMe()).json()); + expect(body.user.profile_gaps?.map((g) => g.field)).toEqual([ + "username", + "github_username", + "city", + ]); + }); + + test("each entry carries blocks and set_on, not just a field name", async () => { + await seedUser({ city: null }); + const body = userMeResponseSchema.parse(await (await usersMe()).json()); + expect(body.user.profile_gaps).toEqual([ + { field: "city", blocks: ["upload_access"], set_on: ["web", "cli"] }, + ]); + }); + + test("a complete account gets an EMPTY list, which is not the same as none", async () => { + await seedUser(); + const body = userMeResponseSchema.parse(await (await usersMe()).json()); + // The CLI prints "nothing outstanding" for [] and "not checked" for absent. + expect(body.user.profile_gaps).toEqual([]); + }); + + test("a pending account cannot reach this endpoint at all", async () => { + // Not a gap-reporting decision: `authMiddleware` refuses an inactive + // account before the route runs, which is why the `email_verified` gap is + // only ever OBSERVED on /auth/me (cookie sessions admit `pending`). Pinned + // so a later widening of that gate is a deliberate change rather than a + // surprise about which payload can carry which gap. + await seedUser({ status: "pending", email_verified: 0 }); + expect((await usersMe()).status).toBe(403); + }); + + test("a verified ORCID iD makes the name halves web-only", async () => { + await seedUser({ given_name: null, orcid_verified: 1 }); + const body = userMeResponseSchema.parse(await (await usersMe()).json()); + expect(body.user.profile_gaps).toEqual([ + { field: "given_name", blocks: ["upload_access", "publication"], set_on: ["web"] }, + ]); + }); + + test("username_auto_assigned is reported, both ways round", async () => { + await seedUser({ username_auto_assigned: 1 }); + const on = userMeResponseSchema.parse(await (await usersMe()).json()); + expect(on.user.username_auto_assigned).toBe(true); + + db = freshDb(); + await seedUser({ username_auto_assigned: 0 }); + const off = userMeResponseSchema.parse(await (await usersMe()).json()); + expect(off.user.username_auto_assigned).toBe(false); + }); +}); + +describe("GET /auth/me carries the same two fields", () => { + test("the dashboard payload validates and lists the same gaps", async () => { + const id = await seedUser({ country: null, family_name: null }); + const parsed = webUserSchema.safeParse((await (await authMe(id)).json()).user); + expect(parsed.success).toBe(true); + if (!parsed.success) return; + expect(parsed.data.profile_gaps.map((g) => g.field)).toEqual(["family_name", "country"]); + expect(parsed.data.username_auto_assigned).toBe(false); + }); + + test("the two-state dashboard `status` does not change what the gaps say", async () => { + // publicUser maps `verified` -> "active" for the dashboard; the gap rules + // read the INTERNAL status, where "pending" is the unverified tier. + const id = await seedUser({ status: "pending", email_verified: 0 }); + const user = webUserSchema.parse((await (await authMe(id)).json()).user); + expect(user.status).toBe("pending"); + expect(user.profile_gaps.map((g) => g.field)).toEqual(["email_verified"]); + }); +}); + +describe("one row, three answers", () => { + test("the refusal's `missing` is exactly the profile gaps, same order", async () => { + const id = await seedUser({ username: null, city: null, country: null }); + const answers = await allThree(id); + expect(answers.missing).toEqual(["username", "city", "country"]); + expect(answers.usersMe).toEqual(answers.missing); + expect(answers.authMe).toEqual(answers.missing); + }); + + test("holds for a row missing every profile field at once", async () => { + const id = await seedUser({ + username: null, + given_name: null, + family_name: null, + github_username: null, + city: null, + country: null, + }); + const answers = await allThree(id); + expect(answers.missing).toEqual([ + "username", + "given_name", + "family_name", + "github_username", + "city", + "country", + ]); + expect(answers.usersMe).toEqual(answers.missing); + expect(answers.authMe).toEqual(answers.missing); + }); + + test("an unverified inbox is answered first, and alone", async () => { + // `email_not_verified` has its own code and its own endpoint and comes + // ahead of the profile check -- but it is the same first gap the payloads + // report, not a fourth vocabulary. Asserted on the PURE precondition + // because `authMiddleware` refuses a `pending` account before the route + // (see above), so this state is unreachable through the endpoint. + const id = await seedUser({ status: "pending", email_verified: 0, city: null }); + const refusal = checkUploadAccessRequest( + { + email_verified: 0, + username: "arivers", + given_name: "Ada", + family_name: "Rivers", + github_username: "adarivers", + city: null, + country: "USA", + }, + WHY, + ); + expect(refusal?.error).toBe("email_not_verified"); + expect(refusal?.missing).toEqual(["email_verified"]); + const web = webUserSchema.parse((await (await authMe(id)).json()).user); + expect(web.profile_gaps.map((g) => g.field)).toEqual(["email_verified", "city"]); + }); + + test("a complete row is refused for nothing and reports nothing", async () => { + const id = await seedUser(); + const answers = await allThree(id); + expect(answers.missing).toEqual([]); + expect(answers.usersMe).toEqual([]); + expect(answers.authMe).toEqual([]); + }); + + test("the pure function and the live route agree on the same row", async () => { + // Belt and braces on the refactor: the SELECT feeding the route could drop + // a column and still return a plausible list. This computes the answer + // straight from the row and compares. + await seedUser({ github_username: null, country: null }); + const row = db + .query< + { + status: string; + email_verified: number; + orcid_verified: number; + username: string | null; + given_name: string | null; + family_name: string | null; + github_username: string | null; + city: string | null; + country: string | null; + }, + [string] + >( + `SELECT status, email_verified, orcid_verified, username, given_name, family_name, + github_username, city, country FROM users WHERE email = ?`, + ) + .get(USER_EMAIL); + if (!row) throw new Error("row vanished"); + const expected = computeProfileGaps({ + status: row.status, + email_verified: row.email_verified === 1, + orcid_verified: row.orcid_verified === 1, + username: row.username, + given_name: row.given_name, + family_name: row.family_name, + github_username: row.github_username, + city: row.city, + country: row.country, + }); + const body = userMeResponseSchema.parse(await (await usersMe()).json()); + expect(body.user.profile_gaps).toEqual(expected); + }); +}); diff --git a/backend/test/profile-username-route.test.ts b/backend/test/profile-username-route.test.ts index e704ead5d..782b3c256 100644 --- a/backend/test/profile-username-route.test.ts +++ b/backend/test/profile-username-route.test.ts @@ -278,6 +278,46 @@ describe("PATCH /auth/profile: username", () => { ).toBe(1); }); + test("refuses a case-variant claim the pre-check could not see", async () => { + // What migration 0080 closes, and the reason a race test is not enough: + // there the two handlers interleave however bun schedules them, and the + // pre-check usually wins. Here the competing row is planted BETWEEN the + // pre-check and the write, deterministically, by a real SQLite trigger on + // the UPDATE the route is about to run -- the same fault-injection shape + // web-email-verification.test.ts uses to block a write. + // + // Without 0080 this passes and the table ends up holding `Ada` and `ada` + // as two accounts: 0001's UNIQUE is BINARY, so to it they are different + // strings, while `nemar admin approve `, the suggestion scan and + // the sign-in claim all compare NOCASE and cannot tell them apart. + const id = seedUser("shift-key@example.org"); + db.run( + `CREATE TRIGGER plant_a_case_variant BEFORE UPDATE OF username ON users + WHEN NEW.username = 'Ada' + BEGIN + INSERT INTO users (username, email, password_hash, status) + VALUES ('ada', 'lowercase@example.org', 'x', 'verified'); + END`, + ); + + const res = await patch(id, { username: "Ada" }); + + // The typed refusal, not a bare 500: the constraint fired, and the catch + // classified it through isUsernameUniqueViolation. + expect(res.status).toBe(409); + expect((await res.json()).error).toBe("username_taken"); + // The batch is one transaction, so neither the claim nor the planted row + // survives -- the account is exactly as it was. + expect(usernameOf(id)).toBeNull(); + expect( + db + .query<{ n: number }, [string]>( + "SELECT COUNT(*) as n FROM users WHERE username = ? COLLATE NOCASE", + ) + .get("ada")?.n, + ).toBe(0); + }); + test("the UNIQUE-violation classifier recognises a real constraint error", async () => { // The safety net in the PATCH's catch block, fed the ACTUAL error bun:sqlite // raises for this table rather than a hand-written string -- a copy would diff --git a/backend/test/username-auto-assign-route.test.ts b/backend/test/username-auto-assign-route.test.ts new file mode 100644 index 000000000..dd610f9f7 --- /dev/null +++ b/backend/test/username-auto-assign-route.test.ts @@ -0,0 +1,688 @@ +/** + * Username assignment at web sign-in (#1268, ADR 0045). + * + * ADR 0042 built the batch path: an admin sweep names the accounts whose + * `username IS NULL`. It has not been run yet, and when it is it closes the + * accounts that exist at that moment and nothing after them — a web sign-up + * whose owner abandons onboarding lands right back in that state. + * Phase 8 adds the lazy path at the three doors a web account comes through, + * and this file drives all three for real: + * + * POST /auth/code/verify the emailed-code sign-in (assignment batched + * into the sign-in's own transaction) + * POST /auth/orcid/finalize the ORCID sign-up + * GET /auth/orcid/callback the ORCID sign-in of an already-linked account + * + * The two ORCID paths assign AFTER the response, chained behind the ORCID name + * refresh they depend on, so those tests wait for the row rather than for the + * response — a real wait on real async work, not a mock of it. + * + * Real engine throughout: bun:sqlite behind realD1 with every migration + * applied, real Hono dispatch, real session issuance, the production + * `hashAuthCode`, and local `Bun.serve()` instances standing in for ORCID's + * token endpoint and public record API. No mocks. + */ + +import type { Database } from "bun:sqlite"; +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, test } from "bun:test"; +import type { Server } from "bun"; +import { Hono } from "hono"; +import { authOrcidRoutes } from "../src/routes/auth-orcid"; +import { authWebRoutes } from "../src/routes/auth-web"; +import { hashAuthCode } from "../src/services/auth-code"; +import { PENDING_COOKIE_NAME, encodeState, signPending } from "../src/services/orcid-auth"; +import { USERNAME_SUFFIX_LIMIT } from "../src/services/username"; +import { + REREAD_USERNAME_SQL, + TAKEN_SQL, + usernameClaimStatement, +} from "../src/services/username-assignment"; +import { issueSession } from "../src/services/web-session"; +import type { Bindings, Variables } from "../src/types/bindings"; +import { freshDb, realD1 } from "./helpers/d1"; + +const ORIGIN = "https://nemar.org"; +const APP = "https://app.nemar.org"; +const ENCRYPTION_KEY = "username-assign-test-encryption-key-012345"; +const CSRF = "csrf-username-assign"; +/** `@nemar.test` is the synthetic-target suffix the non-production email fence + * admits, so nothing here can reach a real inbox (AGENTS.md). */ +const EMAIL = "abandoned@nemar.test"; +const SIGNUP_EMAIL = "brandnew@nemar.test"; +const CALLBACK_ORCID = "0000-0001-5109-353X"; +const PENDING_ORCID = "0000-0002-1825-0097"; + +/** The name ORCID's public record reports, reset per test. Ada Lovelace unless + * a test needs a name the ADR 0042 suggestion cannot be built from. */ +let publicRecordName = { given: "Ada", family: "Lovelace" }; + +let db: Database; +let app: Hono<{ Bindings: Bindings; Variables: Variables }>; +let orcidToken: Server; +let orcidPub: Server; +let realFetch: typeof fetch; + +beforeAll(() => { + orcidToken = Bun.serve({ + port: 0, + fetch(req) { + if (new URL(req.url).pathname === "/oauth/token") { + return Response.json({ orcid: CALLBACK_ORCID, name: "Ada Lovelace", access_token: "x" }); + } + return new Response("not found", { status: 404 }); + }, + }); + // ORCID's PUBLIC record API, which `refreshUserName` reads and whose host is + // derived from a fixed map rather than from ORCID_API_BASE -- so it is + // redirected by moving fetch's destination, not by an env knob. What it + // serves is `publicRecordName`, so a test can put a real record in front of + // the real parser instead of describing one. + orcidPub = Bun.serve({ + port: 0, + fetch: () => + Response.json({ + name: { + "given-names": { value: publicRecordName.given }, + "family-name": { value: publicRecordName.family }, + }, + }), + }); + realFetch = globalThis.fetch; + globalThis.fetch = ((input: RequestInfo | URL, init?: RequestInit) => { + const req = input instanceof Request ? input : new Request(input, init); + const url = new URL(req.url); + if (url.hostname.endsWith("orcid.org")) { + const local = new URL(url.pathname + url.search, `http://127.0.0.1:${orcidPub.port}`); + return realFetch(new Request(local, req)); + } + return realFetch(input as RequestInfo, init); + }) as typeof fetch; +}); + +afterAll(() => { + globalThis.fetch = realFetch; + orcidToken.stop(true); + orcidPub.stop(true); +}); + +/** + * When set, the username scan is diverted to SQL the database refuses. + * + * The sibling of `blockUserWrites()` in web-email-verification.test.ts, which + * makes every `UPDATE users` fail with a real SQLite trigger. A trigger cannot + * fire on a SELECT, so the equivalent fault for a READ is to point that one + * statement -- matched by the production constant, never a copy of it -- at a + * column that does not exist. SQLite itself raises the error, at prepare time, + * exactly where a transient D1 failure would surface; every other statement in + * the request still runs against the real database. + */ +let breakUsernameScan = false; + +/** + * When set, the lost-claim re-read (`REREAD_USERNAME_SQL`) is diverted the + * same way `breakUsernameScan` diverts `TAKEN_SQL` above: pointed at a column + * that does not exist, so SQLite raises at prepare time instead of the + * database quietly returning the row. Proves the re-read's own catch falls + * back to the pre-race values rather than reaching the route's outer catch. + */ +let breakUsernameReread = false; + +function env(): Bindings { + const base = realD1(db); + return { + DB: { + ...base, + prepare(sql: string) { + if (breakUsernameScan && sql === TAKEN_SQL) { + return base.prepare("SELECT no_such_column FROM users"); + } + if (breakUsernameReread && sql === REREAD_USERNAME_SQL) { + return base.prepare("SELECT no_such_column FROM users"); + } + return base.prepare(sql); + }, + }, + ENVIRONMENT: "test", + ENCRYPTION_KEY, + WEB_SESSION_COOKIE_DOMAIN: "", + ORCID_CLIENT_ID: "APP-TEST", + ORCID_CLIENT_SECRET: "test-secret", + ORCID_API_BASE: `http://localhost:${orcidToken.port}`, + APP_BASE_URL: APP, + } as unknown as Bindings; +} + +/** A web/ORCID-shaped row: `username IS NULL`, which is the state this whole + * feature is about. `given`/`family` are what decide whether it can be fixed. */ +function seedWebUser( + email: string, + { given = "Ada", family = "Lovelace", orcid = null as string | null } = {}, +): number { + db.run( + `INSERT INTO users (email, status, signup_source, email_verified, given_name, family_name, + orcid, orcid_verified, service_access) + VALUES (?, 'pending', 'web', 0, ?, ?, ?, ?, 0)`, + [email, given, family, orcid, orcid ? 1 : 0], + ); + const row = db.query<{ id: number }, [string]>("SELECT id FROM users WHERE email = ?").get(email); + if (!row) throw new Error(`seed failed for ${email}`); + return row.id; +} + +/** An unrelated account already holding a username, to force a collision. */ +function seedTakenUsername(username: string): void { + db.run( + `INSERT INTO users (username, email, password_hash, status, role, email_verified) + VALUES (?, ?, 'x', 'approved', 'member', 1)`, + [username, `${username}@nemar.test`], + ); +} + +async function plantCode(email: string, code: string): Promise { + db.run( + `INSERT INTO auth_codes (email, code_hash, expires_at, user_id) + VALUES (?, ?, datetime('now', '+10 minutes'), NULL)`, + [email, await hashAuthCode(code, env())], + ); +} + +function codeVerify(email: string, code: string): Promise { + return app.request( + "/auth/code/verify", + { + method: "POST", + headers: { Origin: ORIGIN, "content-type": "application/json" }, + body: JSON.stringify({ email, code, remember: false }), + }, + env(), + ); +} + +function usernameOf(id: number): { username: string | null; auto: number } { + const row = db + .query<{ username: string | null; username_auto_assigned: number }, [number]>( + "SELECT username, username_auto_assigned FROM users WHERE id = ?", + ) + .get(id); + if (!row) throw new Error(`no user ${id}`); + return { username: row.username, auto: row.username_auto_assigned }; +} + +function auditRows(id: number): { action: string; details: string | null }[] { + return db + .query<{ action: string; details: string | null }, [number]>( + "SELECT action, details FROM audit_log WHERE user_id = ? AND action = 'username_auto_assigned'", + ) + .all(id); +} + +/** + * Wait for an after-response assignment to land. + * + * The two ORCID paths run the name refresh and the assignment behind the + * response (`afterResponse`), so the row is written a few microtasks later. + * This polls the real database rather than sleeping a fixed interval, and gives + * up loudly instead of hanging. + */ +async function waitForUsername(id: number, attempts = 200): Promise { + for (let i = 0; i < attempts; i++) { + const { username } = usernameOf(id); + if (username) return username; + await new Promise((resolve) => setTimeout(resolve, 5)); + } + return null; +} + +/** The same wait for the name the ORCID public record supplies, which + * `refreshUserName` writes immediately BEFORE the assignment is attempted — + * so it is the signal that the chain reached the assignment at all. */ +async function waitForGivenName(id: number, attempts = 200): Promise { + for (let i = 0; i < attempts; i++) { + const row = db + .query<{ given_name: string | null }, [number]>("SELECT given_name FROM users WHERE id = ?") + .get(id); + if (row?.given_name) return row.given_name; + await new Promise((resolve) => setTimeout(resolve, 5)); + } + return null; +} + +/** Everything the route logged during one test. Captured rather than muted so + * a test can assert that an operator would actually see the fact. */ +let logs: string[]; +const realWarn = console.warn; +const realError = console.error; + +beforeEach(() => { + db = freshDb(); + breakUsernameScan = false; + breakUsernameReread = false; + publicRecordName = { given: "Ada", family: "Lovelace" }; + logs = []; + console.warn = (...a: unknown[]) => { + logs.push(a.join(" ")); + }; + console.error = (...a: unknown[]) => { + logs.push(a.join(" ")); + }; + app = new Hono<{ Bindings: Bindings; Variables: Variables }>(); + app.route("/auth", authWebRoutes); + app.route("/auth", authOrcidRoutes); +}); + +afterEach(() => { + console.warn = realWarn; + console.error = realError; +}); + +describe("POST /auth/code/verify", () => { + test("names an abandoned account from its name, in the sign-in itself", async () => { + const id = seedWebUser(EMAIL); + await plantCode(EMAIL, "123456"); + + const res = await codeVerify(EMAIL, "123456"); + expect(res.status).toBe(200); + + // Written by the time the response is built -- not after it. + expect(usernameOf(id)).toEqual({ username: "alovelace", auto: 1 }); + // ...and REPORTED, so the dashboard does not have to re-fetch to learn it. + const body = await res.json(); + expect(body.user.username).toBe("alovelace"); + expect(body.user.username_auto_assigned).toBe(true); + }); + + test("suffixes past a collision rather than failing the sign-in", async () => { + seedTakenUsername("alovelace"); + const id = seedWebUser(EMAIL); + await plantCode(EMAIL, "123456"); + + const res = await codeVerify(EMAIL, "123456"); + expect(res.status).toBe(200); + expect(usernameOf(id).username).toBe("alovelace-2"); + }); + + test("no family name leaves the column NULL, and signs the user in anyway", async () => { + // ADR 0042: nothing is invented from the email local part. Onboarding asks. + const id = seedWebUser(EMAIL, { given: "Prince", family: null as unknown as string }); + await plantCode(EMAIL, "123456"); + + const res = await codeVerify(EMAIL, "123456"); + expect(res.status).toBe(200); + expect(usernameOf(id)).toEqual({ username: null, auto: 0 }); + expect((await res.json()).user.username).toBeNull(); + expect(auditRows(id)).toHaveLength(0); + }); + + test("a name that folds to nothing in ASCII is the same answer", async () => { + const id = seedWebUser(EMAIL, { given: "美子", family: "山田" }); + await plantCode(EMAIL, "123456"); + expect((await codeVerify(EMAIL, "123456")).status).toBe(200); + expect(usernameOf(id).username).toBeNull(); + }); + + test("an account that already has a username keeps it, and is not re-marked", async () => { + const id = seedWebUser(EMAIL); + db.run("UPDATE users SET username = 'chosen' WHERE id = ?", [id]); + await plantCode(EMAIL, "123456"); + + expect((await codeVerify(EMAIL, "123456")).status).toBe(200); + expect(usernameOf(id)).toEqual({ username: "chosen", auto: 0 }); + expect(auditRows(id)).toHaveLength(0); + }); + + test("a saturated base leaves the column NULL and says so in the log", async () => { + // The other end of the suffix search: every variant up to the limit is + // taken, so there is nothing to claim. The account still signs in, still + // holds no handle, and -- because nothing else would ever surface it -- an + // operator gets one line naming the saturated base. + seedTakenUsername("alovelace"); + for (let n = 2; n <= USERNAME_SUFFIX_LIMIT; n++) seedTakenUsername(`alovelace-${n}`); + const id = seedWebUser(EMAIL); + await plantCode(EMAIL, "123456"); + + const res = await codeVerify(EMAIL, "123456"); + + expect(res.status).toBe(200); + expect(usernameOf(id)).toEqual({ username: null, auto: 0 }); + expect(auditRows(id)).toHaveLength(0); + expect((await res.json()).user.username).toBeNull(); + expect(logs.join("\n")).toContain(`every variant of "alovelace" is taken; user ${id}`); + }); + + test("a handle taken between the scan and the write is not stolen", async () => { + // The claim's OTHER guard: `NOT EXISTS`, which is evaluated with the write + // rather than before it. Staged the same way as the race above -- a trigger + // on the session INSERT, the statement immediately before the claim in the + // same batch -- except here the competing row is a DIFFERENT account taking + // the exact handle this sign-in planned to use. + const id = seedWebUser(EMAIL); + db.run( + `CREATE TRIGGER steal_the_handle AFTER INSERT ON web_sessions + BEGIN + INSERT INTO users (username, email, password_hash, status) + VALUES ('alovelace', 'thief@nemar.test', 'x', 'approved'); + END`, + ); + await plantCode(EMAIL, "123456"); + + const res = await codeVerify(EMAIL, "123456"); + + // No exception, no rollback: the sign-in landed whole, minus the username. + expect(res.status).toBe(200); + expect(db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM web_sessions").get()?.n).toBe(1); + expect(usernameOf(id)).toEqual({ username: null, auto: 0 }); + expect(auditRows(id)).toHaveLength(0); + // ...and the handle belongs to whoever got there first. + expect( + db + .query<{ email: string }, [string]>("SELECT email FROM users WHERE username = ?") + .get("alovelace")?.email, + ).toBe("thief@nemar.test"); + }); + + test("the claim reports a lost race rather than raising", async () => { + // The statement's own contract, asserted directly because it is what makes + // batching it inside a sign-in transaction safe: `CLAIM_USERNAME_SQL` is + // ONE statement whose NOT EXISTS arm runs with the write, so a name taken + // in between produces `changes = 0` and NOT a UNIQUE error that would roll + // the whole sign-in back. The route test above proves the behaviour; this + // proves the mechanism it rests on. + const id = seedWebUser(EMAIL); + seedTakenUsername("alovelace"); + + const claim = await usernameClaimStatement(realD1(db), id, "alovelace").run(); + + expect(claim.meta?.changes ?? 0).toBe(0); + expect(usernameOf(id)).toEqual({ username: null, auto: 0 }); + }); + + test("signing in twice assigns once", async () => { + // Two real sign-ins, not one call and an assumption. The second must find + // the username already set and do nothing at all: a second audit row would + // tell an operator the handle was re-derived, and a second claim on an + // account whose owner had since renamed it would undo their choice. + const id = seedWebUser(EMAIL); + await plantCode(EMAIL, "123456"); + expect((await codeVerify(EMAIL, "123456")).status).toBe(200); + expect(usernameOf(id)).toEqual({ username: "alovelace", auto: 1 }); + + await plantCode(EMAIL, "654321"); + const second = await codeVerify(EMAIL, "654321"); + + expect(second.status).toBe(200); + expect(usernameOf(id)).toEqual({ username: "alovelace", auto: 1 }); + expect(auditRows(id)).toHaveLength(1); + expect((await second.json()).user.username).toBe("alovelace"); + }); + + test("a failed username scan does not burn the code or the sign-in", async () => { + // The scan runs AFTER the code has been consumed and BEFORE the batch whose + // catch restores it, so an escaping error would answer a generic 500 and + // spend a code the user typed correctly. The feature is a nudge; it may + // never cost anybody a login. + const id = seedWebUser(EMAIL); + await plantCode(EMAIL, "123456"); + breakUsernameScan = true; + + const res = await codeVerify(EMAIL, "123456"); + + expect(res.status).toBe(200); + // A real session: the cookie is set and the row exists. + expect(res.headers.getSetCookie().some((ck) => ck.startsWith("nemar_session="))).toBe(true); + expect(db.query<{ n: number }, []>("SELECT COUNT(*) AS n FROM web_sessions").get()?.n).toBe(1); + // ...and the sign-in did everything else it does, including promoting the + // account off `pending` (the wire spells that "active"; the column is the + // fact). + expect((await res.json()).user.status).toBe("active"); + expect( + db.query<{ status: string }, [number]>("SELECT status FROM users WHERE id = ?").get(id) + ?.status, + ).toBe("verified"); + + // Nothing was assigned, nothing was claimed to have been. + expect(usernameOf(id)).toEqual({ username: null, auto: 0 }); + expect(auditRows(id)).toHaveLength(0); + // But an operator can see it happened, with the account it happened to. + expect(logs.join("\n")).toContain(`could not pick a username for user id=${id}`); + }); + + test("a username set mid-sign-in is reported, not the NULL we read first", async () => { + // The race: a `PATCH /auth/profile` names the account between this + // request's SELECT and its claim. Staged with SQLite's own trigger + // machinery rather than described -- the session INSERT is the statement + // immediately before the claim in the same batch, so a trigger on it lands + // the competing write in exactly the window the claim's `username IS NULL` + // predicate guards. The claim then reports `changes = 0` (nothing is + // stolen), and the row the response was built from is one revision stale. + const id = seedWebUser(EMAIL); + db.run( + `CREATE TRIGGER name_the_row_mid_signin AFTER INSERT ON web_sessions + BEGIN UPDATE users SET username = 'typed' WHERE id = NEW.user_id; END`, + ); + await plantCode(EMAIL, "123456"); + + const res = await codeVerify(EMAIL, "123456"); + expect(res.status).toBe(200); + + // The claim lost, so nothing was assigned and nothing was audited... + expect(usernameOf(id)).toEqual({ username: "typed", auto: 0 }); + expect(auditRows(id)).toHaveLength(0); + // ...and the response says what the account HAS, not what it had. + const body = await res.json(); + expect(body.user.username).toBe("typed"); + expect(body.user.username_auto_assigned).toBe(false); + }); + + test("a failed lost-claim re-read still signs in with the pre-race row", async () => { + // Same race as the test above, but the re-read that is meant to recover + // from it is itself broken. The fallback has to hold: the sign-in already + // committed by the time this runs, so a broken re-read may not fail the + // login over a value that is a nudge's nudge -- it falls back to the + // pre-race `userRow` the request already read at the top. + const id = seedWebUser(EMAIL); + db.run( + `CREATE TRIGGER name_the_row_mid_signin_reread AFTER INSERT ON web_sessions + BEGIN UPDATE users SET username = 'typed' WHERE id = NEW.user_id; END`, + ); + await plantCode(EMAIL, "123456"); + breakUsernameReread = true; + + const res = await codeVerify(EMAIL, "123456"); + + // Still a real session: a broken re-read must not cost anybody a login. + expect(res.status).toBe(200); + expect(res.headers.getSetCookie().some((ck) => ck.startsWith("nemar_session="))).toBe(true); + + // The row already changed underneath this request... + expect(usernameOf(id)).toEqual({ username: "typed", auto: 0 }); + // ...but the re-read that would have discovered that failed, so the + // response reports the row exactly as it was read at the top: no handle. + const body = await res.json(); + expect(body.user.username).toBeNull(); + expect(body.user.username_auto_assigned).toBe(false); + // An operator can still see it happened, with the account it happened to. + expect(logs.join("\n")).toContain( + `could not re-read the username of user id=${id} after a lost claim`, + ); + }); + + test("the assignment is audited, with the door it came through", async () => { + const id = seedWebUser(EMAIL); + await plantCode(EMAIL, "123456"); + await codeVerify(EMAIL, "123456"); + + const rows = auditRows(id); + expect(rows).toHaveLength(1); + expect(JSON.parse(rows[0].details ?? "{}")).toEqual({ + username: "alovelace", + source: "code_signin", + }); + }); +}); + +describe("POST /auth/orcid/finalize", () => { + async function finalize(): Promise { + const pending = await signPending( + { orcid: PENDING_ORCID, name: "Ada Lovelace", exp: Date.now() + 60_000 }, + ENCRYPTION_KEY, + ); + return app.request( + "/auth/orcid/finalize", + { + method: "POST", + headers: { + Origin: ORIGIN, + Cookie: `${PENDING_COOKIE_NAME}=${pending}`, + "content-type": "application/json", + }, + body: JSON.stringify({ email: SIGNUP_EMAIL, city: "Cambridge", country: "GB" }), + }, + env(), + ); + } + + test("a brand-new sign-up gets its name from ORCID and then a username", async () => { + // The INSERT writes no name at all: both come from the public record, which + // is why the assignment is chained BEHIND the refresh rather than beside it. + const res = await finalize(); + expect(res.status).toBe(200); + const id = (await res.json()).user.id as number; + + expect(await waitForUsername(id)).toBe("alovelace"); + expect(usernameOf(id).auto).toBe(1); + expect(JSON.parse(auditRows(id)[0]?.details ?? "{}")).toEqual({ + username: "alovelace", + source: "orcid_signup", + }); + }); + + test("a collision is suffixed here too", async () => { + seedTakenUsername("alovelace"); + const res = await finalize(); + const id = (await res.json()).user.id as number; + expect(await waitForUsername(id)).toBe("alovelace-2"); + }); +}); + +describe("GET /auth/orcid/callback (sign-in)", () => { + function callback(): Request { + return new Request(`${APP}/auth/orcid/callback?state=${CSRF}&code=fake-code`, { + method: "GET", + headers: { + Cookie: `nemar_oauth_state=${encodeState({ csrf: CSRF, mode: "login", next: "/dashboard" })}`, + }, + }); + } + + test("an already-linked account with no username gets one at sign-in", async () => { + // Deliberately seeded with NO name: the ORCID record supplies it on the way + // through, which is the case a sweep-only design never reaches. + const id = seedWebUser(EMAIL, { + given: null as unknown as string, + family: null as unknown as string, + orcid: CALLBACK_ORCID, + }); + db.run( + "INSERT INTO oauth_identities (user_id, provider, provider_subject, display_name) VALUES (?, 'orcid', ?, 'Ada')", + [id, CALLBACK_ORCID], + ); + + const res = await app.request(callback(), undefined, env()); + expect(res.status).toBe(302); + expect(res.headers.get("Location")).toBe(`${APP}/dashboard`); + + expect(await waitForUsername(id)).toBe("alovelace"); + expect(JSON.parse(auditRows(id)[0]?.details ?? "{}")).toEqual({ + username: "alovelace", + source: "orcid_signin", + }); + }); + + test("a collision is suffixed on this door too", async () => { + // Each door runs the scan itself; a suffix rule proved on one of them is + // not proved on the others. + seedTakenUsername("alovelace"); + const id = seedWebUser(EMAIL, { + given: null as unknown as string, + family: null as unknown as string, + orcid: CALLBACK_ORCID, + }); + db.run( + "INSERT INTO oauth_identities (user_id, provider, provider_subject, display_name) VALUES (?, 'orcid', ?, 'Ada')", + [id, CALLBACK_ORCID], + ); + + expect((await app.request(callback(), undefined, env())).status).toBe(302); + expect(await waitForUsername(id)).toBe("alovelace-2"); + }); + + test("a public-record name with no ASCII to fold to leaves the column NULL", async () => { + // The no-usable-name case on the door where the name arrives LAST: the + // record is read, written to the row, and only then is a handle attempted. + // Nothing is invented from the email local part (ADR 0042), so onboarding + // asks instead. + publicRecordName = { given: "美子", family: "山田" }; + const id = seedWebUser(EMAIL, { + given: null as unknown as string, + family: null as unknown as string, + orcid: CALLBACK_ORCID, + }); + db.run( + "INSERT INTO oauth_identities (user_id, provider, provider_subject, display_name) VALUES (?, 'orcid', ?, 'Ada')", + [id, CALLBACK_ORCID], + ); + + expect((await app.request(callback(), undefined, env())).status).toBe(302); + + // The name landing is the proof the after-response chain actually ran -- + // `refreshUserName` writes it, and the assignment is the statement directly + // after it. Only then is "no username" an answer rather than a not-yet. + expect(await waitForGivenName(id)).toBe("美子"); + expect(await waitForUsername(id, 40)).toBeNull(); + expect(usernameOf(id)).toEqual({ username: null, auto: 0 }); + expect(auditRows(id)).toHaveLength(0); + }); +}); + +describe("PATCH /auth/profile clears the mark", () => { + async function patch(id: number, body: unknown): Promise { + const { cookieIdRaw } = await issueSession(env(), id, false, null, null, "orcid"); + return app.request( + "/auth/profile", + { + method: "PATCH", + headers: { + Origin: ORIGIN, + Cookie: `nemar_session=${cookieIdRaw}`, + "content-type": "application/json", + }, + body: JSON.stringify(body), + }, + env(), + ); + } + + test("changing an auto-assigned username retires the offer", async () => { + const id = seedWebUser(EMAIL); + await plantCode(EMAIL, "123456"); + await codeVerify(EMAIL, "123456"); + expect(usernameOf(id).auto).toBe(1); + + const res = await patch(id, { username: "ada" }); + expect(res.status).toBe(200); + expect(usernameOf(id)).toEqual({ username: "ada", auto: 0 }); + expect((await res.json()).user.username_auto_assigned).toBe(false); + }); + + test("re-saving the SAME username is not a change and keeps the offer", async () => { + // The Settings form sends every field on every save; a routine save of the + // city must not silently retire an offer the user has not answered. + const id = seedWebUser(EMAIL); + await plantCode(EMAIL, "123456"); + await codeVerify(EMAIL, "123456"); + + const res = await patch(id, { username: "alovelace", city: "London" }); + expect(res.status).toBe(200); + expect(usernameOf(id)).toEqual({ username: "alovelace", auto: 1 }); + }); +}); diff --git a/shared/contract/account-copy.ts b/shared/contract/account-copy.ts new file mode 100644 index 000000000..3e717b8cd --- /dev/null +++ b/shared/contract/account-copy.ts @@ -0,0 +1,290 @@ +/** + * Every user-facing sentence about account tiers, upload access, and missing + * profile fields — in one place, so the CLI and the website say the same thing + * (epic #1250 phase 8, #1268; nemarOrg/website#309, #310). + * + * **This file is the source of truth.** `nemarOrg/website`'s + * `src/lib/account-copy.ts` is a transcription of it with the same keys and the + * same strings, the arrangement `publication-block.ts` already has with + * `publicationBlockReasonSchema`. The two repos share no package, so the values + * are transcribed rather than imported, and there is a drift test on each side: + * `test/account-copy-parity.test.ts` here and `test/account-copy-drift.test.ts` + * there, both of which read the OTHER repo's module as text and compare it key + * by key whenever both checkouts sit side by side (each skips, with a note, + * when the other is absent). Changing a sentence here means changing it there + * in the same breath. + * + * **Four rules this file has to keep, because both drift tests and the + * mirroring depend on them:** + * + * 1. *Every value is a literal string.* No template literals, no + * concatenation, no interpolation from a constant — the drift tests read the + * other repo's file as TEXT (neither can import a module from outside its + * own dependency graph), so a computed value would be invisible to them. + * Where a sentence embeds a number that lives somewhere else + * ({@link ACCOUNT_COPY}`["upload_access.request.why_hint"]` and the 20-500 + * char bounds), the literal is written out and `account-copy.test.ts` + * asserts it still agrees with {@link UPLOAD_ACCESS_WHY_MIN_CHARS} / + * {@link UPLOAD_ACCESS_WHY_MAX_CHARS}. + * 2. *Placeholders are `{name}`, filled by {@link fillCopy}.* A sentence with a + * moving part is a template, not a function, so it stays comparable as a + * string. The whole placeholder vocabulary is `{label} {blocks} {web} {cli} + * {date}`, plus `{reason}` on the CLI-only keys below. + * 3. *Keys are stable and dotted.* They are the mirroring contract; renaming + * one is a change on both sides. + * 4. *A `cli.` prefix means the website ignores it.* The mirroring is + * deliberately asymmetric: the website has no surface for sandbox training + * or for a terminal heading, and its drift test reports such keys as a note + * rather than a failure. Everything WITHOUT that prefix is mirrored there + * and must not be changed unilaterally. + * + * What is deliberately NOT here: + * + * - **Refusal messages.** `checkUploadAccessRequest` (backend + * services/upload-access.ts) and `profileRefusal` build their own `message`, + * and both clients prefer the backend's sentence wherever it sent one. What + * this file renders is the `missing` LIST under such a refusal, which is the + * half the two surfaces have to agree on. + * - **Anchors and hrefs.** Website routing, and the CLI has nothing to mirror + * them with; `website/src/lib/profile-gaps.ts` owns them. + * - **Admin status vocabulary.** Those name lifecycle rows for an operator, + * not the account model for its owner. + */ + +/** + * The copy table. Read it through {@link accountCopy} (or index it directly — + * the keys are typed). + */ +export const ACCOUNT_COPY = { + // ------------------------------------------------------------------------- + // Tiers (nemar-cli ADR 0040: unverified -> base -> upload) + // ------------------------------------------------------------------------- + "tier.unverified.label": "Email not verified", + "tier.base.label": "Base access", + "tier.upload.label": "Upload access", + "tier.unverified.lede": "Verify your email address to activate your account.", + // No `tier.base.lede` / `tier.upload.lede`: nothing renders a one-line + // description of those two tiers. The base tier's explanation is + // `upload_access.invite.body` on the card that offers the next step, and + // the upload tier's is `upload_access.granted.body`. Adding a second + // sentence for each would mean two keys to keep in step for one fact. See + // the every-key-has-a-consumer rule in the header. + + // ------------------------------------------------------------------------- + // Upload access (nemar-cli ADR 0042: one request, one admin answer) + // ------------------------------------------------------------------------- + "upload_access.section.title": "Upload access", + "upload_access.section.lede": + "A one-time admin grant. Reviewed against export-control and local-jurisdiction rules.", + "upload_access.state.granted": "Granted", + "upload_access.state.requested": "Requested", + "upload_access.state.not_requested": "Not requested", + "upload_access.invite.title": "Want to contribute a dataset?", + "upload_access.invite.body": + "Browsing, searching and downloading are open to your account already. Uploading needs one extra grant: an admin checks your GitHub handle and location against export-control and local-jurisdiction rules, once.", + "upload_access.gate.title": "Ask for upload access first", + "upload_access.gate.body": + "Your account can browse, search and download everything public already. Uploading needs one extra grant: an admin checks your GitHub handle and where you’re based against export-control and local-jurisdiction rules. You ask once, and it is answered by email.", + "upload_access.request.cta": "Request upload access", + "upload_access.request.form_body": + "Tell us what you plan to deposit. An admin reviews this alongside your name, username, GitHub handle and location, then grants access once.", + "upload_access.request.why_label": "What do you intend to upload?", + // The bounds are `UPLOAD_ACCESS_WHY_MIN_CHARS` / `UPLOAD_ACCESS_WHY_MAX_CHARS` + // in ./user.ts, which is also where the route and the CLI prompt read them. + // Written out rather than interpolated per rule 1 in the header; + // account-copy.test.ts fails if the numbers here stop matching the constants. + "upload_access.request.why_hint": "Describe what you intend to upload in 20-500 characters", + "upload_access.requested.title": "Your request is with an admin", + "upload_access.requested.lede": "An admin is reviewing your request.", + "upload_access.requested.lede_dated": "An admin is reviewing your request from {date}.", + "upload_access.requested.body.upload": + "We’ll email you when there’s a decision, and this page will let you upload as soon as it’s granted.", + "upload_access.requested.body": + "We’ll email you when there’s a decision. Everything else on your account keeps working in the meantime.", + "upload_access.requested.cta": "See the request", + "upload_access.granted.body": + "You can upload datasets from the upload page. Publishing one is a separate, per-dataset review you start from your dashboard.", + "upload_access.unverified.body": + "Verify your email address first — the review happens over email.", + "upload_access.docs_link": "How upload access works", + + // ------------------------------------------------------------------------- + // Profile gaps: the sentence, its two halves, and the vocabulary they use + // + // `gap.sentence` + " " + one of the `gap.set_on.*` templates is what + // `describeProfileGap` (./profile-gaps.ts) composes, and it is the sentence + // BOTH surfaces print: + // + // GitHub handle is missing: needed to request upload access. + // Set it in Settings or run `nemar auth profile set-github`. + // ------------------------------------------------------------------------- + "gap.sentence": "{label} is missing: needed {blocks}.", + "gap.set_on.both": "Set it in {web} or run `{cli}`.", + "gap.set_on.web": "Set it in {web}.", + // No command-only template HERE. Every field in the shared matrix has a web + // location — the CLI half is what a field can lack (a name owned by a + // verified ORCID record) — so a mirrored `Run {cli}.` sentence would be one + // no shared gap could reach. The CLI-only sandbox step is the exception and + // has its own `cli.gap.set_on.cli_only` at the bottom of this file. + "gap.blocks.verified": "to activate your account", + "gap.blocks.upload_access": "to request upload access", + "gap.blocks.publication": "to publish a dataset", + // For a field a client has never heard of (this contract's `profile_gaps` + // grew one after that client shipped), or one that names no block at all. + // Saying nothing about what it stops is better than guessing. + "gap.blocks.unknown": "to finish setting up your account", + // Noun forms, for a surface that REPORTS what a field blocks rather than + // instructing the person who has to fix it — the admin review card lists + // "upload access and publication", not "to request upload access". + "gap.blocks.noun.verified": "account activation", + "gap.blocks.noun.upload_access": "upload access", + "gap.blocks.noun.publication": "publication", + // Where a field a client does not recognise is set. Settings is the only + // honest answer: it is the page that holds the account form. + "gap.set_on.default_web": "Settings", + + // Field labels. These are the same labels a refused upload-access request + // renders from its `missing` array, which is why `why` and `email_verified` + // are here alongside the profile columns. + "gap.field.email_verified.label": "A verified email address", + "gap.field.username.label": "Username", + "gap.field.given_name.label": "Given name", + "gap.field.family_name.label": "Family name", + // Copy for the orcid_verified row lands ahead of the row itself: the + // website added the row in website#312, and the matrix half is Phase 9 + // (#1271). Parity is string-for-string, so the sentences arrive first. + "gap.field.orcid_verified.label": "Verified ORCID iD", + "gap.field.github_username.label": "GitHub handle", + "gap.field.city.label": "City", + "gap.field.country.label": "Country", + "gap.field.why.label": "A description of what you intend to upload", + + // Where each one is set. `.web` is prose that fits "Set it in ___"; `.cli` + // is the exact command, printed inside backticks by the templates above. + "gap.field.email_verified.set_on.web": "the verify step on your dashboard", + "gap.field.email_verified.set_on.cli": "nemar auth resend-verification", + "gap.field.username.set_on.web": "Settings", + "gap.field.username.set_on.cli": "nemar auth profile set-username", + "gap.field.given_name.set_on.web": "Settings", + "gap.field.given_name.set_on.cli": "nemar auth profile set-name", + "gap.field.family_name.set_on.web": "Settings", + "gap.field.family_name.set_on.cli": "nemar auth profile set-name", + // A verified ORCID iD owns the name: it is re-read on every sign-in and + // `PATCH /auth/profile` refuses the edit (`name_is_orcid_canonical`), so + // pointing this account at Settings would point it at a control it does + // not get. No CLI command either — `profile set-name` is refused for the + // same reason. + "gap.field.given_name.set_on.web.orcid": "your ORCID record at orcid.org, then sign in again", + "gap.field.family_name.set_on.web.orcid": "your ORCID record at orcid.org, then sign in again", + "gap.field.orcid_verified.set_on.web": "Settings", + "gap.field.orcid_verified.set_on.cli": "nemar auth profile orcid link", + "gap.field.github_username.set_on.web": "Settings", + "gap.field.github_username.set_on.cli": "nemar auth profile set-github", + "gap.field.city.set_on.web": "Settings", + "gap.field.city.set_on.cli": "nemar auth profile set-location", + "gap.field.country.set_on.web": "Settings", + "gap.field.country.set_on.cli": "nemar auth profile set-location", + "gap.field.why.set_on.web": "the request form in Settings", + "gap.field.why.set_on.cli": "nemar auth request-upload-access", + + // The headings each surface puts the gap list under. + "gaps.title": "What is still missing", + "gaps.none": "Nothing outstanding — every field NEMAR needs is filled in.", + "gaps.nudge.title": "Your profile isn’t finished yet", + "gaps.nudge.cta": "Complete your profile", + "gaps.upload.title": "Before you can ask, your account still needs:", + "gaps.request.title": "Finish these first:", + "gaps.admin.title": "Still missing", + // The admin card reports rather than instructs: an admin cannot set another + // person's city, so "Set it in Settings" would be advice for the wrong + // reader. It names the field and what it blocks, and nothing else. + "gaps.admin.item": "{label} — blocks {blocks}", + "gaps.admin.none": "Nothing missing — every field the request needs is filled in.", + "gaps.admin.not_loaded": "Not loaded — open the user page to see what is still missing.", + + // ------------------------------------------------------------------------- + // Onboarding + // ------------------------------------------------------------------------- + "onboarding.username.auto_assigned.title": "We chose your username from your name", + "onboarding.username.auto_assigned.body": + "Your account needed a username to be listed as a dataset owner, so one was made from your name when you signed in. You can change it once, here, until an admin grants you upload access.", + + // ------------------------------------------------------------------------- + // Welcome (the two steps that are about the account model itself) + // ------------------------------------------------------------------------- + "welcome.unverified.title": "Verify your email address", + "welcome.unverified.body": + "Authorizing with ORCID created your NEMAR account and signed you in. One step left: enter the 6-digit code we emailed you. That activates the account — no admin approval is involved.", + "welcome.active.title": "Your account is active", + "welcome.active.body": + "Authorizing with ORCID created your NEMAR account and signed you in. There is no separate sign-up step and no approval queue: browse, search, and your dashboard all work right now.", + "welcome.upload_access.title": "Ask for upload access when you’re ready to contribute", + "welcome.upload_access.body": + "Reading is open to every verified account. Uploading and compute need one extra grant, because an admin has to check your GitHub handle and location against export-control and local-jurisdiction rules first. You ask once, from Settings.", + + // =========================================================================== + // CLI-ONLY (`cli.` prefix). The website has no surface for these and its + // drift test reports them as a note rather than as missing copy. + // =========================================================================== + + // The heading `nemar auth status` and `nemar auth profile` print the gap + // list under. Not `gaps.title` ("What is still missing"): those two commands + // print several blocks in a row and the reader needs to know which part of + // the account each one is about, not what the list means. + "cli.gaps.section.title": "Profile", + // A CLI that has never refreshed against a backend has no gap list at all, + // which is not the same as an empty one. Same three-state treatment the + // upload-access line already has (ADR 0040). + "cli.gaps.unknown": "not checked — run 'nemar auth status --refresh'", + // The same absence for a command that DID fetch: `nemar auth profile` always + // talks to the backend, so telling it to refresh would be advice it just + // followed. What is missing there is the field, not the fetch. + "cli.gaps.unreported": "not reported by this backend", + // The command-only "set it in" template the website deliberately has no use + // for: every field in the shared matrix has a web location, and the one step + // that does not is sandbox training, which exists only in the terminal. + "cli.gap.set_on.cli_only": "Run `{cli}`.", + "cli.gap.field.sandbox.label": "Sandbox training", + "cli.gap.blocks.sandbox": "to upload a dataset from the CLI", + "cli.gap.field.sandbox.set_on.cli": "nemar sandbox", + + // `nemar dataset upload` preflight, before validation runs. + "cli.upload.preflight.title": "Upload access is not granted yet", + "cli.upload.preflight.body": + "Browsing, searching and downloading are open to your account already. Uploading needs one extra grant: an admin checks your GitHub handle and location against export-control and local-jurisdiction rules, once.", + // Also printed by `nemar auth status` and `nemar auth profile` under the + // upload-access line, which is why it is not named after the preflight. + "cli.upload_access.cta": "Ask for it with `nemar auth request-upload-access`.", + // The grant could not be READ — offline, a 5xx, a backend that predates the + // field. Never reported as "not granted": that is the one answer that sends + // someone who holds the grant off to ask for it again. + "cli.upload.preflight.unchecked": "Upload access could not be checked ({reason}); continuing.", + // The grant was read and is NOT held, but the list of what is missing was not + // reported at all -- a backend deployed before #1268. Absent is not empty: + // printing nothing under a refusal reads as "nothing is missing", which is + // the one thing this screen must never say. + "cli.upload.preflight.gaps_unknown": + "Could not determine what is still missing; run `nemar auth profile` for the list.", + "cli.upload.preflight.dry_run": "Continuing anyway: --dry-run uploads nothing.", +} as const; + +export type AccountCopyKey = keyof typeof ACCOUNT_COPY; + +/** Typed read. Exists so a caller cannot mistype a key and get `undefined` + * printed at a person — the union is the whole point. */ +export function accountCopy(key: AccountCopyKey): string { + return ACCOUNT_COPY[key]; +} + +/** + * Fill a `{name}` template from {@link ACCOUNT_COPY}. + * + * A placeholder with no value is left as-is rather than replaced with an empty + * string: "Set it in {web}." is visibly wrong and gets fixed, while "Set it in + * ." reads like a rendering glitch nobody can attribute. + */ +export function fillCopy(template: string, values: Readonly>): string { + return template.replace(/\{(\w+)\}/g, (whole, name: string) => + Object.hasOwn(values, name) ? values[name] : whole, + ); +} diff --git a/shared/contract/index.ts b/shared/contract/index.ts index 0081fddfa..76fac5610 100644 --- a/shared/contract/index.ts +++ b/shared/contract/index.ts @@ -15,3 +15,8 @@ export * from "./user.js"; export * from "./dataset.js"; export * from "./publication.js"; export * from "./identity.js"; +// Zero-dep string tables, not zod shapes: the copy every surface prints for a +// tier, an upload-access state or a missing profile field, and the matrix that +// decides which fields are missing (#1268, ADR 0045). +export * from "./account-copy.js"; +export * from "./profile-gaps.js"; diff --git a/shared/contract/profile-gaps.ts b/shared/contract/profile-gaps.ts new file mode 100644 index 000000000..ead65111c --- /dev/null +++ b/shared/contract/profile-gaps.ts @@ -0,0 +1,552 @@ +/** + * What an account is still missing, what each missing field blocks, and where + * it is set (epic #1250 phase 8, #1268; nemarOrg/website#309, #310). + * + * **One matrix, one computation, three readers.** `GET /users/me` and + * `GET /auth/me` both carry `profile_gaps` computed by {@link computeProfileGaps}, + * and `checkUploadAccessRequest` (backend services/upload-access.ts) builds its + * `missing` array from the SAME function — which is the point of this module. + * A request refused for `github_username` and a `nemar auth status` that never + * mentioned it is the failure this arrangement makes impossible: there is one + * rule, and both answers are it. + * + * The website transcribes the table (its `src/lib/profile-gaps.ts`) rather than + * importing it, for the reason ./account-copy.ts records, and derives the same + * list locally when talking to a backend that predates `profile_gaps`. Its + * derivation and this one are line-for-line the same rules; `test/profile-gaps- + * matrix.test.ts` here pins every one of the 2^7 field combinations against a + * fixture copied from that table so a change on either side is loud. + * + * **ORCID is not a gap.** The phase-8 matrix lists it as required at CLI signup + * and as the source of the name, and both are true — but a CLI account cannot + * exist without one, and a web account holds every tier without one, so an + * `orcid` entry would name something nothing is waiting on. What an iD changes + * is WHERE THE NAME IS SET (the `.orcid` set-on variants below), and that is + * its whole effect on this list. + * + * **`why` is in the table and is not derivable.** It is part of the refused- + * request vocabulary (`missing` on a 400 from + * `POST /users/me/upload-access/request`), which both surfaces render with + * these same sentences; nothing is stored until the form is submitted, so no + * account row can ever be missing it. `email_verified` is the other member of + * that vocabulary and IS derivable, from `status` / `email_verified`. + * + * **Sandbox training is CLI-only** and is deliberately not in the matrix: the + * website has no such step and mirroring a row it can never render would make + * the shared table a lie. It is declared separately at the bottom of this file + * and rendered through the same sentence machinery (ADR 0045). + */ + +import { ACCOUNT_COPY, type AccountCopyKey, fillCopy } from "./account-copy.js"; +import type { AccountStatus } from "./user.js"; + +/** + * What a missing field stops the account from doing. + * + * Ordered by how soon the account walks into it: an unverified inbox blocks + * everything, an upload-access request comes before there is anything to + * publish. {@link describeProfileGap} names the FIRST of a gap's blocks — the + * nearest wall, not every wall behind it — which is why the order is part of + * the data and not incidental. + */ +export type GapBlock = "verified" | "upload_access" | "publication"; + +/** The fields this contract knows how to describe. A `profile_gaps` entry + * naming anything else still renders (see {@link resolveProfileGap}); it just + * renders with its own name and no command. */ +export type GapField = + | "email_verified" + | "username" + | "given_name" + | "family_name" + | "github_username" + | "city" + | "country" + | "why"; + +/** + * Which surface can set a field. + * + * This is what `set_on` carries on the wire, and it is deliberately the SURFACE + * and not the prose: "Settings" is a website noun and + * `nemar auth profile set-github` a CLI one, and neither is something the + * backend should have to spell for two clients that already know their own + * vocabulary (they read it out of ./account-copy.ts). What the wire is telling + * a client is whether the field is reachable from where the reader is standing + * — a name owned by a verified ORCID record answers `["web"]`, because no CLI + * command sets it. + */ +export type GapSurface = "web" | "cli"; + +interface ProfileGapDefinition { + /** Blocks, nearest first. */ + readonly blocks: readonly GapBlock[]; + /** Copy keys. Split out rather than inlined so the sentences stay in + * ./account-copy.ts, which is the file the website mirrors. */ + readonly labelKey: AccountCopyKey; + readonly webKey: AccountCopyKey; + /** `null` when no CLI command sets it. */ + readonly cliKey: AccountCopyKey | null; + /** Where a verified ORCID iD moves the "set it in" half. Only the two name + * halves have one: with an iD linked the record owns the name and + * `PATCH /auth/profile` refuses the edit (`name_is_orcid_canonical`), on + * both credentials since ADR 0044. */ + readonly orcidWebKey?: AccountCopyKey; + /** + * When this field counts as missing on a given account row, or `null` for a + * field that is not account state at all. + * + * THE RULE LIVES IN THE TABLE, which is the point. {@link computeProfileGaps} + * used to carry a hand-written if-chain beside this matrix, so adding a row + * here and forgetting the branch there produced a field that every surface + * could describe and none could ever raise -- silently, because the matrix + * and the derivation had no way to disagree out loud. `Record` + * now makes the predicate part of what a row IS, so a new field does not + * compile until someone has said when it is missing. + * + * `null` is the honest spelling of "never raised from a row" and replaces the + * old `derivable: boolean`: one field rather than two that had to agree, and + * {@link DERIVABLE_GAP_FIELDS} is derived from it. + */ + readonly isMissing: ((account: ProfileGapAccount) => boolean) | null; +} + +/** + * The matrix, as data. + * + * Order is prompt order, and it is the order every surface renders: the + * account's own step first (verify), then who you are, then the two + * export-control fields, then the request text. It is also the order + * `checkUploadAccessRequest` builds `missing` in, because that function now + * builds it from here — so a refusal and a status nudge list the same things + * the same way round by construction rather than by two people agreeing. + */ +export const PROFILE_GAP_MATRIX: Record = { + email_verified: { + // Everything past browsing needs a proved inbox, and the upload-access + // review itself happens over email (`email_not_verified` is its own + // refusal, ahead of the profile check). + blocks: ["verified", "upload_access"], + labelKey: "gap.field.email_verified.label", + webKey: "gap.field.email_verified.set_on.web", + cliKey: "gap.field.email_verified.set_on.cli", + // `status === "pending"` is the unverified tier; an explicit `false` says + // the same thing from the newer flag. `undefined`/`null` on an active + // account is a caller that does not report the flag, NOT an unproved inbox + // -- treating it as one would show a verify step to accounts with nothing + // to verify. + isMissing: (a) => a.status === "pending" || a.email_verified === false, + }, + username: { + blocks: ["upload_access"], + labelKey: "gap.field.username.label", + webKey: "gap.field.username.set_on.web", + cliKey: "gap.field.username.set_on.cli", + // `undefined` is "could not be read" and does NOT raise the gap; see the + // field's note on ProfileGapAccount. + isMissing: (a) => a.username !== undefined && isBlank(a.username), + }, + given_name: { + // A DOI cites a person (ADR 0041), so the name outlives the request that + // first asks for it. + blocks: ["upload_access", "publication"], + labelKey: "gap.field.given_name.label", + webKey: "gap.field.given_name.set_on.web", + cliKey: "gap.field.given_name.set_on.cli", + orcidWebKey: "gap.field.given_name.set_on.web.orcid", + // Raised even under a verified ORCID iD. The gap is real either way: the + // upload-access request refuses with `missing: ["given_name", + // "family_name"]` whatever owns the name, and publication is blocked with + // `owner_name_missing`. What the iD changes is where it is set. + isMissing: (a) => isBlank(a.given_name), + }, + family_name: { + blocks: ["upload_access", "publication"], + labelKey: "gap.field.family_name.label", + webKey: "gap.field.family_name.set_on.web", + cliKey: "gap.field.family_name.set_on.cli", + orcidWebKey: "gap.field.family_name.set_on.web.orcid", + isMissing: (a) => isBlank(a.family_name), + }, + github_username: { + blocks: ["upload_access", "publication"], + labelKey: "gap.field.github_username.label", + webKey: "gap.field.github_username.set_on.web", + cliKey: "gap.field.github_username.set_on.cli", + isMissing: (a) => isBlank(a.github_username), + }, + city: { + blocks: ["upload_access"], + labelKey: "gap.field.city.label", + webKey: "gap.field.city.set_on.web", + cliKey: "gap.field.city.set_on.cli", + isMissing: (a) => isBlank(a.city), + }, + country: { + blocks: ["upload_access"], + labelKey: "gap.field.country.label", + webKey: "gap.field.country.set_on.web", + cliKey: "gap.field.country.set_on.cli", + isMissing: (a) => isBlank(a.country), + }, + why: { + blocks: ["upload_access"], + labelKey: "gap.field.why.label", + webKey: "gap.field.why.set_on.web", + cliKey: "gap.field.why.set_on.cli", + // `null`, not a predicate that always answers false: nothing is stored + // until the request form is submitted, so no account ROW can be missing it. + // It is in the table only so a refusal's `missing: ["why"]` renders. + isMissing: null, + }, +}; + +/** Every known field, in prompt order. */ +export const GAP_FIELDS = Object.keys(PROFILE_GAP_MATRIX) as readonly GapField[]; + +/** The subset {@link computeProfileGaps} can raise from an account row — every + * field except `why`, which only a submitted form can be missing. Derived + * from the predicates rather than declared beside them, so the two cannot + * drift apart. */ +export const DERIVABLE_GAP_FIELDS: readonly GapField[] = GAP_FIELDS.filter( + (field) => PROFILE_GAP_MATRIX[field].isMissing !== null, +); + +const GAP_BLOCK_COPY: Record = { + verified: "gap.blocks.verified", + upload_access: "gap.blocks.upload_access", + publication: "gap.blocks.publication", +}; + +/** "upload access", "upload access and publication" — for a surface that + * REPORTS rather than instructs (the website's admin review card). + * Deliberately noun-phrased, not the sentence fragments above. */ +const GAP_BLOCK_NOUNS: Record = { + verified: "gap.blocks.noun.verified", + upload_access: "gap.blocks.noun.upload_access", + publication: "gap.blocks.noun.publication", +}; + +function isGapBlock(value: unknown): value is GapBlock { + return value === "verified" || value === "upload_access" || value === "publication"; +} + +/** Whitespace counts as absent, matching `resolveUploaderIdentity`'s trim + * (backend services/uploader-identity.ts): a row holding " " has no city. */ +function isBlank(value: string | null | undefined): boolean { + return (value ?? "").trim().length === 0; +} + +/** + * The account columns the gap rules read. + * + * Structural rather than a `Pick<>` of any one wire shape, because four + * different rows are passed in: the `/users/me` join, the `/auth/me` session + * row, the upload-access precondition SELECT, and (on the website) an + * `AdminUserDetail`. Every field is optional so a partial row is expressible; + * what stops a backend from silently dropping one is that its own row + * interface names the columns, and the route tests assert the resulting gap. + */ +export interface ProfileGapAccount { + /** + * `"pending"` is the unverified tier (ADR 0040). + * + * The declared vocabulary rather than `string`, because the rule below turns + * on one literal and a caller passing the dashboard's collapsed + * `"active"`/`"pending"` value, or a typo, would silently take the other + * branch. The four values are the ones `users.status` can hold (a CHECK + * constraint since migration 0001), which is what every caller reads it from. + */ + readonly status?: AccountStatus | null; + readonly email_verified?: boolean | null; + /** + * `undefined` means the username could not be READ, which is not the same as + * a NULL username and does not raise the gap: prompting someone to set a + * handle they may already have is worse than omitting a line they can still + * reach from Settings. The website hits that case for real (it resolves the + * username through a separate fetch that can fail); server-side the column is + * always in hand, and the rule is kept identical so the two derivations are + * the same rules rather than nearly the same rules. + */ + readonly username?: string | null; + readonly given_name?: string | null; + readonly family_name?: string | null; + readonly github_username?: string | null; + readonly city?: string | null; + readonly country?: string | null; + /** Moves the name halves' `set_on` to the ORCID record. */ + readonly orcid_verified?: boolean | null; +} + +/** One `profile_gaps` entry, exactly as it goes on the wire. */ +export interface ProfileGapEntry { + readonly field: string; + readonly blocks: readonly GapBlock[]; + readonly set_on: readonly GapSurface[]; +} + +/** + * The same entry as it ARRIVES, before anything has narrowed it. + * + * Deliberately loose, and deliberately declared ONCE. Two readers take entries + * in this shape — {@link resolveWireProfileGaps} and the CLI's `ProfileGapView` + * (src/lib/account-gaps.ts) — and they were both spelling it inline, alongside + * two zod schemas describing the same thing (the wire's `profileGapSchema` and + * the config cache's). Four declarations of one shape is three chances for a + * relaxation on one side to go unnoticed on another, which is precisely the + * drift this whole module exists to end. + * + * `unknown` rather than the narrow types, because both sources can hand over + * something older or newer than this build: a backend from before the + * vocabulary grew, or a config cache written by a different CLI version and + * round-tripped through JSON. Narrowing is `resolveWireProfileGaps`'s job, and + * an entry it cannot use is dropped rather than fatal. + * + * The two schemas stay schemas — they validate, and a type cannot — but their + * inferred types are checked at their declaration sites against + * {@link GapFieldsStayOptional}, so a schema that makes `blocks` or `set_on` + * required fails to compile. The runtime half of the same guarantee is + * test/contract-schemas.test.ts "a profile_gaps entry may carry only its + * field name". + */ +export interface ProfileGapWireEntry { + readonly field?: unknown; + readonly blocks?: unknown; + readonly set_on?: unknown; +} + +/** + * Compile-time only: resolves to `never` — and so fails to compile at a + * declaration site that assigns it `true` — the moment `blocks` or `set_on` + * stops accepting `undefined` on `T`. + * + * This is the guarantee `resolveWireProfileGaps` below and the config-cache + * reader (src/lib/config.ts) actually rest on: an entry naming only `field` + * must still parse and render, falling back to the matrix for whatever it + * omits. The declaration this replaced, `AssignableTo`, + * never enforced that — every field on {@link ProfileGapWireEntry} is typed + * `unknown`, so any object shape is assignable to it, required fields + * included (checked against tsc 5.9.3 strict: making `blocks` required on + * `profileGapSchema` still passed the old check). This one reads the two + * fields it actually cares about off `T` directly instead of going through + * that interface. + */ +export type GapFieldsStayOptional = + undefined extends T["blocks"] ? (undefined extends T["set_on"] ? true : never) : never; + +/** Where a field can be set, given what owns the name on this account. */ +export function gapSurfaces(field: string, orcidVerified = false): GapSurface[] { + const def = Object.hasOwn(PROFILE_GAP_MATRIX, field) + ? PROFILE_GAP_MATRIX[field as GapField] + : undefined; + // An unknown field still has a home: Settings holds the account form. + if (!def) return ["web"]; + const underOrcid = orcidVerified && def.orcidWebKey !== undefined; + return underOrcid || def.cliKey === null ? ["web"] : ["web", "cli"]; +} + +/** + * Every gap on this account, in matrix order. + * + * The one computation. `GET /users/me`, `GET /auth/me` and the upload-access + * request preconditions all call it, which is what makes them unable to + * disagree — the property epic #1250 phase 8 exists to establish. + */ +export function computeProfileGaps(account: ProfileGapAccount): ProfileGapEntry[] { + const orcidVerified = account.orcid_verified === true; + const entry = (field: GapField): ProfileGapEntry => ({ + field, + blocks: PROFILE_GAP_MATRIX[field].blocks, + set_on: gapSurfaces(field, orcidVerified), + }); + + // Iterating the matrix rather than restating it: matrix order IS prompt + // order, and every rule is a property of the row it belongs to, so a field + // cannot be described here and un-raisable, or raised in an order the + // refusal does not share. + return GAP_FIELDS.filter((field) => PROFILE_GAP_MATRIX[field].isMissing?.(account) === true).map( + entry, + ); +} + +/** Just the field names, in matrix order — the shape + * `checkUploadAccessRequest` puts in `missing`. */ +export function profileGapFields(account: ProfileGapAccount): string[] { + return computeProfileGaps(account).map((gap) => gap.field); +} + +/** One gap resolved to everything a surface needs to print it. */ +export interface ResolvedProfileGap { + /** As the API spells it. A string, not {@link GapField}: an entry naming a + * field this build has never heard of is rendered, not dropped. */ + readonly field: string; + readonly label: string; + readonly blocks: readonly GapBlock[]; + /** Prose that fits "Set it in ___". */ + readonly setOnWeb: string; + /** The exact CLI command, or null when none sets it. */ + readonly setOnCli: string | null; + /** False when this build has no definition for `field`. */ + readonly known: boolean; +} + +/** + * Resolve one field name into a renderable gap. + * + * An unrecognised field is kept rather than dropped: the vocabulary is closed + * today, and silently swallowing a value it grows tomorrow would tell someone + * their request failed for no reason at all. It renders with its own name, the + * generic Settings destination, and no command. + */ +export function resolveProfileGap( + field: string, + options: { readonly orcidVerified?: boolean; readonly blocks?: readonly GapBlock[] } = {}, +): ResolvedProfileGap { + const orcidVerified = options.orcidVerified === true; + const def = Object.hasOwn(PROFILE_GAP_MATRIX, field) + ? PROFILE_GAP_MATRIX[field as GapField] + : undefined; + if (!def) { + return { + field, + label: field, + blocks: options.blocks ?? [], + setOnWeb: ACCOUNT_COPY["gap.set_on.default_web"], + setOnCli: null, + known: false, + }; + } + const underOrcid = orcidVerified && def.orcidWebKey !== undefined; + return { + field, + blocks: options.blocks ?? def.blocks, + label: ACCOUNT_COPY[def.labelKey], + setOnWeb: ACCOUNT_COPY[underOrcid ? (def.orcidWebKey as AccountCopyKey) : def.webKey], + // No command under a verified iD: `nemar auth profile set-name` is refused + // for the same reason the web fields are withheld. + setOnCli: underOrcid || def.cliKey === null ? null : ACCOUNT_COPY[def.cliKey], + known: true, + }; +} + +/** Position in {@link GAP_FIELDS}, or the end for an unknown field. */ +function fieldOrder(field: string): number { + const index = (GAP_FIELDS as readonly string[]).indexOf(field); + return index === -1 ? GAP_FIELDS.length : index; +} + +/** + * Resolve a bare list of field names — a refused request's `missing` array — + * into gaps. + * + * Order is the caller's, because a refusal's order is the backend's deliberate + * one and it is already matrix order (`checkUploadAccessRequest` builds it from + * {@link computeProfileGaps}). + */ +export function resolveProfileGaps( + fields: readonly string[], + options: { readonly orcidVerified?: boolean } = {}, +): ResolvedProfileGap[] { + return fields.map((field) => resolveProfileGap(field, { orcidVerified: options.orcidVerified })); +} + +/** + * Resolve wire entries into gaps, re-sorted into matrix order so a backend that + * emits them in another order still renders the same list. + * + * `blocks` is taken from the wire when it sends a usable one, because the + * backend is the authority on what a field blocks TODAY; the matrix is the + * fallback, and the two agree by construction. Unknown block values are dropped + * rather than printed raw, and an entry left with none falls back to the matrix. + */ +export function resolveWireProfileGaps( + entries: readonly ProfileGapWireEntry[], + options: { readonly orcidVerified?: boolean } = {}, +): ResolvedProfileGap[] { + return entries + .filter((entry) => !!entry && typeof entry.field === "string") + .map((entry) => { + const blocks = Array.isArray(entry.blocks) ? entry.blocks.filter(isGapBlock) : []; + return resolveProfileGap(entry.field as string, { + orcidVerified: options.orcidVerified, + blocks: blocks.length > 0 ? blocks : undefined, + }); + }) + .sort((a, b) => fieldOrder(a.field) - fieldOrder(b.field)); +} + +/** + * The half of the sentence after the label — "is missing: needed to request + * upload access. Set it in Settings or run `nemar auth profile set-github`." + * + * Exported so a surface can link or colour the LABEL and still print one + * sentence: `label + " " + profileGapTail(gap)` and {@link describeProfileGap} + * are the same words either way. + */ +export function profileGapTail(gap: ResolvedProfileGap): string { + const first = gap.blocks[0]; + const blocks = first ? ACCOUNT_COPY[GAP_BLOCK_COPY[first]] : ACCOUNT_COPY["gap.blocks.unknown"]; + // `{label}` is filled with "" here and re-attached by the caller, so the + // template stays one string in account-copy.ts. + const need = fillCopy(ACCOUNT_COPY["gap.sentence"], { label: "", blocks }).trimStart(); + const setOn = gap.setOnCli + ? fillCopy(ACCOUNT_COPY["gap.set_on.both"], { web: gap.setOnWeb, cli: gap.setOnCli }) + : fillCopy(ACCOUNT_COPY["gap.set_on.web"], { web: gap.setOnWeb }); + return `${need} ${setOn}`; +} + +/** + * The one sentence every surface prints for a gap: + * + * > GitHub handle is missing: needed to request upload access. Set it in + * > Settings or run `nemar auth profile set-github`. + * + * It names the FIRST block only. A GitHub handle blocks publication as well as + * the request, but the request is the wall in front of the person reading it; + * listing the ones behind it makes a longer sentence that helps nobody decide + * what to do next. The full list stays on `gap.blocks` for callers that want it. + */ +export function describeProfileGap(gap: ResolvedProfileGap): string { + return `${gap.label} ${profileGapTail(gap)}`; +} + +/** "upload access and publication" — the full block list, for a surface that + * reports rather than instructs. */ +export function describeProfileGapBlocks(gap: ResolvedProfileGap): string { + const nouns = gap.blocks.map((block) => ACCOUNT_COPY[GAP_BLOCK_NOUNS[block]]); + if (nouns.length === 0) return ""; + if (nouns.length === 1) return nouns[0]; + return `${nouns.slice(0, -1).join(", ")} and ${nouns[nouns.length - 1]}`; +} + +/** + * Sandbox training, the one step that exists in the terminal and nowhere else + * (ADR 0045). + * + * Not a member of {@link PROFILE_GAP_MATRIX} on purpose: that table is the + * mirrored one, and a row the website can never render would make it a lie + * about what the two surfaces share. It is rendered through the same + * {@link describeProfileGap} machinery so the CLI prints one shape of sentence + * rather than two, and it carries no `blocks` from {@link GapBlock} because + * what it blocks — a real upload from this machine — is not one of the three + * things the shared vocabulary names. + */ +export const CLI_SANDBOX_GAP = { + field: "sandbox", + label: ACCOUNT_COPY["cli.gap.field.sandbox.label"], + command: ACCOUNT_COPY["cli.gap.field.sandbox.set_on.cli"], +} as const; + +/** "Sandbox training is missing: needed to upload a dataset from the CLI. Run + * `nemar sandbox`." Composed from the shared templates, so it reads as one of + * the list rather than as an aside bolted onto it. */ +export function describeSandboxGap(): string { + const need = fillCopy(ACCOUNT_COPY["gap.sentence"], { + label: CLI_SANDBOX_GAP.label, + blocks: ACCOUNT_COPY["cli.gap.blocks.sandbox"], + }); + const setOn = fillCopy(ACCOUNT_COPY["cli.gap.set_on.cli_only"], { + cli: CLI_SANDBOX_GAP.command, + }); + return `${need} ${setOn}`; +} diff --git a/shared/contract/user.ts b/shared/contract/user.ts index 75269da63..e45856887 100644 --- a/shared/contract/user.ts +++ b/shared/contract/user.ts @@ -10,6 +10,88 @@ */ import { z } from "zod"; +import type { GapFieldsStayOptional } from "./profile-gaps.js"; + +/** + * What `users.status` holds — the account tier, declared once (#1268, ADR 0040). + * + * CLOSED, and closed by the database rather than by optimism: migration 0001 + * put `CHECK (status IN ('pending', 'verified', 'approved', 'revoked'))` on the + * column and 0026's table rebuild carried the same four values over, so a fifth + * cannot be written without a migration. Every `SET status =` in `backend/src` + * writes one of them — `'verified'` (email-verification.ts, auth.ts), + * `'approved'` (routes/admin/users.ts, migration 0062), `'revoked'` + * (db/user-tombstone.ts) — and signup inserts the `'pending'` default. + * + * Declared here because it was being compared as a bare string on both sides of + * the wire (`status === "pending"` decides the `email_verified` gap in + * ./profile-gaps.ts) while typed `z.string()`, so a typo in either place was a + * silently-false comparison rather than a compile error. Adding a status is a + * schema change AND a contract change on both surfaces already; a closed enum + * on the wire costs nothing that was not already owed. + */ +export const accountStatusSchema = z.enum(["pending", "verified", "approved", "revoked"]); +export type AccountStatus = z.infer; + +/** + * What `/auth/me` reports as `status`, which is NOT {@link accountStatusSchema}. + * + * `userStatusForDashboard` (backend routes/auth-web.ts) collapses `approved` + * and `verified` into `"active"`, because the dashboard's second state exists + * to carry a button — "pending" means exactly "verify your email" and nothing + * else. `"revoked"` is the defensive tail of that mapping (`?? row.status` for + * a status it will not translate) and is unreachable today: every door that + * calls `publicUser` refuses a revoked account before it gets there. + * + * Two enums rather than one because they are two vocabularies, and folding + * them would let a wire value that means "verify your inbox" be compared + * against a column value that means "an admin has not looked yet". + */ +export const webAccountStatusSchema = z.enum(["active", "pending", "revoked"]); +export type WebAccountStatus = z.infer; + +/** + * One `profile_gaps` entry (#1268, ADR 0045). + * + * Carried by BOTH user payloads — `GET /users/me` for the CLI and + * `GET /auth/me` for the dashboard — and computed by the one function in + * ./profile-gaps.ts that the upload-access preconditions also build their + * `missing` array from, so the two cannot disagree about what an account still + * needs. + * + * `field` is a plain string rather than the `GapField` union: the vocabulary is + * closed today, and a client that dropped an entry it did not recognise would + * tell a user their request failed for no reason at all. `blocks` says what the + * absence stops, nearest first; `set_on` says which surfaces can set it, and is + * `["web"]` alone for a name owned by a verified ORCID record, where no CLI + * command applies. Both are `.passthrough()`-friendly enums-as-strings for the + * same reason `field` is. + * + * ONLY `field` IS REQUIRED, and that is not laxity — it is the same rule the + * two renderers already follow. `resolveWireProfileGaps` treats a missing or + * unusable `blocks` as "fall back to the matrix" and reads `set_on` not at all; + * the CLI's own config-cache schema (src/lib/config.ts) has both `.optional()` + * so a cache written by an older build still parses. Demanding them HERE would + * have been the strictest link in a chain nothing else tightens, and + * `getCurrentUser` throws `ApiError` on any schema mismatch — so one entry + * missing one key would take down every `/users/me` consumer over a field the + * renderer was ready to do without. + */ +export const profileGapSchema = z + .object({ + field: z.string(), + blocks: z.array(z.string()).optional(), + set_on: z.array(z.string()).optional(), + }) + .passthrough(); +export type ProfileGapWire = z.infer; + +/** What this schema parses must remain something the renderers accept: neither + * `blocks` nor `set_on` may become required. The alias resolves to `never` + * and stops compiling the moment either one does; the runtime half of the + * same rule is test/contract-schemas.test.ts "a profile_gaps entry may carry + * only its field name". */ +export const _profileGapWireIsRenderable: GapFieldsStayOptional = true; /** A NEMAR user as returned inside the /users/me envelope. */ export const userSchema = z @@ -41,11 +123,26 @@ export const userSchema = z * matters most for `email_verified` -- telling someone their confirmed * inbox is unconfirmed sends them to redeem a code they do not need. */ - status: z.string().optional(), + status: accountStatusSchema.optional(), email_verified: z.boolean().optional(), orcid_verified: z.boolean().optional(), given_name: z.string().nullable().optional(), family_name: z.string().nullable().optional(), + /** + * What this account is still missing, and what each absence blocks (#1268, + * ADR 0045). `.optional()` like everything else added since #1254: a + * backend that predates phase 8 sends no such key, and an ABSENT list is + * "this API cannot say" while an EMPTY one is "nothing is missing". The CLI + * renders those two differently and must not collapse them. + */ + profile_gaps: z.array(profileGapSchema).optional(), + /** + * True when the username on this account was derived from the name rather + * than chosen (#1268, ADR 0045) — by the backfill sweep or at a web sign-in + * — and has not been changed since. Optional for the same reason; it is + * what lets a surface offer "we picked this, change it if you like". + */ + username_auto_assigned: z.boolean().optional(), }) .passthrough(); export type ContractUser = z.infer; @@ -138,6 +235,14 @@ export type AdminUsersListResponse = z.infer; @@ -299,8 +405,9 @@ export const UPLOAD_ACCESS_WHY_MAX_CHARS = 500; * (backend `publicUser`). * * NOT the same shape as {@link userSchema}, which is the CLI's `/users/me` - * envelope: this one reports `status` as the dashboard's two-state value - * ("active" / "pending"), carries the profile fields the Settings page edits, + * envelope: this one reports `status` as the dashboard's collapsed value + * ({@link webAccountStatusSchema}, not the column's own vocabulary), carries + * the profile fields the Settings page edits, * and omits everything about API tokens and sandbox state. They are two * audiences, not one shape with optional halves. * @@ -317,7 +424,7 @@ export const webUserSchema = z email: z.string(), username: z.string().nullable(), role: z.string(), - status: z.string(), + status: webAccountStatusSchema, email_verified: z.boolean(), given_name: z.string().nullable(), family_name: z.string().nullable(), @@ -330,6 +437,16 @@ export const webUserSchema = z service_access: z.boolean(), service_access_granted_at: z.string().nullable(), upload_access_requested_at: z.string().nullable(), + /** + * The two phase-8 additions (#1268, ADR 0045). REQUIRED here, unlike their + * `.optional()` twins on {@link userSchema}: that schema is what the CLI + * parses a possibly-older backend's `/users/me` with, while this one is + * only ever used to assert what THIS backend sends, and a `publicUser` + * that stopped sending either should fail the route test rather than + * quietly serve a dashboard that can no longer say what is missing. + */ + profile_gaps: z.array(profileGapSchema), + username_auto_assigned: z.boolean(), }) .passthrough(); export type WebUser = z.infer; diff --git a/src/commands/admin.ts b/src/commands/admin.ts index cfd86fed5..a12e1349f 100644 --- a/src/commands/admin.ts +++ b/src/commands/admin.ts @@ -6568,6 +6568,13 @@ backfillNamesCommand return; } + // Computed before the --json branch returns, so a scripted caller reading + // the exit code sees the same verdict as someone reading the summary below. + // A lookup failure leaves the row a candidate for the next run, and an + // unknown remainder means the batch's own bookkeeping failed. Neither is + // fatal, but a caller must not read either as a clean sweep. + if (res.lookup_failed > 0 || res.remaining === null) process.exitCode = 1; + if (options.json) { console.log(JSON.stringify(res, null, 2)); return; @@ -6612,10 +6619,6 @@ backfillNamesCommand ), ); } - // A lookup failure leaves the row a candidate for the next run, and an - // unknown remainder means the batch's own bookkeeping failed. Neither is - // fatal, but a caller must not read either as a clean sweep. - if (res.lookup_failed > 0 || res.remaining === null) process.exitCode = 1; }); adminCommand.addCommand(backfillNamesCommand); @@ -6660,6 +6663,31 @@ backfillUsernamesCommand return; } + // Computed before the --json branch returns, so a scripted caller reading + // the exit code sees the same verdict as someone reading the summary below. + // + // A lookup failure or a conflict leaves the row a candidate for the next + // run, an undelivered verify message leaves an account unable to finish + // onboarding, and an unknown remainder means the batch's own bookkeeping + // failed. None is fatal, but a caller must not read any of them as a clean + // sweep -- an unreported verify failure is exactly how "one message per + // account" quietly became "none" for the accounts it failed on. + // + // `exhausted` is in the list because it used to BE a conflict: splitting it + // out (#1268 review) must not quietly turn a run that finished nothing into + // a clean exit. It is the stronger case of the two -- a conflict clears + // itself on the next run and a saturated base never does. + if ( + res.lookup_failed > 0 || + res.conflict > 0 || + (res.exhausted ?? 0) > 0 || + (res.verify_failed ?? 0) > 0 || + (res.verify_rate_limited ?? 0) > 0 || + res.remaining === null + ) { + process.exitCode = 1; + } + if (options.json) { console.log(JSON.stringify(res, null, 2)); return; @@ -6675,7 +6703,7 @@ backfillUsernamesCommand res.apply ? res.assigned : res.would_assign } single_name=${res.single_name} no_name=${res.no_name} lookup_failed=${ res.lookup_failed - } conflict=${res.conflict} verify_sent=${res.verify_sent} verify_failed=${ + } conflict=${res.conflict} exhausted=${res.exhausted ?? 0} verify_sent=${res.verify_sent} verify_failed=${ res.verify_failed ?? 0 } verify_rate_limited=${res.verify_rate_limited ?? 0} remaining=${ res.remaining ?? "unknown" @@ -6703,6 +6731,10 @@ backfillUsernamesCommand ); } else if (r.outcome === "conflict") { console.log(` ${chalk.yellow("conflict ")} ${who}: ${r.error}`); + } else if (r.outcome === "exhausted") { + // Deliberately not folded in with `conflict` above: re-running the + // sweep fixes a conflict and can never fix this one. + console.log(` ${chalk.yellow("exhausted ")} ${who}: ${r.error}; pick a username by hand`); } else { console.log(` ${chalk.red("error ")} ${who}: ${r.error}`); } @@ -6730,21 +6762,6 @@ backfillUsernamesCommand ), ); } - // A lookup failure or a conflict leaves the row a candidate for the next - // run, an undelivered verify message leaves an account unable to finish - // onboarding, and an unknown remainder means the batch's own bookkeeping - // failed. None is fatal, but a caller must not read any of them as a clean - // sweep -- an unreported verify failure is exactly how "one message per - // account" quietly became "none" for the accounts it failed on. - if ( - res.lookup_failed > 0 || - res.conflict > 0 || - (res.verify_failed ?? 0) > 0 || - (res.verify_rate_limited ?? 0) > 0 || - res.remaining === null - ) { - process.exitCode = 1; - } }); adminCommand.addCommand(backfillUsernamesCommand); diff --git a/src/commands/auth.ts b/src/commands/auth.ts index b5bdfebae..59bc5ded7 100644 --- a/src/commands/auth.ts +++ b/src/commands/auth.ts @@ -20,10 +20,13 @@ import chalk from "chalk"; import { Command, InvalidArgumentError } from "commander"; import inquirer from "inquirer"; import ora from "ora"; +import { accountCopy } from "../../shared/contract/account-copy.js"; +import { resolveProfileGaps } from "../../shared/contract/profile-gaps.js"; import { UPLOAD_ACCESS_WHY_MAX_CHARS, UPLOAD_ACCESS_WHY_MIN_CHARS, } from "../../shared/contract/user.js"; +import { printGapList, printProfileGaps } from "../lib/account-gaps.js"; import { type OrcidNameResponse, type ProfilePatchRequest, @@ -664,6 +667,15 @@ export async function statusAction(options: { refresh?: boolean }): Promise !a.active); @@ -1273,34 +1299,6 @@ Examples: // for it. Before #1253 the 403 could only point at the support page. // ============================================================================ -/** - * Where a user actually fixes each precondition, keyed on the field names the - * API sends in `missing`. - * - * Most of these point at Settings on nemar.org because that is where the field - * lives: username, name, GitHub handle, city and country are edited on the - * account settings page (nemarOrg/website#301), and that page is what a user is - * meant to use whether they arrived from the browser or the CLI. THIS COPY - * ASSUMES #301 IS DEPLOYED -- releasing epic #1250 is gated on it, because - * until then these lines name a page with nothing on it, which is the exact - * failure ADR 0040 spent a year fixing. - * - * `email_verified` is the exception and points at a real CLI command, because - * that step genuinely has one. - */ -const UPLOAD_ACCESS_FIX: Record = { - email_verified: "run 'nemar auth resend-verification' and click the link in your inbox", - username: "run 'nemar auth profile set-username '", - given_name: - "comes from your ORCID record when one is linked; otherwise 'nemar auth profile set-name --given '", - family_name: - "comes from your ORCID record when one is linked; otherwise 'nemar auth profile set-name --family '", - github_username: "run 'nemar auth profile set-github '", - city: "run 'nemar auth profile set-location --city '", - country: "run 'nemar auth profile set-location --country '", - why: "pass a longer --why, or answer the prompt", -}; - /** * Say so when the request landed but no admin was reached. * @@ -1375,16 +1373,18 @@ const requestUploadAccessCmd = authCommand const spinner = ora("Submitting upload access request...").start(); try { const result = await requestUploadAccess(why); + // Both outcomes are stated in the website's own words (#1268, ADR 0045): + // a person who asked from the dashboard and then checked from a terminal + // must not be told two different things about one request. if (result.already_requested) { - spinner.info("You already have an open upload access request"); - console.log(chalk.dim(" An admin reviews it once; you will get an email when it lands.")); + spinner.info(accountCopy("upload_access.requested.title")); + console.log(chalk.dim(` ${accountCopy("upload_access.requested.lede")}`)); warnIfAdminsNotNotified(result); return; } - spinner.succeed("Upload access requested"); + spinner.succeed(accountCopy("upload_access.requested.title")); console.log(); - console.log(" A NEMAR admin reviews the request once."); - console.log(" You will get an email when upload access is granted."); + console.log(` ${accountCopy("upload_access.requested.body")}`); console.log(chalk.dim(" Check any time with 'nemar auth status --refresh'.")); warnIfAdminsNotNotified(result); } catch (error) { @@ -1400,12 +1400,17 @@ const requestUploadAccessCmd = authCommand // where to fix it rather than making the user map an error sentence back // onto a settings form (ADR 0042). if (error.missing && error.missing.length > 0) { - console.log(); - console.log(" Still needed:"); - for (const field of error.missing) { - const fix = UPLOAD_ACCESS_FIX[field] ?? "update it in Settings on nemar.org"; - console.log(` ${chalk.yellow(field)} — ${fix}`); - } + // `orcidVerified` from the config cache, because a refusal names the + // FIELD and not the account state (#1268). It is what decides whether + // the name halves point at `nemar auth profile set-name` or at the + // ORCID record -- with a verified iD linked the record owns the name + // and the PATCH refuses the edit, so the command would be advice that + // cannot work. Absent (never refreshed) reads as false, which is the + // pre-#1268 wording rather than a wrong one. + printGapList( + accountCopy("gaps.request.title"), + resolveProfileGaps(error.missing, { orcidVerified: getConfig().orcidVerified === true }), + ); console.log(); console.log(chalk.dim(" Settings: https://nemar.org/settings")); } @@ -1515,15 +1520,24 @@ export async function profileAction(): Promise { if (user.service_access === true) { console.log(` Upload access: ${chalk.green("granted")}`); } else if (user.service_access === false) { - console.log( - ` Upload access: ${chalk.yellow("not granted")} ${chalk.dim( - "(one-time admin approval; see https://nemar.org/support)", - )}`, - ); + console.log(` Upload access: ${chalk.yellow("not granted")}`); + console.log(` ${chalk.dim(accountCopy("upload_access.section.lede"))}`); + console.log(` ${chalk.cyan(accountCopy("cli.upload_access.cta"))}`); } else { console.log(` Upload access: ${chalk.dim("unknown (backend did not report it)")}`); } + // The same block `nemar auth status` prints, from a live fetch rather than + // from the cache. `undefined` here is a backend that predates phase 8, not a + // stale cache, so it says so rather than telling the user to refresh -- this + // command just did. + printProfileGaps({ + gaps: user.profile_gaps, + orcidVerified: user.orcid_verified === true, + sandboxCompleted: user.sandbox_completed, + unknownReason: accountCopy("cli.gaps.unreported"), + }); + console.log(); console.log(chalk.bold("Where to change each")); console.log(` Email ${chalk.cyan("nemar auth profile set-email
")}`); @@ -1654,6 +1668,7 @@ async function refreshStoredAccount(): Promise { setConfig("email", user.email); setConfig("githubUsername", user.github_username ?? undefined); if (user.service_access !== undefined) setConfig("serviceAccess", user.service_access); + if (user.orcid_verified !== undefined) setConfig("orcidVerified", user.orcid_verified); setConfig("role", user.role); } catch (error) { console.log( diff --git a/src/commands/dataset.ts b/src/commands/dataset.ts index efa5ce6d6..8401b4c77 100644 --- a/src/commands/dataset.ts +++ b/src/commands/dataset.ts @@ -26,6 +26,7 @@ import { Command, Option } from "commander"; import inquirer from "inquirer"; import ora from "ora"; import { formatBytesCli } from "../../shared/bytes.js"; +import { describeSandboxGap } from "../../shared/contract/profile-gaps.js"; import { LICENSE_TIERS } from "../../shared/license-tiers.js"; import { RangeParseError } from "../../shared/range.js"; import { addCi } from "../lib/api/admin.js"; @@ -191,6 +192,7 @@ import { showUploadPlan, } from "../lib/upload/plan.js"; import { + checkUploadAccessStep, checkUploadPrerequisites, validateBidsStep, verifyGhCli, @@ -546,15 +548,19 @@ Examples: // Step 1: Check authentication requireAuth(); - // Step 1b: Check sandbox training + // Step 1b: Check sandbox training. CLI-only, and stated in the one sentence + // every other gap is stated in (#1268, ADR 0045) -- see describeSandboxGap. if (!isSandboxCompleted()) { - console.log(chalk.yellow("Sandbox training required")); + console.log(chalk.yellow(describeSandboxGap())); console.log(); - console.log("You must complete sandbox training before uploading real datasets."); - console.log("This verifies your setup and familiarizes you with the workflow."); - console.log(); - console.log("Run sandbox training with:"); - console.log(chalk.cyan(" nemar sandbox")); + console.log("It verifies your setup and familiarizes you with the workflow."); + process.exit(1); + } + + // Step 1c: Upload access, before anything expensive runs. A missing grant + // is a hard stop for a real upload; --dry-run continues, since it uploads + // nothing and the plan is what the user asked to see. + if ((await checkUploadAccessStep({ dryRun: options.dryRun })).status === "fail") { process.exit(1); } @@ -565,7 +571,7 @@ Examples: process.exit(1); } - // Step 1c: Check required tools + // Step 1d: Check required tools await checkPrerequisitesForCommand("upload"); // Step 2: Check prerequisites diff --git a/src/lib/account-gaps.ts b/src/lib/account-gaps.ts new file mode 100644 index 000000000..f93b20436 --- /dev/null +++ b/src/lib/account-gaps.ts @@ -0,0 +1,127 @@ +/** + * How the terminal prints what an account is still missing (#1268, ADR 0045). + * + * Every sentence here comes out of `shared/contract/account-copy.ts` through + * `shared/contract/profile-gaps.ts`, the pair the website transcribes — so + * `nemar auth status`, `nemar auth profile`, the upload preflight, a refused + * upload-access request and the dashboard all say one thing about a missing + * GitHub handle rather than five near-misses. This module owns only the + * terminal's part: indentation, colour, and which list goes under which + * heading. + * + * NOTHING IS DERIVED HERE. The list of gaps is the backend's (`profile_gaps` on + * `/users/me`), because the rule that decides it is the same rule an + * upload-access request is refused by, and a CLI that recomputed it locally + * would be the second implementation phase 8 exists to delete. What the CLI + * adds is the one gap the backend has no business knowing about: sandbox + * training, which exists in this program and nowhere else. + */ + +import chalk from "chalk"; +import { ACCOUNT_COPY } from "../../shared/contract/account-copy.js"; +import { + type ProfileGapWireEntry, + type ResolvedProfileGap, + describeProfileGap, + describeSandboxGap, + profileGapTail, + resolveWireProfileGaps, +} from "../../shared/contract/profile-gaps.js"; + +/** The heading both `auth status` and `auth profile` put the list under. */ +export const PROFILE_SECTION_TITLE = ACCOUNT_COPY["cli.gaps.section.title"]; + +/** + * What the CLI knows about this account's gaps. + * + * `undefined` gaps and an EMPTY list are different answers and are printed + * differently: an absent list means nobody has asked the backend yet (or the + * backend predates phase 8), while an empty one is the backend saying nothing + * is missing. Collapsing them would tell someone their profile is complete on + * the strength of never having looked. + */ +export interface ProfileGapView { + /** The wire entries, or undefined when the list could not be read. + * + * Typed loosely on purpose, and typed loosely ONCE: the same view is built + * from a live `profile_gaps` and from the config cache, where the arrays + * round-trip through JSON and a vocabulary this build predates must survive + * rather than fail to parse. `ProfileGapWireEntry` is that shape's single + * declaration and `resolveWireProfileGaps` does the narrowing. */ + gaps: readonly ProfileGapWireEntry[] | undefined; + /** Moves the two name halves' "set it in" to the ORCID record. */ + orcidVerified?: boolean; + /** + * Sandbox training, the CLI-only step. `undefined` leaves it out entirely + * rather than assuming it is outstanding — an account that has never been + * refreshed has no cached answer, and inventing one would print a hard gate + * at somebody who has already passed it. + */ + sandboxCompleted?: boolean; + /** Printed instead of the list when `gaps` is undefined. */ + unknownReason?: string; +} + +/** Every line the block prints, without the heading — exported so a test can + * assert the sentences without parsing terminal output. */ +export function profileGapLines(view: ProfileGapView): string[] { + if (view.gaps === undefined) { + return [view.unknownReason ?? ACCOUNT_COPY["cli.gaps.unknown"]]; + } + const lines = resolveWireProfileGaps(view.gaps, { orcidVerified: view.orcidVerified }).map( + describeProfileGap, + ); + // Last, and deliberately so: everything above also blocks the person on the + // website, and this one is a step they take here. Nothing they can fix + // elsewhere goes below something only a terminal can finish. + if (view.sandboxCompleted === false) lines.push(describeSandboxGap()); + return lines.length > 0 ? lines : [ACCOUNT_COPY["gaps.none"]]; +} + +/** True when the view has an answer and the answer is "nothing outstanding". */ +function nothingOutstanding(view: ProfileGapView): boolean { + const lines = profileGapLines(view); + return lines.length === 1 && lines[0] === ACCOUNT_COPY["gaps.none"]; +} + +/** + * Print the "Profile" block: a heading, then one line per gap. + * + * `nemar auth status` and `nemar auth profile` call this with the same view and + * get the same block; the only difference between them is where the list came + * from — a cached refresh, versus the fetch `profile` always does. + * + * Only actual gaps are highlighted. "Nothing outstanding" and "not checked" are + * statements about the absence of work and are dimmed, because a yellow line + * saying nothing is wrong is how a person learns to stop reading yellow lines. + */ +export function printProfileGaps(view: ProfileGapView): void { + const quiet = view.gaps === undefined || nothingOutstanding(view); + console.log(); + console.log(chalk.bold(PROFILE_SECTION_TITLE)); + for (const line of profileGapLines(view)) { + console.log(` ${quiet ? chalk.dim(line) : chalk.yellow(line)}`); + } +} + +/** + * Print a bare gap list under a caller-supplied heading — a refused + * upload-access request, or the upload preflight. + * + * The heading is one of `gaps.request.title` / `gaps.upload.title`, and the + * lines are the same sentences the block above prints, with the label picked + * out so a list of six is scannable. That is what makes a refusal and a status + * nudge read alike rather than merely mean alike. + */ +export function printGapList( + title: string, + gaps: readonly ResolvedProfileGap[], + options: { indent?: string } = {}, +): void { + const indent = options.indent ?? " "; + console.log(); + console.log(`${indent}${title}`); + for (const gap of gaps) { + console.log(`${indent} ${chalk.yellow(gap.label)} ${profileGapTail(gap)}`); + } +} diff --git a/src/lib/api/admin.ts b/src/lib/api/admin.ts index 7f0d6f827..66cb7a407 100644 --- a/src/lib/api/admin.ts +++ b/src/lib/api/admin.ts @@ -1562,6 +1562,10 @@ export interface BackfillUsernamesResponse { no_name: number; lookup_failed: number; conflict: number; + /** Rows whose suggested base was saturated up to the suffix limit. Optional + * for the same reason the verify counters are: a backend that predates the + * split reports those rows inside `conflict`. */ + exhausted?: number; /** Verify-message outcomes, counted across BOTH passes. Optional so a * backend that predates the retry pass is rendered as unknown, not zero. */ verify_sent: number; diff --git a/src/lib/config.ts b/src/lib/config.ts index bf20f4bea..e398bd2a1 100644 --- a/src/lib/config.ts +++ b/src/lib/config.ts @@ -21,6 +21,7 @@ import { homedir } from "node:os"; import { join } from "node:path"; import Conf from "conf"; import { z } from "zod"; +import type { GapFieldsStayOptional } from "../../shared/contract/profile-gaps.js"; export const DEFAULT_API_URL = "https://api.nemar.org"; @@ -77,6 +78,33 @@ export function getConfigDir(): string { return process.env.NEMAR_CONFIG_DIR || join(homedir(), ".config", "nemar"); } +/** + * One cached `profile_gaps` entry, as it comes back off disk. + * + * Separate from the wire's `profileGapSchema` (shared/contract/user.ts) because + * it validates a different source -- a JSON file this or an older build wrote, + * not an HTTP response -- but it must stay loose in exactly the same way, so + * both are pinned to the one declaration of that shape below. + */ +const cachedProfileGapSchema = z + .object({ + field: z.string(), + blocks: z.array(z.string()).optional(), + set_on: z.array(z.string()).optional(), + }) + .passthrough(); + +/** A cache entry must stay something the gap renderer can take: neither + * `blocks` nor `set_on` may become required, or a cache file missing either + * key (an older build's, or one that never wrote it) would stop parsing + * instead of falling back to the matrix. Fails to compile, not throw at a + * user's terminal -- the runtime half of the same rule is + * test/contract-schemas.test.ts "a profile_gaps entry may carry only its + * field name". */ +export const _cachedProfileGapIsRenderable: GapFieldsStayOptional< + z.infer +> = true; + // Per-account configuration schema const accountSchema = z.object({ apiKey: z.string().optional(), @@ -94,6 +122,31 @@ const accountSchema = z.object({ * as unknown rather than guessing "not granted". */ serviceAccess: z.boolean().optional(), + /** + * What the account was still missing as of the last `auth status --refresh` + * (#1268, ADR 0045) — the backend's `profile_gaps`, cached beside + * `serviceAccess` and for the same reason: `auth status` stays usable + * offline, and absent means "never refreshed", which the Profile block + * reports as not-checked rather than as "nothing missing". + * + * Stored as the WIRE entries rather than as rendered sentences, so a CLI + * upgrade re-renders an old cache through its new copy table instead of + * replaying yesterday's wording. `blocks`/`set_on` are loose string arrays + * for the same reason they are on the wire: a vocabulary this build has not + * heard of must round-trip rather than fail to parse. + */ + profileGaps: z.array(cachedProfileGapSchema).optional(), + /** + * Whether a VERIFIED ORCID iD is linked, as of the last refresh (#1268). + * + * Cached for one reason: it decides where a missing NAME is set. With an iD + * linked the record owns the name and `PATCH /auth/profile` refuses the edit, + * so telling that person to run `nemar auth profile set-name` is advice that + * cannot work. A refused upload-access request names the field and not the + * account state, so the renderer has nowhere else to learn it. Absent + * defaults to false, which is the pre-#1268 wording. + */ + orcidVerified: z.boolean().optional(), /** * Cached from `/auth/login` and `auth status --refresh` (#1256). Not * authoritative -- always re-check with the backend for anything diff --git a/src/lib/upload/preflight.ts b/src/lib/upload/preflight.ts index f3eaa50a2..68bb6d322 100644 --- a/src/lib/upload/preflight.ts +++ b/src/lib/upload/preflight.ts @@ -1,11 +1,15 @@ /** - * Upload pipeline: preflight steps (prerequisites, GitHub CLI auth, BIDS - * validation). + * Upload pipeline: preflight steps (upload access, prerequisites, GitHub CLI + * auth, BIDS validation). * * Moved verbatim from the upload action in commands/dataset.ts (#907, * epic #902); the only intentional changes are import paths, the * step-function wrappers (process.exit -> return FAIL; the command * sequencer owns exits), and printStepFailure at the gh-auth failure site. + * + * `checkUploadAccessStep` is the one that was not moved from anywhere: it is + * new in #1268 (ADR 0045) and runs FIRST, because an account without the + * upload grant cannot finish this command however good the dataset is. */ import { existsSync } from "node:fs"; @@ -13,6 +17,11 @@ import { resolve } from "node:path"; import chalk from "chalk"; import ora from "ora"; import { formatBytesCli } from "../../../shared/bytes.js"; +import { accountCopy, fillCopy } from "../../../shared/contract/account-copy.js"; +import { resolveWireProfileGaps } from "../../../shared/contract/profile-gaps.js"; +import { printGapList } from "../account-gaps.js"; +import { getCurrentUser } from "../api/auth.js"; +import { errorDetail } from "../api/errors.js"; import { checkDenoInstalled, formatValidationResult, @@ -59,6 +68,102 @@ export async function detectVirtualMemoryLimit(): Promise { + const spinner = ora("Checking upload access...").start(); + + let user: Awaited>; + try { + user = await getCurrentUser(); + } catch (error) { + spinner.warn( + fillCopy(accountCopy("cli.upload.preflight.unchecked"), { reason: errorDetail(error) }), + ); + console.log(); + return ok(); + } + + if (user.service_access === true) { + spinner.succeed("Upload access granted"); + console.log(); + return ok(); + } + if (user.service_access === undefined) { + // Absent is not "no" (ADR 0040): telling someone who holds the grant that + // they do not is the one answer worth avoiding at any cost. + spinner.warn( + fillCopy(accountCopy("cli.upload.preflight.unchecked"), { + reason: "this backend does not report it", + }), + ); + console.log(); + return ok(); + } + + spinner.warn(accountCopy("cli.upload.preflight.title")); + console.log(); + console.log(` ${accountCopy("cli.upload.preflight.body")}`); + + if (user.profile_gaps === undefined) { + // Absent is not empty, the same three-state honesty `service_access` gets + // above: a backend that predates #1268 sends no list, and silence under a + // refusal reads as "nothing is missing". Say which it is, and where the + // list can still be had. + console.log(); + console.log(` ${chalk.dim(accountCopy("cli.upload.preflight.gaps_unknown"))}`); + } else { + // Only the gaps that stop the REQUEST. A gap that blocks publication and + // nothing else is real, and is not what this page is about; listing it here + // would make the shortest path to an upload look longer than it is. + const blocking = resolveWireProfileGaps(user.profile_gaps, { + orcidVerified: user.orcid_verified === true, + }).filter((gap) => gap.blocks.includes("upload_access")); + if (blocking.length > 0) { + printGapList(accountCopy("gaps.upload.title"), blocking); + } + } + console.log(); + console.log(` ${chalk.cyan(accountCopy("cli.upload_access.cta"))}`); + + if (options.dryRun) { + console.log(` ${chalk.dim(accountCopy("cli.upload.preflight.dry_run"))}`); + console.log(); + return ok(); + } + console.log(); + return FAIL; +} + /** Step 2: Check prerequisites (git-annex, GitHub SSH). */ export async function checkUploadPrerequisites(): Promise { const spinner = ora("Checking prerequisites...").start(); diff --git a/test/account-copy-parity.test.ts b/test/account-copy-parity.test.ts new file mode 100644 index 000000000..db2f8b402 --- /dev/null +++ b/test/account-copy-parity.test.ts @@ -0,0 +1,244 @@ +/** + * Drift check between `shared/contract/account-copy.ts` and the website's + * transcription of it (#1268, ADR 0045; nemarOrg/website#310). + * + * The two repos share no package, so the website's `src/lib/account-copy.ts` is + * a copy of this contract rather than an import of it. A copy with nothing + * watching it is a fork with a delay, which is what this test exists to + * prevent: every key that is NOT `cli.`-prefixed must exist there, carrying the + * same string. + * + * **It reads the website file as TEXT, and does not import it.** A dynamic + * import would drag that repo's module graph (astro, vite-resolved specifiers) + * in for a table of string constants and would fail for reasons that have + * nothing to do with drift. Text extraction has one requirement in exchange, + * documented in account-copy.ts and enforced below: every value is a plain + * string literal. + * + * **The extractor is proved on THIS file, which is the same shape.** A + * developer running the suite may have no website checkout at all, so the + * reader that will be pointed at a file they cannot see is first pointed at one + * they can — and those assertions run either way. + * + * **CI IS THE POINT, and it used to be excluded from it.** This block ended in + * an early `return` when no website checkout was found, so the parity claim was + * enforced on developer machines and nowhere else — a green required check that + * had never once compared the two files. The `unit-pure` job now sparse-checks + * out `nemarOrg/website`'s `src/lib/account-copy.ts` (public, no token) and + * names it in `NEMAR_WEBSITE_ACCOUNT_COPY`. + * + * So there are two modes, and the difference between them is deliberate: + * + * `NEMAR_WEBSITE_ACCOUNT_COPY` set that path IS the contract. A file that + * is not there FAILS, naming the path and + * the variable — because the only way to + * reach it is a checkout step that broke, + * and a silent skip would hand the check + * straight back to nobody. + * unset search the sibling checkouts a developer + * is likely to have. Absent is a VISIBLE + * skip (bun's `describe.skipIf`), not a + * pass: a skipped test is reported as + * skipped, and an early `return` is + * reported as success. + * + * The website's own `test/account-copy-drift.test.ts` is the mirror image of + * this file, and is getting the same treatment via `NEMAR_CLI_ACCOUNT_COPY`. + */ + +import { describe, expect, test } from "bun:test"; +import { existsSync, readFileSync } from "node:fs"; +import { isAbsolute, join } from "node:path"; +import { ACCOUNT_COPY } from "../shared/contract/account-copy.js"; +import { + UPLOAD_ACCESS_WHY_MAX_CHARS, + UPLOAD_ACCESS_WHY_MIN_CHARS, +} from "../shared/contract/user.js"; + +const REPO_ROOT = join(import.meta.dir, ".."); +const CONTRACT_FILE = join(REPO_ROOT, "shared", "contract", "account-copy.ts"); + +/** + * Where a website checkout is expected to sit relative to this one. + * + * Both repos live under one parent directory, and this repo is often a + * WORKTREE (`account-tiers-phase8/`, `epic-account-tiers/`, ...) rather than + * `nemar-cli/` itself — so the search is one level up from the repo root + * whatever the root is called. + * + * Only consulted when {@link WEBSITE_ENV_VAR} is unset; CI names the file + * outright rather than hoping it landed somewhere on this list. + */ +const WEBSITE_CANDIDATES = [ + join(REPO_ROOT, "..", "website", "src", "lib", "account-copy.ts"), + join(REPO_ROOT, "..", "..", "website", "src", "lib", "account-copy.ts"), +]; + +/** Set by the `unit-pure` job to the sparse checkout it made. A relative value + * is resolved against the repo root, which is the workflow's working + * directory. */ +const WEBSITE_ENV_VAR = "NEMAR_WEBSITE_ACCOUNT_COPY"; +const declaredRaw = process.env[WEBSITE_ENV_VAR]; +const DECLARED_PATH = + declaredRaw && declaredRaw.trim() !== "" + ? isAbsolute(declaredRaw) + ? declaredRaw + : join(REPO_ROOT, declaredRaw) + : null; + +/** + * Pull `"key": "value"` pairs out of a TypeScript source file. + * + * Byte-for-byte the extractor the website's drift test uses (its + * `test/copy-drift.ts`), so the two halves of the mirror cannot disagree about + * what they are comparing. Comments are stripped first so a key quoted in prose + * cannot become an entry; a value that is not a plain string literal is + * invisible to it, which is why the literal-only rule exists and is asserted + * below. + */ +function extractCopy(source: string): Map { + const withoutComments = source.replace(/\/\*[\s\S]*?\*\//g, " ").replace(/^[ \t]*\/\/.*$/gm, " "); + const entries = new Map(); + const pattern = /"([A-Za-z0-9_.]+)"\s*:\s*"((?:[^"\\]|\\.)*)"/g; + for (const match of withoutComments.matchAll(pattern)) { + entries.set(match[1], match[2].replace(/\\(.)/g, "$1")); + } + return entries; +} + +/** Keys the website is expected to mirror: everything without the CLI prefix. */ +const SHARED_KEYS = Object.keys(ACCOUNT_COPY).filter((key) => !key.startsWith("cli.")); + +describe("the copy table's own rules", () => { + test("every value is a plain string literal, so the drift check can see it", () => { + // The failure this prevents: a template literal or a concatenation is + // extracted as NOTHING, so a silently unread key becomes a silently + // unchecked key on both sides of the mirror. + const extracted = extractCopy(readFileSync(CONTRACT_FILE, "utf8")); + expect(Object.fromEntries(extracted)).toEqual({ ...ACCOUNT_COPY }); + }); + + test("the extractor ignores a key quoted inside a comment", () => { + const source = [ + '/* the key "tier.base.label": "Not this one" appears in prose */', + '// "tier.base.label": "Nor this one"', + 'const X = { "tier.base.label": "Base access" };', + ].join("\n"); + expect(extractCopy(source).get("tier.base.label")).toBe("Base access"); + expect(extractCopy(source).size).toBe(1); + }); + + test("the extractor invents nothing for a computed value", () => { + const source = 'const X = { "a.b": `${MIN}-${MAX} chars`, "c.d": "plain" };'; + expect(extractCopy(source).has("a.b")).toBe(false); + expect(extractCopy(source).get("c.d")).toBe("plain"); + }); + + test("the why-hint's written-out bounds still match the constants", () => { + // Rule 1 forbids interpolating them, so this is what keeps the sentence + // honest when the bounds move. + expect(ACCOUNT_COPY["upload_access.request.why_hint"]).toBe( + `Describe what you intend to upload in ${UPLOAD_ACCESS_WHY_MIN_CHARS}-${UPLOAD_ACCESS_WHY_MAX_CHARS} characters`, + ); + }); + + test("every placeholder comes from the declared vocabulary", () => { + const allowed = new Set(["label", "blocks", "web", "cli", "date", "reason"]); + for (const [key, value] of Object.entries(ACCOUNT_COPY)) { + for (const [, name] of value.matchAll(/\{(\w+)\}/g)) { + expect({ key, name, allowed: [...allowed] }).toEqual({ + key, + name: allowed.has(name) ? name : `UNDECLARED:${name}`, + allowed: [...allowed], + }); + } + } + }); + + test("keys are dotted lowercase tokens, which is what makes them extractable", () => { + for (const key of Object.keys(ACCOUNT_COPY)) { + expect(key).toMatch(/^[a-z0-9_]+(\.[a-z0-9_]+)+$/); + } + }); +}); + +/** Where the website's table is, or null when nobody has told us and none of + * the usual sibling checkouts is present. A DECLARED path counts as found + * whether or not it exists — the existence check is a failing assertion, not a + * reason to skip. */ +const FOUND = DECLARED_PATH ?? WEBSITE_CANDIDATES.find((path) => existsSync(path)) ?? null; +const NO_CHECKOUT = FOUND === null; + +if (NO_CHECKOUT) { + console.info( + `[account-copy parity] skipping: no website checkout at ${WEBSITE_CANDIDATES.join(" or ")}, ` + + `and ${WEBSITE_ENV_VAR} is unset. CI sets it; this is the developer-machine path.`, + ); +} + +/** + * The website's table, or a failure that says exactly what to fix. + * + * The throw is the whole difference between this file and the version that + * silently returned: when CI has named a path, a file that is not there means + * the checkout step broke, and that must fail the job rather than quietly + * disable the check it exists to run. + */ +function websiteCopy(): Map { + if (FOUND === null) throw new Error("unreachable: the parity block is skipped without a file"); + if (!existsSync(FOUND)) { + throw new Error( + `${WEBSITE_ENV_VAR}=${declaredRaw} resolves to ${FOUND}, which does not exist. ` + + "That variable is the contract: CI points it at the sparse checkout of " + + "nemarOrg/website made by the unit-pure job. If that step changed or was removed, " + + "fix the workflow -- do not unset the variable to make this pass.", + ); + } + return extractCopy(readFileSync(FOUND, "utf8")); +} + +describe.skipIf(NO_CHECKOUT)("website parity", () => { + const found = FOUND as string; + + test("every shared key matches the website string for string", () => { + const theirs = websiteCopy(); + const drifted: string[] = []; + const absent: string[] = []; + for (const key of SHARED_KEYS) { + const mine = ACCOUNT_COPY[key as keyof typeof ACCOUNT_COPY]; + const yours = theirs.get(key); + if (yours === undefined) absent.push(key); + else if (yours !== mine) { + drifted.push(`${key}\n nemar-cli: ${mine}\n website: ${yours}`); + } + } + expect( + drifted, + `Copy drifted from ${found}. This contract is the source of truth; update the website.`, + ).toEqual([]); + expect( + absent, + `Keys absent from ${found}. Every non-cli key here is meant to be mirrored there.`, + ).toEqual([]); + }); + + test("nothing the website carries is missing here", () => { + // The other direction, and the one this repo can actually fix: a key the + // website has and this contract does not is a sentence with no source of + // truth. The website's own drift test fails on exactly this. + const theirs = websiteCopy(); + const unmirrored = [...theirs.keys()].filter((key) => !(key in ACCOUNT_COPY)); + expect(unmirrored, "Add these to shared/contract/account-copy.ts.").toEqual([]); + }); + + test("CLI-only keys are prefixed, and are the only ones the website may lack", () => { + // Rule 4. Enforced here rather than trusted: an unprefixed key the website + // has no surface for would fail its drift test in the other repo, where + // whoever added it is not looking. + const cliOnly = Object.keys(ACCOUNT_COPY).filter((key) => key.startsWith("cli.")); + expect(cliOnly.length).toBeGreaterThan(0); + const theirs = websiteCopy(); + for (const key of cliOnly) expect(theirs.has(key)).toBe(false); + for (const key of SHARED_KEYS) expect(theirs.has(key)).toBe(true); + }); +}); diff --git a/test/admin-backfill-usernames-cli.test.ts b/test/admin-backfill-usernames-cli.test.ts index 3aceef6c3..ac0759fc8 100644 --- a/test/admin-backfill-usernames-cli.test.ts +++ b/test/admin-backfill-usernames-cli.test.ts @@ -218,6 +218,44 @@ describe("nemar admin backfill-usernames", () => { } }); + test("a saturated base is rendered apart from a conflict, and still exits 1", async () => { + // `exhausted` used to arrive as `conflict`, which the exit-code rule + // already treats as unfinished work. Splitting it out must not quietly + // turn a run that assigned nothing into a clean exit, and the line must + // say what a retry cannot fix. + seedAuthenticatedConfig(); + const server = startServer({ + ...DRY_RUN_REPLY, + scanned: 1, + would_assign: 0, + exhausted: 1, + remaining: 1, + results: [ + { + id: 25, + email: "saturated@example.org", + orcid: null, + outcome: "exhausted", + given_name: "Ada", + family_name: "Lovelace", + verify: "not_attempted", + error: "No free variant of 'alovelace' within the suffix limit", + }, + ], + }); + try { + const result = await runCli(["admin", "backfill-usernames"], server.url); + expect(result.exitCode).toBe(1); + expect(result.stdout).toContain("exhausted "); + expect(result.stdout).toContain("No free variant of 'alovelace' within the suffix limit"); + expect(result.stdout).toContain("pick a username by hand"); + // And it is counted in its own column, not folded into conflict=. + expect(result.stdout).toContain("conflict=0 exhausted=1"); + } finally { + server.stop(); + } + }); + test("a run that left work behind exits non-zero", async () => { seedAuthenticatedConfig(); const server = startServer({ @@ -362,4 +400,38 @@ describe("nemar admin backfill-usernames", () => { server.stop(); } }); + + test("--json exits non-zero when the response carries exhausted work", async () => { + // The exit code used to be computed AFTER the --json branch's early + // return, so a scripted caller reading `--json` output saw exit 0 on a run + // that left exhausted or conflicting rows behind. The two output modes + // must agree. + seedAuthenticatedConfig(); + const server = startServer({ + ...DRY_RUN_REPLY, + scanned: 1, + would_assign: 0, + exhausted: 1, + remaining: 1, + results: [ + { + id: 25, + email: "saturated@example.org", + orcid: null, + outcome: "exhausted", + given_name: "Ada", + family_name: "Lovelace", + verify: "not_attempted", + error: "No free variant of 'alovelace' within the suffix limit", + }, + ], + }); + try { + const result = await runCli(["admin", "backfill-usernames", "--json"], server.url); + expect(result.exitCode).toBe(1); + expect(JSON.parse(result.stdout)).toMatchObject({ exhausted: 1 }); + } finally { + server.stop(); + } + }); }); diff --git a/test/auth-gaps-cli.test.ts b/test/auth-gaps-cli.test.ts new file mode 100644 index 000000000..ca5d0f954 --- /dev/null +++ b/test/auth-gaps-cli.test.ts @@ -0,0 +1,454 @@ +/** + * What the terminal actually prints about a missing profile field (#1268, + * ADR 0045), driven through the real entry point (`bun run src/index.ts ...`). + * + * The sentences are asserted VERBATIM, and that is the point of the file: they + * are the strings `shared/contract/account-copy.ts` holds and the website + * transcribes, so a CLI that started paraphrasing them would be the drift phase + * 8 exists to prevent — and paraphrasing is exactly what a test asserting + * `toContain("username")` would let through. + * + * Harness mirrors test/auth-status-upload-access.test.ts: a real subprocess CLI + * invocation against a real local HTTP server via TEST_API_URL, an isolated + * NEMAR_CONFIG_DIR, and a real on-disk config store. No mocks. + */ + +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { spawn } from "bun"; + +const CLI_ENTRY = join(import.meta.dir, "..", "src", "index.ts"); +const REPO_ROOT = join(import.meta.dir, ".."); + +/** The exact sentences. Spelled out rather than imported so this file fails + * when the copy table changes, instead of following it. */ +const USERNAME_LINE = + "Username is missing: needed to request upload access. Set it in Settings or run `nemar auth profile set-username`."; +const GITHUB_LINE = + "GitHub handle is missing: needed to request upload access. Set it in Settings or run `nemar auth profile set-github`."; +const GIVEN_ORCID_LINE = + "Given name is missing: needed to request upload access. Set it in your ORCID record at orcid.org, then sign in again."; +const SANDBOX_LINE = + "Sandbox training is missing: needed to upload a dataset from the CLI. Run `nemar sandbox`."; +const NOTHING_LINE = "Nothing outstanding — every field NEMAR needs is filled in."; +const NOT_CHECKED_LINE = "not checked — run 'nemar auth status --refresh'"; +const UNREPORTED_LINE = "not reported by this backend"; +const PREFLIGHT_TITLE = "Upload access is not granted yet"; +const PREFLIGHT_CTA = "Ask for it with `nemar auth request-upload-access`."; +const PREFLIGHT_DRY_RUN = "Continuing anyway: --dry-run uploads nothing."; +const UPLOAD_GAPS_TITLE = "Before you can ask, your account still needs:"; +const PREFLIGHT_GAPS_UNKNOWN = + "Could not determine what is still missing; run `nemar auth profile` for the list."; +/** Printed by the step AFTER the preflight, so it is how a test proves the + * pipeline continued rather than stopped. */ +const CONTINUED_MARKER = "Path does not exist"; + +interface UserOptions { + service_access?: boolean; + profile_gaps?: { field: string; blocks: string[]; set_on: string[] }[] | undefined; + orcid_verified?: boolean; + sandbox_completed?: boolean; +} + +/** Serves the real /users/me envelope shape: `{ user, token }`. */ +function startMeServer(options: UserOptions = {}) { + const server = Bun.serve({ + port: 0, + fetch(req) { + const url = new URL(req.url); + if (url.pathname === "/notices") return Response.json({ notices: [] }); + if (url.pathname === "/datasets/facets") return Response.json({}); + return Response.json({ + user: { + id: 12, + username: "harlow", + email: "harlow@example.org", + github_username: "harlow-gh", + role: "member", + orcid: "0000-0002-1825-0097", + orcid_verified: options.orcid_verified ?? false, + status: "verified", + email_verified: true, + given_name: "Ada", + family_name: "Harlow", + created_at: "2026-05-01T00:00:00Z", + approved_at: null, + dataset_count: 0, + sandbox_completed: options.sandbox_completed ?? true, + sandbox_completed_at: null, + sandbox_dataset_id: null, + username_auto_assigned: false, + ...(options.service_access === undefined + ? {} + : { service_access: options.service_access }), + ...(options.profile_gaps === undefined ? {} : { profile_gaps: options.profile_gaps }), + }, + token: null, + }); + }, + }); + return { url: `http://localhost:${server.port}`, stop: () => server.stop(true) }; +} + +/** A backend that is up but broken: /users/me 500s. Neither 401 nor 403. */ +function startBrokenMeServer() { + const server = Bun.serve({ + port: 0, + fetch(req) { + const url = new URL(req.url); + if (url.pathname === "/notices") return Response.json({ notices: [] }); + if (url.pathname === "/datasets/facets") return Response.json({}); + return Response.json({ error: "Database is having a moment" }, { status: 500 }); + }, + }); + return { url: `http://localhost:${server.port}`, stop: () => server.stop(true) }; +} + +const GAP = (field: string, blocks: string[], setOn = ["web", "cli"]) => ({ + field, + blocks, + set_on: setOn, +}); + +let configDir: string; + +function seedAuthenticatedConfig(extra: Record = {}): void { + writeFileSync( + join(configDir, "config.json"), + JSON.stringify({ + activeAccount: "harlow", + accounts: { harlow: { apiKey: "test-user-key", username: "harlow", ...extra } }, + }), + ); +} + +function storedAccount(): Record { + return JSON.parse(readFileSync(join(configDir, "config.json"), "utf8")).accounts.harlow; +} + +beforeEach(() => { + configDir = mkdtempSync(join(tmpdir(), "nemar-auth-gaps-cli-")); +}); + +afterEach(() => { + rmSync(configDir, { recursive: true, force: true }); +}); + +async function runCli(args: string[], testApiUrl: string) { + const env = { + ...process.env, + NEMAR_CONFIG_DIR: configDir, + TEST_API_URL: testApiUrl, + NEMAR_NO_UPDATE_CHECK: "1", + NO_COLOR: "1", + }; + env.FORCE_COLOR = undefined; + env.CLICOLOR_FORCE = undefined; + const proc = spawn({ + cmd: ["bun", "run", CLI_ENTRY, ...args], + cwd: REPO_ROOT, + env, + stdin: "ignore", + stdout: "pipe", + stderr: "pipe", + }); + const stdout = await new Response(proc.stdout).text(); + const stderr = await new Response(proc.stderr).text(); + const exitCode = await proc.exited; + return { stdout, stderr, exitCode, all: `${stdout}${stderr}` }; +} + +describe("nemar auth status: the Profile block", () => { + test("--refresh prints one exact sentence per gap, and caches them", async () => { + seedAuthenticatedConfig(); + const server = startMeServer({ + service_access: false, + profile_gaps: [ + GAP("username", ["upload_access"]), + GAP("github_username", ["upload_access", "publication"]), + ], + }); + try { + const result = await runCli(["auth", "status", "--refresh"], server.url); + expect(result.exitCode).toBe(0); + expect(result.stdout).toContain("Profile"); + expect(result.stdout).toContain(USERNAME_LINE); + expect(result.stdout).toContain(GITHUB_LINE); + expect(storedAccount().profileGaps).toEqual([ + { field: "username", blocks: ["upload_access"], set_on: ["web", "cli"] }, + { + field: "github_username", + blocks: ["upload_access", "publication"], + set_on: ["web", "cli"], + }, + ]); + } finally { + server.stop(); + } + }); + + test("the cached list is what a later offline status prints", async () => { + // No server: the second run does not refresh, so it can only be reading + // what the first run wrote. + seedAuthenticatedConfig({ profileGaps: [GAP("username", ["upload_access"])] }); + const result = await runCli(["auth", "status"], "http://127.0.0.1:1"); + expect(result.exitCode).toBe(0); + expect(result.stdout).toContain(USERNAME_LINE); + }); + + test("an empty list says so, and is not confused with never having looked", async () => { + seedAuthenticatedConfig(); + const server = startMeServer({ service_access: true, profile_gaps: [] }); + try { + const result = await runCli(["auth", "status", "--refresh"], server.url); + expect(result.stdout).toContain(NOTHING_LINE); + expect(result.stdout).not.toContain(NOT_CHECKED_LINE); + } finally { + server.stop(); + } + }); + + test("a never-refreshed account reports not-checked rather than 'nothing missing'", async () => { + seedAuthenticatedConfig(); + const result = await runCli(["auth", "status"], "http://127.0.0.1:1"); + expect(result.stdout).toContain(NOT_CHECKED_LINE); + expect(result.stdout).not.toContain(NOTHING_LINE); + }); + + test("a failed refresh reports not-checked instead of a stale list", async () => { + // The cache says one thing; the refresh the user asked for did not land. + // Printing the cache would present a claim about the past as the present. + seedAuthenticatedConfig({ profileGaps: [GAP("username", ["upload_access"])] }); + const server = startBrokenMeServer(); + try { + const result = await runCli(["auth", "status", "--refresh"], server.url); + expect(result.stdout).toContain(NOT_CHECKED_LINE); + expect(result.stdout).not.toContain(USERNAME_LINE); + // ...and the cache is not overwritten, so an offline run still has it. + expect(storedAccount().profileGaps).toHaveLength(1); + } finally { + server.stop(); + } + }); + + test("a backend that omits profile_gaps leaves the cache alone", async () => { + seedAuthenticatedConfig({ profileGaps: [GAP("city", ["upload_access"])] }); + const server = startMeServer({ service_access: false, profile_gaps: undefined }); + try { + await runCli(["auth", "status", "--refresh"], server.url); + expect(storedAccount().profileGaps).toEqual([ + { field: "city", blocks: ["upload_access"], set_on: ["web", "cli"] }, + ]); + } finally { + server.stop(); + } + }); + + test("sandbox training is listed last, and only when it is outstanding", async () => { + seedAuthenticatedConfig(); + const server = startMeServer({ + service_access: false, + sandbox_completed: false, + profile_gaps: [GAP("username", ["upload_access"])], + }); + try { + const result = await runCli(["auth", "status", "--refresh"], server.url); + expect(result.stdout).toContain(SANDBOX_LINE); + expect(result.stdout.indexOf(SANDBOX_LINE)).toBeGreaterThan( + result.stdout.indexOf(USERNAME_LINE), + ); + } finally { + server.stop(); + } + + const done = startMeServer({ + service_access: false, + sandbox_completed: true, + profile_gaps: [GAP("username", ["upload_access"])], + }); + try { + const result = await runCli(["auth", "status", "--refresh"], done.url); + expect(result.stdout).not.toContain(SANDBOX_LINE); + } finally { + done.stop(); + } + }); + + test("whoami prints the same block (it is the same action)", async () => { + seedAuthenticatedConfig({ profileGaps: [GAP("username", ["upload_access"])] }); + const result = await runCli(["auth", "whoami"], "http://127.0.0.1:1"); + expect(result.stdout).toContain(USERNAME_LINE); + }); +}); + +describe("nemar auth profile", () => { + test("prints the same sentences, from a live fetch", async () => { + seedAuthenticatedConfig(); + const server = startMeServer({ + service_access: false, + profile_gaps: [GAP("github_username", ["upload_access", "publication"])], + }); + try { + const result = await runCli(["auth", "profile"], server.url); + expect(result.exitCode).toBe(0); + expect(result.stdout).toContain(GITHUB_LINE); + } finally { + server.stop(); + } + }); + + test("a verified ORCID iD points the name at orcid.org, not at Settings", async () => { + seedAuthenticatedConfig(); + const server = startMeServer({ + service_access: false, + orcid_verified: true, + profile_gaps: [GAP("given_name", ["upload_access", "publication"], ["web"])], + }); + try { + const result = await runCli(["auth", "profile"], server.url); + expect(result.stdout).toContain(GIVEN_ORCID_LINE); + expect(result.stdout).not.toContain("nemar auth profile set-name`"); + } finally { + server.stop(); + } + }); + + test("an older backend is told apart from a complete profile", async () => { + // `profile` always fetches, so "run --refresh" would be advice it just + // followed. What is missing is the field, not the fetch. + seedAuthenticatedConfig(); + const server = startMeServer({ service_access: false, profile_gaps: undefined }); + try { + const result = await runCli(["auth", "profile"], server.url); + expect(result.stdout).toContain(UNREPORTED_LINE); + expect(result.stdout).not.toContain(NOTHING_LINE); + } finally { + server.stop(); + } + }); +}); + +describe("nemar dataset upload: the upload-access preflight", () => { + const NO_SUCH_PATH = join(tmpdir(), "nemar-no-such-dataset-dir"); + + test("a missing grant stops the run before anything else happens", async () => { + seedAuthenticatedConfig({ sandboxCompleted: true }); + const server = startMeServer({ + service_access: false, + profile_gaps: [ + GAP("username", ["upload_access"]), + GAP("github_username", ["upload_access", "publication"]), + ], + }); + try { + const result = await runCli(["dataset", "upload", NO_SUCH_PATH], server.url); + expect(result.exitCode).toBe(1); + expect(result.all).toContain(PREFLIGHT_TITLE); + expect(result.all).toContain(UPLOAD_GAPS_TITLE); + expect(result.all).toContain(USERNAME_LINE); + expect(result.all).toContain(GITHUB_LINE); + expect(result.all).toContain(PREFLIGHT_CTA); + // It really stopped: the very next step never ran. + expect(result.all).not.toContain(CONTINUED_MARKER); + } finally { + server.stop(); + } + }); + + test("a publication-only gap is not listed as something blocking the request", async () => { + seedAuthenticatedConfig({ sandboxCompleted: true }); + const server = startMeServer({ + service_access: false, + profile_gaps: [GAP("given_name", ["publication"])], + }); + try { + const result = await runCli(["dataset", "upload", NO_SUCH_PATH], server.url); + expect(result.all).toContain(PREFLIGHT_TITLE); + expect(result.all).not.toContain(UPLOAD_GAPS_TITLE); + } finally { + server.stop(); + } + }); + + test("a backend that reports no gap list says so instead of nothing", async () => { + // Not granted, and the list itself is absent -- an older backend. Printing + // the refusal with no reasons under it reads as "nothing is missing", which + // is a claim this build cannot make. `service_access` already distinguishes + // absent from false; so must this. + seedAuthenticatedConfig({ sandboxCompleted: true }); + const server = startMeServer({ service_access: false, profile_gaps: undefined }); + try { + const result = await runCli(["dataset", "upload", NO_SUCH_PATH], server.url); + expect(result.exitCode).toBe(1); + expect(result.all).toContain(PREFLIGHT_TITLE); + expect(result.all).toContain(PREFLIGHT_GAPS_UNKNOWN); + // Not the list heading: there is no list. + expect(result.all).not.toContain(UPLOAD_GAPS_TITLE); + } finally { + server.stop(); + } + }); + + test("--dry-run warns and continues, because it uploads nothing", async () => { + seedAuthenticatedConfig({ sandboxCompleted: true }); + const server = startMeServer({ service_access: false, profile_gaps: [] }); + try { + const result = await runCli(["dataset", "upload", NO_SUCH_PATH, "--dry-run"], server.url); + expect(result.all).toContain(PREFLIGHT_TITLE); + expect(result.all).toContain(PREFLIGHT_DRY_RUN); + expect(result.all).toContain(CONTINUED_MARKER); + } finally { + server.stop(); + } + }); + + test("a granted account passes straight through", async () => { + seedAuthenticatedConfig({ sandboxCompleted: true }); + const server = startMeServer({ service_access: true, profile_gaps: [] }); + try { + const result = await runCli(["dataset", "upload", NO_SUCH_PATH], server.url); + expect(result.all).toContain("Upload access granted"); + expect(result.all).not.toContain(PREFLIGHT_TITLE); + expect(result.all).toContain(CONTINUED_MARKER); + } finally { + server.stop(); + } + }); + + test("an unreachable backend warns and continues rather than inventing a refusal", async () => { + // The server enforces the gate regardless; failing an upload because a + // status endpoint was briefly down would refuse something nobody refused. + seedAuthenticatedConfig({ sandboxCompleted: true }); + const result = await runCli(["dataset", "upload", NO_SUCH_PATH], "http://127.0.0.1:1"); + expect(result.all).toContain("Upload access could not be checked"); + expect(result.all).toContain(CONTINUED_MARKER); + }); + + test("a backend that does not report the grant is not treated as a refusal", async () => { + seedAuthenticatedConfig({ sandboxCompleted: true }); + const server = startMeServer({ service_access: undefined }); + try { + const result = await runCli(["dataset", "upload", NO_SUCH_PATH], server.url); + expect(result.all).toContain("this backend does not report it"); + expect(result.all).toContain(CONTINUED_MARKER); + } finally { + server.stop(); + } + }); + + test("sandbox training is still the CLI-only gate, in the shared sentence", async () => { + seedAuthenticatedConfig({ sandboxCompleted: false }); + const server = startMeServer({ service_access: true, profile_gaps: [] }); + try { + const result = await runCli(["dataset", "upload", NO_SUCH_PATH], server.url); + expect(result.exitCode).toBe(1); + expect(result.all).toContain(SANDBOX_LINE); + // Ahead of the upload-access check: it costs no network call. + expect(result.all).not.toContain("Upload access granted"); + } finally { + server.stop(); + } + }); +}); diff --git a/test/auth-request-upload-access-cli.test.ts b/test/auth-request-upload-access-cli.test.ts index a75766d64..04ace14c9 100644 --- a/test/auth-request-upload-access-cli.test.ts +++ b/test/auth-request-upload-access-cli.test.ts @@ -51,12 +51,12 @@ function startServer(reply: { status: number; body: unknown }): CaptureServer { let configDir: string; -function seedAuthenticatedConfig(): void { +function seedAuthenticatedConfig(extra: Record = {}): void { writeFileSync( join(configDir, "config.json"), JSON.stringify({ activeAccount: "requester", - accounts: { requester: { apiKey: "test-user-key" } }, + accounts: { requester: { apiKey: "test-user-key", ...extra } }, }), ); } @@ -106,8 +106,13 @@ describe("nemar auth request-upload-access", () => { expect(server.bodies).toEqual([{ why: WHY }]); // ora writes its spinner lines to stderr, so success is asserted over // the combined output. - expect(`${result.stdout}${result.stderr}`).toContain("Upload access requested"); - expect(result.stdout).toContain("reviews the request once"); + // The website's own sentences since #1268 (ADR 0045): someone who asked + // from the dashboard and checks from a terminal must not be told two + // different things about one request. + expect(`${result.stdout}${result.stderr}`).toContain("Your request is with an admin"); + expect(result.stdout).toContain( + "We\u2019ll email you when there\u2019s a decision. Everything else on your account keeps working in the meantime.", + ); } finally { server.stop(); } @@ -122,9 +127,7 @@ describe("nemar auth request-upload-access", () => { try { const result = await runCli(["auth", "request-upload-access", "--why", WHY], server.url); expect(result.exitCode).toBe(0); - expect(`${result.stdout}${result.stderr}`).toContain( - "already have an open upload access request", - ); + expect(`${result.stdout}${result.stderr}`).toContain("Your request is with an admin"); } finally { server.stop(); } @@ -198,9 +201,19 @@ describe("nemar auth request-upload-access", () => { // fields is settable from the CLI, so the fix names the command -- // sending someone to a browser for a field they can set from the shell // they are already in is the dead end ADR 0042 was left with. - expect(out).toContain("nemar auth profile set-username "); - expect(out).toContain("nemar auth profile set-location --city "); - expect(out).toContain("nemar auth profile set-location --country "); + // + // Since #1268 the WORDING is shared/contract/account-copy.ts and is the + // sentence the website prints for the same gap (ADR 0045), which is why + // the command appears bare rather than with an argument placeholder. + expect(out).toContain( + "Username is missing: needed to request upload access. Set it in Settings or run `nemar auth profile set-username`.", + ); + expect(out).toContain( + "City is missing: needed to request upload access. Set it in Settings or run `nemar auth profile set-location`.", + ); + expect(out).toContain( + "Country is missing: needed to request upload access. Set it in Settings or run `nemar auth profile set-location`.", + ); // The website is still the other half of the same rules. expect(out).toContain("https://nemar.org/settings"); } finally { @@ -208,6 +221,52 @@ describe("nemar auth request-upload-access", () => { } }); + test("a verified ORCID iD sends the name gap to orcid.org, not to a command", async () => { + // The refusal names the FIELD, not the account state, so the renderer reads + // `orcidVerified` from the config cache (#1268). With an iD linked the + // record owns the name and `PATCH /auth/profile` refuses the edit -- naming + // `nemar auth profile set-name` here would be advice that cannot work. + seedAuthenticatedConfig({ orcidVerified: true }); + const server = startServer({ + status: 400, + body: { + error: "profile_incomplete", + message: "Complete your profile before requesting upload access: given_name", + missing: ["given_name"], + }, + }); + try { + const result = await runCli(["auth", "request-upload-access", "--why", WHY], server.url); + const out = `${result.stdout}${result.stderr}`; + expect(out).toContain( + "Given name is missing: needed to request upload access. Set it in your ORCID record at orcid.org, then sign in again.", + ); + expect(out).not.toContain("nemar auth profile set-name"); + } finally { + server.stop(); + } + }); + + test("without a linked iD the same gap names the command", async () => { + seedAuthenticatedConfig(); + const server = startServer({ + status: 400, + body: { + error: "profile_incomplete", + message: "Complete your profile before requesting upload access: given_name", + missing: ["given_name"], + }, + }); + try { + const result = await runCli(["auth", "request-upload-access", "--why", WHY], server.url); + expect(`${result.stdout}${result.stderr}`).toContain( + "Given name is missing: needed to request upload access. Set it in Settings or run `nemar auth profile set-name`.", + ); + } finally { + server.stop(); + } + }); + test("an unverified inbox is pointed at the CLI command that fixes it", async () => { // The one precondition the CLI CAN act on today, so it must not be swept // into the generic "go to Settings" advice. diff --git a/test/auth-status-upload-access.test.ts b/test/auth-status-upload-access.test.ts index 39942ffe9..0dbe45c0c 100644 --- a/test/auth-status-upload-access.test.ts +++ b/test/auth-status-upload-access.test.ts @@ -21,7 +21,12 @@ import { spawn } from "bun"; const CLI_ENTRY = join(import.meta.dir, "..", "src", "index.ts"); const REPO_ROOT = join(import.meta.dir, ".."); -const NOT_GRANTED_HINT = "(one-time admin approval; see https://nemar.org/support)"; +/** The explanation under a "not granted" line. Since #1268 it is the website's + * own sentence (shared/contract/account-copy.ts, ADR 0045) rather than a + * pointer at the support page, which was all there was before ADR 0042 built + * the request. */ +const NOT_GRANTED_HINT = + "A one-time admin grant. Reviewed against export-control and local-jurisdiction rules."; /** Serves the real /users/me envelope shape: `{ user, token }`. */ function startMeServer(serviceAccess: boolean | undefined) { diff --git a/test/cli.test.ts b/test/cli.test.ts index a474cfa90..069e38196 100644 --- a/test/cli.test.ts +++ b/test/cli.test.ts @@ -574,7 +574,9 @@ describe("CLI Dataset Upload", () => { const { stdout, exitCode } = await runCli(["dataset", "upload", "/tmp/test"], ctx); expect(exitCode).toBe(1); - expect(stdout).toContain("Sandbox training required"); + // The sandbox gap is stated in the one sentence every other gap uses + // (#1268, ADR 0045); the CTA is still the bare command name. + expect(stdout).toContain("Sandbox training is missing"); expect(stdout).toContain("nemar sandbox"); }); diff --git a/test/config-rename-account.unit.test.ts b/test/config-rename-account.unit.test.ts index cc04fe27b..8f2364895 100644 --- a/test/config-rename-account.unit.test.ts +++ b/test/config-rename-account.unit.test.ts @@ -18,7 +18,7 @@ * NEMAR_CONFIG_DIR. No mocks. */ -import { afterAll, beforeEach, describe, expect, test } from "bun:test"; +import { afterAll, beforeAll, beforeEach, describe, expect, test } from "bun:test"; import { mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; @@ -54,6 +54,23 @@ function onDisk(): { return JSON.parse(readFileSync(configPath, "utf8")); } +/** + * What `NEMAR_CONFIG_DIR` held before this file touched it. + * + * `bun test` runs every file in ONE process, so this variable is shared state + * and restoring it is not tidiness. It used to be "restored" with + * `process.env.NEMAR_CONFIG_DIR = undefined`, which does not unset anything: it + * assigns the STRING "undefined", and `getConfigDir()` reads that as a relative + * path — so every later test in the run wrote its config into `./undefined/` in + * the working directory, and a test that expected the real default silently got + * a different one. + */ +let previousConfigDir: string | undefined; + +beforeAll(() => { + previousConfigDir = process.env.NEMAR_CONFIG_DIR; +}); + beforeEach(() => { mkdirSync(testDir, { recursive: true }); process.env.NEMAR_CONFIG_DIR = testDir; @@ -61,7 +78,11 @@ beforeEach(() => { }); afterAll(() => { - process.env.NEMAR_CONFIG_DIR = undefined; + // `delete` when it was unset, because assigning undefined is what caused + // this. The store cache is dropped after, so nothing keeps reading the old + // directory. + if (previousConfigDir === undefined) delete process.env.NEMAR_CONFIG_DIR; + else process.env.NEMAR_CONFIG_DIR = previousConfigDir; __resetStoreCacheForTesting(); rmSync(testDir, { recursive: true, force: true }); }); diff --git a/test/contract-schemas.test.ts b/test/contract-schemas.test.ts index ee4eaf368..5faa45ef8 100644 --- a/test/contract-schemas.test.ts +++ b/test/contract-schemas.test.ts @@ -454,6 +454,72 @@ describe("user /me envelope schema", () => { userMeResponseSchema.parse({ id: 1, username: "u", email: "e@x.org", role: "user" }), ).toThrow(); }); + + test("status is the column's closed vocabulary, not any string", () => { + // The four values migration 0001's CHECK constraint admits. The reason it + // is closed on the wire too: `status === "pending"` is what raises the + // email_verified gap, and a typo or a stray value there is a comparison + // that silently never matches rather than a compile or parse error. + const user = { + id: 1, + username: null, + email: "e@x.org", + github_username: null, + role: "user", + }; + for (const status of ["pending", "verified", "approved", "revoked"]) { + expect(userMeResponseSchema.parse({ user: { ...user, status }, token: null }).user.status).toBe( + status, + ); + } + // "active" is the DASHBOARD's collapsed value and belongs to /auth/me, not + // here; mixing the two vocabularies is the confusion the split prevents. + expect(() => + userMeResponseSchema.parse({ user: { ...user, status: "active" }, token: null }), + ).toThrow(); + }); + + test("a profile_gaps entry may carry only its field name", () => { + // getCurrentUser throws ApiError on ANY mismatch, so a required `set_on` + // would make one thin entry break every /users/me consumer -- over a key + // the renderer never reads (resolveWireProfileGaps takes `field` and + // `blocks`, and falls back to the matrix for a `blocks` it cannot use). + const parsed = userMeResponseSchema.parse({ + user: { + id: 1, + username: null, + email: "e@x.org", + github_username: null, + role: "user", + profile_gaps: [ + { field: "username", blocks: ["upload_access"] }, + { field: "city" }, + ], + }, + token: null, + }); + expect(parsed.user.profile_gaps?.map((g) => g.field)).toEqual(["username", "city"]); + expect(parsed.user.profile_gaps?.[1].set_on).toBeUndefined(); + }); + + test("a profile_gaps entry still needs its field name", () => { + // The one key the renderers cannot do without: an entry with no `field` is + // dropped by resolveWireProfileGaps, so accepting it would carry a line + // nothing can ever print. + expect(() => + userMeResponseSchema.parse({ + user: { + id: 1, + username: null, + email: "e@x.org", + github_username: null, + role: "user", + profile_gaps: [{ blocks: ["upload_access"] }], + }, + token: null, + }), + ).toThrow(); + }); }); describe("neuroschema dataset schema", () => { diff --git a/test/profile-gaps-matrix.test.ts b/test/profile-gaps-matrix.test.ts new file mode 100644 index 000000000..70a9184b0 --- /dev/null +++ b/test/profile-gaps-matrix.test.ts @@ -0,0 +1,325 @@ +/** + * The shared profile-gap matrix, pinned field by field (#1268, ADR 0045). + * + * Two things are being held still here, and they fail for different reasons. + * + * 1. THE TABLE. `WEBSITE_TABLE` below is transcribed from + * `nemarOrg/website`'s `src/lib/profile-gaps.ts` + `account-copy.ts` as + * they shipped in website#310 — what each field blocks, the prose that + * fits "Set it in ___", and the exact command. If `PROFILE_GAP_MATRIX` + * stops agreeing with it, one of the two repos has moved and the other has + * not, which is the whole failure mode phase 8 exists to prevent. This is + * the local half of that guard; `account-copy-parity.test.ts` is the half + * that reads the other checkout when it is present. + * + * 2. THE DERIVATION, over every one of the 2^7 combinations of the seven + * derivable fields. The oracle is built by FILTERING the fixture table + * rather than by re-running the production rules, so the test can disagree + * with the implementation: it says "these fields are blank, therefore these + * entries in this order", and `computeProfileGaps` has to arrive at the same + * answer from the row. + * + * No mocks: `computeProfileGaps` is pure and is called directly. + */ + +import { describe, expect, test } from "bun:test"; +import { ACCOUNT_COPY } from "../shared/contract/account-copy.js"; +import { + DERIVABLE_GAP_FIELDS, + GAP_FIELDS, + type GapField, + PROFILE_GAP_MATRIX, + type ProfileGapAccount, + computeProfileGaps, + describeProfileGap, + describeProfileGapBlocks, + describeSandboxGap, + profileGapFields, + resolveProfileGap, + resolveWireProfileGaps, +} from "../shared/contract/profile-gaps.js"; + +interface TableRow { + blocks: string[]; + /** Prose that fits "Set it in ___". */ + web: string; + /** The exact command, or null when none sets it. */ + cli: string | null; + /** Where a verified ORCID iD moves the web half. */ + orcidWeb?: string; + label: string; + derivable: boolean; +} + +/** Transcribed from nemarOrg/website `src/lib/profile-gaps.ts` (website#310). */ +const WEBSITE_TABLE: Record = { + email_verified: { + blocks: ["verified", "upload_access"], + web: "the verify step on your dashboard", + cli: "nemar auth resend-verification", + label: "A verified email address", + derivable: true, + }, + username: { + blocks: ["upload_access"], + web: "Settings", + cli: "nemar auth profile set-username", + label: "Username", + derivable: true, + }, + given_name: { + blocks: ["upload_access", "publication"], + web: "Settings", + cli: "nemar auth profile set-name", + orcidWeb: "your ORCID record at orcid.org, then sign in again", + label: "Given name", + derivable: true, + }, + family_name: { + blocks: ["upload_access", "publication"], + web: "Settings", + cli: "nemar auth profile set-name", + orcidWeb: "your ORCID record at orcid.org, then sign in again", + label: "Family name", + derivable: true, + }, + github_username: { + blocks: ["upload_access", "publication"], + web: "Settings", + cli: "nemar auth profile set-github", + label: "GitHub handle", + derivable: true, + }, + city: { + blocks: ["upload_access"], + web: "Settings", + cli: "nemar auth profile set-location", + label: "City", + derivable: true, + }, + country: { + blocks: ["upload_access"], + web: "Settings", + cli: "nemar auth profile set-location", + label: "Country", + derivable: true, + }, + why: { + blocks: ["upload_access"], + web: "the request form in Settings", + cli: "nemar auth request-upload-access", + label: "A description of what you intend to upload", + derivable: false, + }, +}; + +const TABLE_FIELDS = Object.keys(WEBSITE_TABLE); +/** The seven the derivation can raise from a row; `why` needs a submitted form. */ +const DERIVABLE = TABLE_FIELDS.filter((f) => WEBSITE_TABLE[f].derivable) as GapField[]; + +/** A complete account: nothing missing, nothing to report. */ +function fullAccount(): Required> & { status: string } { + return { + status: "verified", + email_verified: true, + username: "alovelace", + given_name: "Ada", + family_name: "Lovelace", + github_username: "adalovelace", + city: "London", + country: "GB", + orcid_verified: false, + }; +} + +/** Blank out `fields` on an otherwise complete account. */ +function accountMissing(fields: readonly string[]): ProfileGapAccount { + const account: Record = fullAccount(); + for (const field of fields) { + account[field] = field === "email_verified" ? false : null; + } + return account as ProfileGapAccount; +} + +describe("the matrix agrees with the website's table", () => { + test("every field, and only those fields, in the same order", () => { + expect([...GAP_FIELDS]).toEqual(TABLE_FIELDS); + expect([...DERIVABLE_GAP_FIELDS]).toEqual(DERIVABLE); + }); + + for (const field of TABLE_FIELDS) { + const row = WEBSITE_TABLE[field]; + test(`${field}: blocks, label and both set-on halves`, () => { + expect([...PROFILE_GAP_MATRIX[field as GapField].blocks]).toEqual(row.blocks); + const gap = resolveProfileGap(field); + expect(gap.label).toBe(row.label); + expect(gap.setOnWeb).toBe(row.web); + expect(gap.setOnCli).toBe(row.cli); + expect(gap.known).toBe(true); + }); + + test(`${field}: a verified ORCID iD moves the name halves and nothing else`, () => { + const gap = resolveProfileGap(field, { orcidVerified: true }); + if (row.orcidWeb) { + // The record owns the name, `PATCH /auth/profile` refuses the edit, and + // so there is no command either. + expect(gap.setOnWeb).toBe(row.orcidWeb); + expect(gap.setOnCli).toBeNull(); + } else { + expect(gap.setOnWeb).toBe(row.web); + expect(gap.setOnCli).toBe(row.cli); + } + }); + } +}); + +describe("computeProfileGaps over every field combination", () => { + // 2^7 = 128 subsets of the derivable fields. + const subsets: string[][] = []; + for (let mask = 0; mask < 1 << DERIVABLE.length; mask++) { + subsets.push(DERIVABLE.filter((_, i) => (mask & (1 << i)) !== 0)); + } + + test("raises exactly the blank fields, in table order, for all 128", () => { + expect(subsets).toHaveLength(128); + for (const missing of subsets) { + // The oracle: the fixture's own order, filtered to what was blanked. + const expected = TABLE_FIELDS.filter((f) => missing.includes(f)); + const gaps = computeProfileGaps(accountMissing(missing)); + expect(gaps.map((g) => g.field)).toEqual(expected); + for (const gap of gaps) { + expect([...gap.blocks]).toEqual(WEBSITE_TABLE[gap.field].blocks); + expect([...gap.set_on]).toEqual(WEBSITE_TABLE[gap.field].cli ? ["web", "cli"] : ["web"]); + } + } + }); + + test("`why` is never raised from a row, however empty the row is", () => { + // Nothing is stored until the form is submitted, so no account can be + // missing it -- it exists in the table only for a refusal to render. + expect(profileGapFields(accountMissing(DERIVABLE))).not.toContain("why"); + }); + + test("ORCID is never a gap, linked or not", () => { + for (const orcid_verified of [true, false]) { + const fields = profileGapFields({ ...accountMissing(DERIVABLE), orcid_verified }); + expect(fields).not.toContain("orcid"); + } + }); + + test("a verified iD moves set_on to web-only for the name halves", () => { + const gaps = computeProfileGaps({ + ...accountMissing(["given_name", "family_name", "city"]), + orcid_verified: true, + }); + const setOn = Object.fromEntries(gaps.map((g) => [g.field, [...g.set_on]])); + expect(setOn.given_name).toEqual(["web"]); + expect(setOn.family_name).toEqual(["web"]); + // The export-control fields are unaffected: ORCID owns a name, not a city. + expect(setOn.city).toEqual(["web", "cli"]); + }); + + test("whitespace counts as absent", () => { + // A row holding " " has no city, matching resolveUploaderIdentity's trim. + expect(profileGapFields({ ...fullAccount(), city: " " })).toEqual(["city"]); + }); + + test("`status: pending` raises the inbox even when the flag says otherwise", () => { + expect(profileGapFields({ ...fullAccount(), status: "pending" })).toEqual(["email_verified"]); + }); + + test("an unknown email_verified on an active account is not an unproved inbox", () => { + // A caller that does not report the flag must not make everyone re-verify. + expect(profileGapFields({ ...fullAccount(), email_verified: undefined })).toEqual([]); + }); + + test("an unreadable username is not a missing one", () => { + // `undefined` is "could not be read" (the website resolves it separately); + // prompting for a handle someone may already have is the worse answer. + expect(profileGapFields({ ...fullAccount(), username: undefined })).toEqual([]); + expect(profileGapFields({ ...fullAccount(), username: null })).toEqual(["username"]); + }); +}); + +describe("the sentences", () => { + test("the one every surface prints, composed from the shared templates", () => { + expect(describeProfileGap(resolveProfileGap("github_username"))).toBe( + "GitHub handle is missing: needed to request upload access. Set it in Settings or run `nemar auth profile set-github`.", + ); + }); + + test("it names the nearest wall, not every wall behind it", () => { + // github_username blocks publication too; the request is what is in front + // of the reader. + const sentence = describeProfileGap(resolveProfileGap("github_username")); + expect(sentence).not.toContain("publish"); + }); + + test("a name under a verified iD points at orcid.org and offers no command", () => { + expect(describeProfileGap(resolveProfileGap("given_name", { orcidVerified: true }))).toBe( + "Given name is missing: needed to request upload access. Set it in your ORCID record at orcid.org, then sign in again.", + ); + }); + + test("an unrecognised field renders rather than disappearing", () => { + const gap = resolveProfileGap("institutional_email"); + expect(gap.known).toBe(false); + expect(gap.label).toBe("institutional_email"); + expect(describeProfileGap(gap)).toBe( + "institutional_email is missing: needed to finish setting up your account. Set it in Settings.", + ); + }); + + test("the full block list is noun-phrased for a surface that reports", () => { + expect(describeProfileGapBlocks(resolveProfileGap("given_name"))).toBe( + "upload access and publication", + ); + expect(describeProfileGapBlocks(resolveProfileGap("city"))).toBe("upload access"); + }); + + test("sandbox is CLI-only, is not in the shared matrix, and reads like the rest", () => { + expect(GAP_FIELDS).not.toContain("sandbox" as GapField); + expect(describeSandboxGap()).toBe( + "Sandbox training is missing: needed to upload a dataset from the CLI. Run `nemar sandbox`.", + ); + }); + + test("`gaps.none` is the answer to an account with nothing outstanding", () => { + expect(computeProfileGaps(fullAccount())).toEqual([]); + expect(ACCOUNT_COPY["gaps.none"]).toBe( + "Nothing outstanding — every field NEMAR needs is filled in.", + ); + }); +}); + +describe("resolveWireProfileGaps", () => { + test("re-sorts into table order, so a backend's order does not leak", () => { + const resolved = resolveWireProfileGaps([ + { field: "country", blocks: ["upload_access"] }, + { field: "email_verified", blocks: ["verified", "upload_access"] }, + { field: "username", blocks: ["upload_access"] }, + ]); + expect(resolved.map((g) => g.field)).toEqual(["email_verified", "username", "country"]); + }); + + test("takes blocks from the wire, and falls back to the table", () => { + // The backend is the authority on what a field blocks TODAY. + const [fromWire] = resolveWireProfileGaps([{ field: "city", blocks: ["publication"] }]); + expect([...fromWire.blocks]).toEqual(["publication"]); + const [fromTable] = resolveWireProfileGaps([{ field: "city" }]); + expect([...fromTable.blocks]).toEqual(["upload_access"]); + }); + + test("drops block values it does not know rather than printing them raw", () => { + const [gap] = resolveWireProfileGaps([{ field: "city", blocks: ["teleportation"] }]); + // Every value filtered out means an empty list, which falls back to the + // table -- a build that predates a new vocabulary still says something true. + expect([...gap.blocks]).toEqual(["upload_access"]); + }); + + test("ignores an entry with no field name", () => { + expect(resolveWireProfileGaps([{ blocks: ["upload_access"] }, { field: "city" }])).toHaveLength( + 1, + ); + }); +});