Repository navigation
Phase 4: identity uniqueness - #1264
neuromechanist merged 18 commits into
Conversation
📦 Package PreviewYour package is ready to test! bun install -g nemar-cli@PR1264Version: 🤖 Updated on each push. |
Review round 1 — all 16 items addressedEvery item fixed on this branch; nothing declined as a false positive. Six commits, Offline tier 4336 pass, 20 skip, 1 fail (the expected A. Deploy blocker1. The 79-char GLOB. Confirmed exactly as reported, and worse than a lint issue — statement 1 (the I probed the limit rather than guessing: D1 accepts a 50-character LIKE/GLOB pattern and rejects a 51-character one. Replaced with a pattern-free shape plus one 8-character GLOB: WHERE LENGTH(orcid) = 19
AND SUBSTR(orcid, 19, 1) = 'x' -- BINARY collation: 'X' must NOT match
AND SUBSTR(orcid, 5, 1) = '-'
AND SUBSTR(orcid, 10, 1) = '-'
AND SUBSTR(orcid, 15, 1) = '-'
AND REPLACE(SUBSTR(orcid, 1, 18), '-', '') NOT GLOB '*[^0-9]*';Validated per-value on real D1 ( Two proof runs, both recorded in the migration header:
The recurrence guard is One thing worth flagging: B. Everyday regression2. Fixed. All three schemas use Worth recording why the obvious test does not work. My first version signed up mixed-case and retrieved with another case — and it passed with the exact-case lookup restored, because once the request is normalised, an exact-case lookup still matches every row this phase creates. The regression only exists for the ~600 rows already in the catalog, stored as typed. The tests therefore seed a legacy row directly ( Also fixed the same latent bug the reviewer flagged in 10 — the signup inline schema now imports C. Races and typed errors3. Finalize's 4. 5. Both 6. Only 7. All three All four are tested with real fault injection ( D. Correctness8. 9. Real bug, fixed. 10. Both done — the admin fixture route and the signup inline schema use 11. Used rather than deleted: signup's two hand-rolled GitHub queries now go through E. TestsAll added:
Every new assertion mutation-checked, one perturbation at a time with a revert between each. All caught, each by the expected test. F. Comments and docs13. Replaced with your wording. Confirmed the claim was false — RegExpRouter throws 14. 15. All three. The "re-derivable by the DOI enrichment pass" claim was false — 16. Ready for the phase-3 rebase; 0076/0042 are theirs and my numbers stand. |
Review round 2 — items 17-19All three fixed in 17.
|
01b718e to
29fcda5
Compare
Rebased onto Phase 3 (
|
| File | Conflict | Resolution |
|---|---|---|
backend/src/services/profile.ts |
Both sides added an import to the same line | Kept both — normalizeGithubHandle from ./identity and Phase 3's validateUsernameFormat from ./username |
src/lib/api/admin.ts |
Both added type imports; both appended a trailing block sharing one );\n} closer |
Kept both import groups; split the shared closer so Phase 3's backfillUsernames and my getUserDuplicates/clearIdentityConflict each close properly, and restored my section's // ==== header |
test/admin-route-inventory.unit.test.ts |
Both added rows to EXPECTED_ENTRIES |
Kept both. Rewrote my comment: it claimed "RegExpRouter matches the static segment first", which round 1 established is false — it now says the route is reachable only because registerUserDuplicateRoutes runs first |
src/commands/admin.ts |
Both appended a new command block | Took Phase 3's file, re-appended my duplicates block verbatim, re-added my three imports (clearIdentityConflict, getUserDuplicates, DuplicateGroup) |
src/commands/auth.ts |
Both appended a new command block, interleaved badly by git | Same approach: Phase 3's file plus my profile block verbatim, plus errorDetail on the errors import |
.context/decisions/README.md |
Both added an index line | Kept both, 0042 then 0043 |
backend/src/routes/auth-web.ts |
/code/verify's user SELECT — Phase 3 added three columns, I added COLLATE NOCASE |
Kept both: the three columns and the NOCASE equality |
The tri-state, as instructed
validateGitHubUsername now returns found / not_found / unavailable, and every github_in_use refusal runs behind it:
- CLI signup (
auth.ts) — Phase 3'sunavailable→ 503 andnot_found→ 400 guards come first, thenconst githubUser = githubLookup.user, then myfindGithubHolder(db, githubUser.login)+identityRefusal("github_in_use"). PATCH /auth/profile(auth-web.ts) — same order, with myidentityRefusalon both the pre-check and the canonical-login re-check, andisUniqueViolationOn(err, "github_username")in the catch (Phase 3'susernameUNIQUE handling sits right after it).
A GitHub outage is now a 503 "try again in a few minutes", never a duplicate-identity 409 — the two failures can no longer be confused in either direction.
Two things the rebase turned up
1. backend/test/profile-username-route.test.ts failed (4 tests). Phase 3's seedUser defaults every row to one shared ORCID_ID, so any test seeding two rows died on UNIQUE constraint failed: users.orcid. The fixture was manufacturing exactly the duplicate 0077 exists to prevent — the same thing I found in web-email-verification.test.ts in round 1, in a file written in parallel. Each seeded row now gets a distinct iD from a counter reset per test; callers passing an explicit orcid are untouched. All 28 pass. This is the constraint working, not a regression — worth knowing that any future fixture reusing an iD across live rows will now fail loudly at the seed.
2. AGENTS.md rows were incomplete. Phase 3 updated the user-flow paragraph (kept intact) but not the CLI tables or the load-bearing ADR list. Since I was editing those exact lines I completed them rather than leaving them wrong: added request-upload-access to the nemar auth row, backfill-usernames to nemar admin, and ADR 0042 to the load-bearing list alongside 0040 and 0043. Flagging it because it is Phase 3 content I touched, not mine.
Verified after the auto-merges
Git auto-merged several files it did not flag, so I checked the result rather than trusting it. No Phase 3 content was lost anywhere — git diff origin/...HEAD shows deletions only on the four lines I deliberately edited. Confirmed present: all five signup identity checks, all three NOCASE lookups in auth.ts (the only exact-case users-by-email lookup left in the backend is the deliberate seed-web-user fixture), the claimedByAnother race mapping in auth-orcid.ts, my five /users/me fields alongside Phase 3's, and my optional contract fields alongside theirs.
Housekeeping
Migration 0077's header and ADR 0043's numbering note both said "until phase 3 is rebased in, the gapless check fails by construction". That is now past tense and wrong, so both were rewritten to record that 0076/0042 and 0077/0043 coexist and touch disjoint columns. Route-total pin updated 100 → 102 (Phase 3's two entries plus mine).
Nothing outstanding from my side.
fa82400
into
feature/issue-1250-epic-account-tiers
Closes #1254
Part of #1250
Summary
One person, one account. An ORCID iD, an email address (case-insensitively) or a GitHub handle now backs at most one live NEMAR account. The database enforces it, the application refuses it first with a typed error that says what to do, and every write normalises the identifier so all three rules compare the same strings.
Production is the reason: rows 42 and 43 both carry ORCID
0000-0002-1974-1293withorcid_verified=1. Theoauth_identitiesrow left 42 (an unlink, or an identity insert whose rollback did not run) whileusers.orcidstayed behind on it, so the second sign-up saw an unclaimed iD and made a second account.Migration 0077 — flag, then partial-index
A plain
CREATE UNIQUE INDEX ON users(orcid)fails on 42/43 and takes the deploy with it, and making it buildable would mean deleting or merging someone's real account inside a migration. So 0077 addsusers.identity_conflict, flags every non-canonical duplicate, and creates the unique indexes as partial ones that exclude a flagged row.oauth_identitiesrow, lowest id only if none does. In the 42/43 case the lowest id is the orphan, so "lowest id wins" would have crowned the row that cannot sign in.COLLATE NOCASE.xcheck digit, narrowly (aGLOBmatching only a bare iD), so a case variant cannot slip past an index that compares exactly.Typed error codes (the website renders these)
Declared once in
shared/contract/identity.tswith their messages, so the CLI, the website and the JSONmessageread the same wording:orcid_in_useorcid_already_linkedorcid_linked_otheremail_in_use/auth/email/change/{request,verify}, CLI signupgithub_in_useidentity_conflict_remainsPOST /admin/users/:id/clear-identity-conflicterrordeliberately carries different things on the two families of route, because different things read it and both are already right: browser-facing routes put the CODE there (the website reads it there, andemail_in_usepredates this phase), CLI signup keeps its short human label there because that is the field the CLI prints.codeandmessageare purely additive on both.Refusals and normalisation
users.orcidORoauth_identities(it only checked the identity table), and refuses a case-variant email.findOrcidHolderbefore writing, so relink cannot take an iD off another account.users.orcidas well asorcid_verified. The citation value is re-derivable (a re-link refills it, and so does DOI enrichment); the false claim on an identifier is not worth keeping, and it is exactly the shape row 42 is in.backend/src/services/identity.ts): email trimmed+lowercased, ORCID reduced to a bare iD with an uppercase check digit, GitHub handle trimmed with a leading@stripped.PATCH /auth/profilenow imports the GitHub rule instead of spelling it out again, and five hand-rolled copies of the email field schema collapse into one — which also fixes a latent bug: they ran.email()before their own trim, so a pasted" ada@lab.org "400'd.New surfaces
GET /admin/users/duplicates— groups live accounts sharing an identifier, marks the canonical row, and carries id / username / email / created_at / has_oauth_identity / dataset_count / identity_conflict per row.POST /admin/users/:id/clear-identity-conflict— clears the flag only when no collision remains; 409 with the colliding rows otherwise.nemar admin duplicates [--json] [--clear <id>]— a new top-level admin command (not a flag onadmin users). Exits non-zero when a duplicate exists.nemar auth profile— every identifier with its verification state, the access tier, and a "where to change each" footer. Always fetched, never cached: it is the command someone runs before changing an identifier.GET /users/megainsstatus,email_verified,orcid_verified,given_name,family_name(all.optional()in the contract — absence renders as "unknown", never as "no").ADR 0043
one person, one account: the flag-and-partial-index strategy and why (deploy safety, no deletion), self-service remediation first, manual merge last.Testing
bun run lint,bun run typecheckclean. Offline (pure) tier: 4295 pass, 20 skip, 1 fail — the single failure isADR index > ADR numbers are unique and gapless from 0001, by construction: 0042 belongs to phase 3 (#1253), which is landing in parallel. Same situation ADR 0041 documented, and it goes green at rebase. If 0042 lands first this ADR renumbers with it.New tests, all real-engine (bun:sqlite behind
realD1, real Hono dispatch, real bcrypt/HMAC/session issuance; the only external boundaries are localBun.serve()stand-ins for ORCID and GitHub reached through the bindings production already uses to point at a mirror):backend/test/identity-uniqueness-migration.test.ts(11) — the migration over a fixture carrying the 42/43 shape, a three-row case-variant email group, a pair colliding on both identifiers, a legacy lowercase check digit, clean rows and a tombstone. Asserts the whole-catalog property (every live group ends with exactly one unflagged row), both indexes and their DDL, that a later duplicate INSERT is refused, that a flagged duplicate can still be inserted, and that a flagged row stays readable.backend/test/identity-refusals-route.test.ts(24) — every refusal at every entry point, plus normalisation and the unlink release.backend/test/user-duplicates-route.test.ts(16) — the report and the clear endpoint. The canonical mark is asserted against the flags migration 0077 itself wrote, not against a second copy of the rule.backend/test/users-me-profile-fields.test.ts(3) — the source of the new/users/mefields.test/auth-profile-cli.test.ts(7),test/admin-duplicates-cli.test.ts(8) — real subprocess CLI runs against a local server. Classified into tier 2 by CI'sTEST_API_URL|runCli\(grep (same asauth-status-upload-access.test.ts), but self-contained: verified passing withTEST_API_URL/TEST_ADMIN_API_KEY/TEST_USER_API_KEYunset. No live-backend file is touched by this phase.Every new assertion was mutation-checked, one perturbation at a time, reverting between each (20 mutations; each failed the expected test and only that test). Two survivors were found and fixed rather than explained away:
validateGitHubUsername, so the tests now also assert no GitHub API call was spent — which is the real reason for that ordering (a duplicate must not cost a rate-limited request against a shared token).email_verifiedfrom/users/meleft every CLI test green, because the CLI test serves its own payload.backend/test/users-me-profile-fields.test.tswas added to pin the source.Two pre-existing things the work turned up and fixed:
GET /admin/users/duplicatesinitially 404'd: Hono runs every matching handler in registration order, andGET /users/:usernamematches/users/duplicates.registerUserDuplicateRoutesnow runs first, with the reason recorded at the call site. Caught by driving the HTTP route rather than the handler.backend/test/web-email-verification.test.ts'sseedWebUserreused one ORCID iD for every row, so it was manufacturing exactly the duplicate this phase forbids. Each seeded row now gets its own iD.Rollout note
Row 42 was soft-deleted in production on 2026-09-05, after this branch was written. Consequences for the deploy:
deleted_at IS NULL, so the tombstoned row is not a candidate and row 43 keeps0000-0002-1974-1293unflagged. The migration is still correct for that catalog — it flags nothing where there is now nothing to flag — and the fixtures keep the 42/43 shape because it is the shape the RULES have to handle, not a claim about today's rows.CREATE UNIQUE INDEXstatements were the part that could have failed on live data, and the deletion only removes candidates.users.orcidbut leftorcid_verified = 1— the same half-state that produced 42/43 in the first place.user-tombstone.tsnow clears both, so no future tombstone can do it, but row 42 keeps the stale flag until someone repairs it. It is inert (every reader gates ondeleted_at IS NULL), so I have not added a repair statement to a migration that is already proven against real D1. One line if it is wanted:Notes for the reviewer
identity_conflictwhile writing it. That is safe because it only ever flags non-minimum rows, so the minimum it computes stays unflagged and is order-independent; the reasoning is written on the statement and pinned by the "colliding on BOTH identifiers" fixture.COLLATE NOCASE. Statement (2) plus normalisation at every write is what makes that complete rather than a gap.normalizeOrcidin ORCID finalize is unreachable as a repair (verifyPendingalready rejects a non-canonical iD) and is kept as a guarantee about what gets stored. Commented as such.9176d311). 0076/0042 are theirs, 0077/0043 are mine, and the ADR index's gapless check is green. Both phases' behaviour is kept everywhere they overlapped; thegithub_in_userefusals now run behind Phase 3's tri-statevalidateGitHubUsername, so a GitHub outage is a 503 rather than a duplicate-identity 409.bun run migrations:d1-checkis new and is a release check, not per-PR CI (~1 min). It exists because migration 0077 shipped a 79-character GLOB that bun:sqlite runs and D1 rejects at 50 — every test green, deploy aborted mid-file. AGENTS.md's release section names it.users-by-email lookup is nowCOLLATE NOCASE. Normalising the request is not sufficient on its own: it only fixes rows this phase creates, and the ~600 rows already in the catalog are stored exactly as typed. The one deliberate exception is the non-productionseed-web-usertest fixture, which stays exact-case so a fixture cannot silently adopt a pre-existing legacy row; it already fails loudly if its own INSERT/UPDATE disagree.