feat(oauth): dual-write accountAuthorizations to a scopeId v2 table behind flags by nshirley · Pull Request #20891 · mozilla/fxa

nshirley · 2026-07-20T17:48:03Z

Because

accountAuthorizations keys its rows on a wide VARCHAR scope column, which bloats the clustered primary key (and every future index). This is Phase 1 of the FXA-14169 epic to migrate that column to an integer scopeId foreign key.
The table is live with millions of rows, so the migration has to be staged: we stand up a shadow table and dual-write to it behind flags, so the cutover can happen gradually with no downtime and nothing changes until we deliberately turn it on.

This pull request

Adds the accountAuthorizations_v2 table (integer scopeId FK to scopes, no scope string) and seeds the common non-URL scopes, via a new fxa_oauth migration.
Adds two config flags — dualWriteV2 and readV2 — both defaulting off.
With dualWriteV2 on, writes each consent to v2 alongside v1 using a small in-memory scope→id cache; the v2 write is best-effort and isolated, so it can never affect the authoritative v1 write. With readV2 on, consent lookups check v2 first and fall back to v1.
Adds an idempotent v1 → v2 backfill script that stays inert until run.

Issue that this pull request solves

Closes: FXA-14094

Checklist

Put an x in the boxes that apply

My commit is GPG signed.
If applicable, I have modified or added tests which pass locally.
I have added necessary documentation (if appropriate).
I have verified that my changes render correctly in RTL (if appropriate).
I have manually reviewed all AI generated code.

How to review (Optional)

Key files/areas to focus on: the migration (patch-038-039.sql), the dual-write + v2-first/v1-fallback reads in lib/oauth/db/mysql/index.js, and the flag wiring in lib/oauth/db/index.js.
Suggested review order: migration → config flags → scopes-cache.ts → db-layer dual-write/read → route metrics → backfill script.
Risky or complex parts: the live consent write path — the v2 write sits in its own try/catch so a v2 failure can't regress v1, and both flags default off.

Screenshots (Optional)

Please attach the screenshots of the changes made in case of change in user interface.

Other information (Optional)

Both flags ship off, so merging changes nothing at runtime. Enable order once deployed: dual-write → backfill → read.
The accountAuthorization.recorded metric now emits a per-row count with a version tag (FXA-14202), so its dashboards/alerts should move in lockstep.

Copilot

Pull request overview

This PR implements Phase 1 of the accountAuthorizations scope→scopeId migration in the OAuth MySQL DB by introducing a v2 shadow table and wiring the auth-server to dual-write (and optionally read) via feature flags, plus operational tooling to backfill existing rows.

Changes:

Add accountAuthorizations_v2 (with scopeId FK to scopes) and seed common non-URL scopes via an fxa_oauth schema patch.
Add dualWriteV2 and readV2 config flags (default off) and implement v2 dual-write + v2-first/v1-fallback reads in the MySQL OAuth DB layer.
Add a resumable/idempotent v1→v2 backfill script with unit tests, and extend route/integration tests + metrics tagging (version=v1|v2) for observability.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/fxa-auth-server/test/remote/account_consents.in.spec.ts	Adds integration coverage for dual-write and v2-first read behavior behind flags.
packages/fxa-auth-server/scripts/backfill-account-authorizations-v2/backfill-account-authorizations-v2.ts	New backfill script to copy v1 rows into v2 via keyset pagination + idempotent upsert.
packages/fxa-auth-server/scripts/backfill-account-authorizations-v2/backfill-account-authorizations-v2.spec.ts	Unit tests for backfill helpers and end-to-end `run()` behavior with stubbed queries.
packages/fxa-auth-server/lib/routes/oauth/authorization.spec.ts	Updates/extends route tests for version-tagged metrics and v2 dual-write observability.
packages/fxa-auth-server/lib/routes/oauth/authorization.js	Emits `recorded` as per-row-count with `version` tag; emits missing-scope + v2 failure metrics/logging.
packages/fxa-auth-server/lib/oauth/scopes-cache.ts	Introduces an in-process scope→id cache used by the v2 dual-write/read paths.
packages/fxa-auth-server/lib/oauth/scopes-cache.spec.ts	Unit tests for cache hit/miss behavior and non-negative caching semantics.
packages/fxa-auth-server/lib/oauth/db/mysql/index.js	Implements v2 shadow-table upserts and v2-first reads; adds bulk scope-id resolution helper.
packages/fxa-auth-server/lib/oauth/db/index.js	Wires new flags into the DB API for dual-write and readV2 behavior.
packages/fxa-auth-server/config/index.ts	Adds the new `oauthServer.accountAuthorizations.{dualWriteV2,readV2}` convict config flags.
packages/db-migrations/databases/fxa_oauth/target-patch.json	Bumps fxa_oauth schema patch level to 39.
packages/db-migrations/databases/fxa_oauth/patches/patch-038-039.sql	Creates `accountAuthorizations_v2` and seeds common non-URL scopes.
packages/db-migrations/databases/fxa_oauth/patches/patch-039-038.sql	Adds the (commented) reverse patch reference for completeness.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    const resolved = new Map<string, number>();
+    const uncached: string[] = [];
+    const seen = new Set<string>();
+
+    // Dedupe input and split into cache hits vs. what needs a DB lookup.
+    for (const scope of scopes) {
+      if (seen.has(scope)) {
+        continue;
+      }
+      seen.add(scope);
+      const cachedId = this.cache.get(scope);
+      if (cachedId !== undefined) {
+        resolved.set(scope, cachedId);
+      } else {
+        uncached.push(scope);
+      }
+    }
+
+    if (uncached.length > 0) {
+      const rows = await this.resolver(uncached);
+      for (const { scope, id } of rows) {
+        this.cache.set(scope, id);
+        resolved.set(scope, id);
+      }
+    }


nshirley · 2026-07-23T19:26:15Z

+    //
+    // I'm not entierly sure about this, going to revisit before merge.
+    const tags = {


Yes, I do intend to revisit this. It's not great, and kind of a mess. Might be worth ripping this out/reverting and just a separate ticket to see if we can get more granular metrics.

The reasoning behind it is; today, the accountAuthorization.recorded fires only once per request, so when it's counted it looks like we have a much lower number of authorizations than we do given an authorization is one-to-many with the authorized scopes. Ideally we emit an accountAuthorization.recorded for each number of authorized scope OR at least include a count signal on the metric that we can sum off of

vbudhram

@nshirley I have not run the script but the changes seem straight forward and low risk (new table, kill switch). Maybe you can update the ticket or file a new ticket to document the steps in the migration. Ie first dual write, backfill, enable V2 read, remove V1 logic, drop old column.

vbudhram · 2026-07-24T14:54:04Z

+ * `scopeId` from the scopes table. Walks v1 in primary-key order via keyset
+ * pagination and upserts each batch into v2.
+ *
+ * Resolve-only, matching the live dual-write: a v1 row whose `scope` has no


vbudhram · 2026-07-24T15:20:06Z

  }

-  async _findAccountConsentForSignIn(uid, scope, service) {
+  // Existence check for the sign-in consent row. When readV2 is set, checks


Just confirming the plan is that the backfill is run before reading from V2? Otherwise, this route would be doing both lookups (probably ok too)

…ehind flags Because: * accountAuthorizations keys on a wide VARCHAR `scope`. Phase 1 of FXA-14169 migrates it to an integer scopeId FK via a shadow table, dual-writing until cutover. Everything is flag-gated (default off), so merging changes nothing. This commit: * Adds the accountAuthorizations_v2 table + scopes-seed migration and the dualWriteV2 / readV2 config flags. * Dual-writes consents to v2 (resolve-only, isolated from the v1 write) via an in-mem scope->id cache, and adds v2-first / v1-fallback reads. * Adds an idempotent v1->v2 backfill script (inert; running it is FXA-14095). Refs FXA-14094

nshirley force-pushed the FXA-14094 branch from 0357ee6 to e53f0dc Compare July 21, 2026 19:23

nshirley marked this pull request as ready for review July 22, 2026 16:00

nshirley requested a review from a team as a code owner July 22, 2026 16:00

Copilot AI review requested due to automatic review settings July 22, 2026 16:00

Copilot started reviewing on behalf of nshirley July 22, 2026 16:00 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

vbudhram reviewed Jul 24, 2026

View reviewed changes

nshirley force-pushed the FXA-14094 branch from e53f0dc to 34b5d80 Compare July 24, 2026 22:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(oauth): dual-write accountAuthorizations to a scopeId v2 table behind flags#20891

feat(oauth): dual-write accountAuthorizations to a scopeId v2 table behind flags#20891
nshirley wants to merge 1 commit into
mainfrom
FXA-14094

nshirley commented Jul 20, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

nshirley Jul 23, 2026

Uh oh!

vbudhram left a comment

Uh oh!

vbudhram Jul 24, 2026

Uh oh!

vbudhram Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

nshirley commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Because

This pull request

Issue that this pull request solves

Checklist

How to review (Optional)

Screenshots (Optional)

Other information (Optional)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

nshirley Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

vbudhram left a comment

Choose a reason for hiding this comment

Uh oh!

vbudhram Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

vbudhram Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nshirley commented Jul 20, 2026 •

edited

Loading