Add Amazon SES delivery and signed event handling

.github/workflows/terraform.yml

@@ -22,4 +22,4 @@ jobs:
       - run: terraform fmt -check -recursive infra
       - run: terraform -chdir=infra init -backend=false -input=false
       - run: terraform -chdir=infra validate
-      - run: node --test infra/check-web-plan.test.mjs
+      - run: node --test infra/check-web-plan.test.mjs infra/ses-isolation.test.mjs

infra/README.md

@@ -1,8 +1,8 @@
 # Orbit infrastructure
 
-This directory is one Terraform state for the Render production and staging resources, Cloudflare DNS and Turnstile, and AWS SSM parameters. `production.tf` imports the existing API and declares the production database, web service, and landing site. `staging.tf` declares the staging database, API, web service, and landing site. `configuration.tf` owns four environment groups and their AWS SSM inputs. `cloudflare.tf` owns the zone, DNS records, widget, and its secret parameters. `variables.tf` contains the cutover switches and image digests. `versions.tf` pins the providers and configures the encrypted S3 state with native locking.
+This directory is one Terraform state for the Render production and staging resources, Cloudflare DNS and Turnstile, and AWS SSM parameters. `production.tf` imports the existing API and declares the production database, web service, and landing site. `staging.tf` declares the staging database, API, web service, and landing site. `configuration.tf` owns four environment groups and their AWS SSM inputs. `cloudflare.tf` owns the zone, existing DNS records, widget, and its secret parameters. `ses.tf` owns SES identities, new DNS records, event routing, and send credentials. `variables.tf` contains the cutover switches and image digests. `versions.tf` pins the providers and configures the encrypted S3 state with native locking.
 
-The Render provider reads `RENDER_API_KEY` from the environment. The Cloudflare provider reads `CLOUDFLARE_API_TOKEN` from the environment. The AWS provider uses the default credential chain in `us-east-2`. The S3 state contains decrypted SecureString values, including the Turnstile secret, so access to the bucket and its lock file must stay restricted. The existing API is imported with its service ID; do not remove that import block or replace the service.
+The Render provider reads `RENDER_API_KEY` from the environment. The Cloudflare provider reads `CLOUDFLARE_API_TOKEN` from the environment. The AWS provider uses the default credential chain in `us-east-2`. The S3 state contains decrypted SecureString values, including the Turnstile and SES secrets, so access to the bucket and its lock file must stay restricted. The existing API is imported with its service ID; do not remove that import block or replace the service.
 
 ## SSM parameters
 
@@ -67,6 +67,10 @@ Every staging secret must be independently generated or issued for staging. `/or
 
 Google Play uses the configured service account for both live and license-tester subscription purchases. Set up license testers on the internal and closed tracks, then verify a test purchase through `play/verify`. The staging Pub/Sub push subscription must use `https://api-staging.useorbit.org/api/subscriptions/play/rtdn` as both its endpoint and OIDC audience, with the configured `GooglePlay__RtdnServiceAccountEmail` as the token issuer. Confirm an RTDN push reaches staging and updates the linked purchase. The production API's non-secret values mirror the existing Render service; staging overrides its environment, CORS origin, return URLs, RTDN audience, and billing IDs.
 
+Terraform creates `/orbit/<environment>/api/Ses__AccessKeyId` and `/orbit/<environment>/api/Ses__SecretAccessKey` as SecureString parameters and places the matching values in each Render API environment group. `orbit-api-ses` can send with the production configuration sets; `orbit-api-ses-staging` can send with the staging sets. Both users can send from the two shared SES identities. The production sets `orbit-transactional` and `orbit-marketing` publish to `orbit-ses-events`, which subscribes only to `https://api.useorbit.org/api/email/ses-events`. The staging sets `orbit-staging-transactional` and `orbit-staging-marketing` publish to `orbit-ses-events-staging`, which subscribes only to `https://api-staging.useorbit.org/api/email/ses-events`. Each API group receives its own `Ses__TransactionalConfigurationSet`, `Ses__MarketingConfigurationSet`, and `Ses__TopicArn`. Both groups also provide `Ses__Region`, `Ses__FromEmail`, `Ses__MarketingFromEmail`, and `Ses__SupportEmail`. `Email__Provider` comes from separate `production_email_provider` and `staging_email_provider` Terraform variables, each defaulting to `Resend`. After SES production access and identity verification are complete, set the intended environment's variable to `Ses` in `infra/local.tfvars`, plan, and apply. Roll back by setting it to `Resend` in that file, planning, and applying again. Confirm each SNS subscription through its own signed webhook before enabling that environment's SES provider.
+
+The SES DKIM CNAME records and the MAIL FROM MX and SPF records are managed directly in `ses.tf` using the SES identity outputs. They are unproxied and use `bounce.send.useorbit.org` and `bounce.updates.useorbit.org`, leaving the existing Resend records intact. The Cloudflare zone is created on the Free plan without a zone subscription resource because that resource requires billing scope unavailable to the DNS token.
+
 Terraform creates `/orbit/production/api/BotProtection__SecretKey` and `/orbit/staging/api/BotProtection__SecretKey` as SecureString parameters from the managed Turnstile widget. Both API environment groups read the corresponding parameter. `BotProtection__Enabled` remains at its current setting. The public `turnstile_site_key` output is for the web and landing builds.
 
 ## DNS cutover

infra/cloudflare.tf

@@ -6,13 +6,6 @@ resource "cloudflare_zone" "orbit" {
   type = "full"
 }
 
-resource "cloudflare_zone_subscription" "orbit" {
-  zone_id = cloudflare_zone.orbit.id
-  rate_plan = {
-    id = "free"
-  }
-}
-
 locals {
   existing_dns_records = {
     apex_a = {

infra/configuration.tf

@@ -49,6 +49,14 @@ locals {
     Jwt__ExpiryMinutes                  = "0"
     Jwt__Issuer                         = "OrbitApi"
     Jwt__RefreshExpiryDays              = "90"
+    Email__Provider                     = var.production_email_provider
+    Ses__Region                         = "us-east-2"
+    Ses__FromEmail                      = "Orbit <noreply@send.useorbit.org>"
+    Ses__MarketingFromEmail             = "Orbit <news@updates.useorbit.org>"
+    Ses__SupportEmail                   = "contact@useorbit.org"
+    Ses__TransactionalConfigurationSet  = aws_sesv2_configuration_set.orbit["transactional"].configuration_set_name
+    Ses__MarketingConfigurationSet      = aws_sesv2_configuration_set.orbit["marketing"].configuration_set_name
+    Ses__TopicArn                       = aws_sns_topic.ses_events.arn
     Resend__FromEmail                   = "Orbit <noreply@send.useorbit.org>"
     Resend__SupportEmail                = "contact@useorbit.org"
     Sentry__Environment                 = "production"
@@ -65,19 +73,23 @@ locals {
   }
 
   staging_api_values = merge(local.production_api_values, {
-    ASPNETCORE_ENVIRONMENT         = "Staging"
-    Database__MigrateOnStartup     = "true"
-    Cors__AllowedOrigins__0        = "https://staging.useorbit.org"
-    Frontend__BaseUrl              = "https://staging.useorbit.org"
-    GooglePlay__RtdnAudience       = "https://api-staging.useorbit.org/api/subscriptions/play/rtdn"
-    Google__AllowedRedirectUris__0 = "https://staging.useorbit.org/auth-callback"
-    Google__AllowedRedirectUris__1 = "https://app.useorbit.org/auth-callback"
-    Marketing__ApiBaseUrl          = "https://api-staging.useorbit.org"
-    Sentry__Environment            = "staging"
-    Stripe__CancelUrl              = "https://staging.useorbit.org/upgrade"
-    Stripe__SuccessUrl             = "https://staging.useorbit.org/settings?subscription=success"
-    Supabase__Url                  = "https://staging-storage-disabled.invalid"
-    Waitlist__ApiBaseUrl           = "https://api-staging.useorbit.org"
+    Email__Provider                    = var.staging_email_provider
+    Ses__TransactionalConfigurationSet = aws_sesv2_configuration_set.staging["transactional"].configuration_set_name
+    Ses__MarketingConfigurationSet     = aws_sesv2_configuration_set.staging["marketing"].configuration_set_name
+    Ses__TopicArn                      = aws_sns_topic.ses_events_staging.arn
+    ASPNETCORE_ENVIRONMENT             = "Staging"
+    Database__MigrateOnStartup         = "true"
+    Cors__AllowedOrigins__0            = "https://staging.useorbit.org"
+    Frontend__BaseUrl                  = "https://staging.useorbit.org"
+    GooglePlay__RtdnAudience           = "https://api-staging.useorbit.org/api/subscriptions/play/rtdn"
+    Google__AllowedRedirectUris__0     = "https://staging.useorbit.org/auth-callback"
+    Google__AllowedRedirectUris__1     = "https://app.useorbit.org/auth-callback"
+    Marketing__ApiBaseUrl              = "https://api-staging.useorbit.org"
+    Sentry__Environment                = "staging"
+    Stripe__CancelUrl                  = "https://staging.useorbit.org/upgrade"
+    Stripe__SuccessUrl                 = "https://staging.useorbit.org/settings?subscription=success"
+    Supabase__Url                      = "https://staging-storage-disabled.invalid"
+    Waitlist__ApiBaseUrl               = "https://api-staging.useorbit.org"
   })
 
   production_web_values = {
@@ -147,6 +159,8 @@ resource "render_env_group" "production_api" {
     { for key in local.api_secret_keys : key => { value = data.aws_ssm_parameter.production_api[key].value } },
     { for key, value in local.production_api_database_values : key => { value = value } },
     { BotProtection__SecretKey = { value = aws_ssm_parameter.turnstile_secret["production"].value } },
+    { Ses__AccessKeyId = { value = aws_ssm_parameter.api_ses_access_key_id["production"].value } },
+    { Ses__SecretAccessKey = { value = aws_ssm_parameter.api_ses_secret_access_key["production"].value } },
   )
 }
 
@@ -158,6 +172,8 @@ resource "render_env_group" "staging_api" {
     { for key in local.staging_api_billing_keys : key => { value = data.aws_ssm_parameter.staging_api_billing[key].value } },
     { for key in local.database_keys : key => { value = local.staging_render_npgsql } },
     { BotProtection__SecretKey = { value = aws_ssm_parameter.turnstile_secret["staging"].value } },
+    { Ses__AccessKeyId = { value = aws_ssm_parameter.api_ses_access_key_id["staging"].value } },
+    { Ses__SecretAccessKey = { value = aws_ssm_parameter.api_ses_secret_access_key["staging"].value } },
   )
 }
 

infra/example.tfvars

@@ -1,2 +1,4 @@
-production_web_digest = "sha256:0000000000000000000000000000000000000000000000000000000000000000"
-staging_web_digest    = "sha256:0000000000000000000000000000000000000000000000000000000000000000"
+production_web_digest     = "sha256:0000000000000000000000000000000000000000000000000000000000000000"
+staging_web_digest        = "sha256:0000000000000000000000000000000000000000000000000000000000000000"
+production_email_provider = "Resend"
+staging_email_provider    = "Resend"

infra/ses-isolation.test.mjs

@@ -0,0 +1,66 @@
+import assert from "node:assert/strict"
+import { readFileSync } from "node:fs"
+import { dirname, join } from "node:path"
+import { fileURLToPath } from "node:url"
+import { test } from "node:test"
+
+const root = dirname(fileURLToPath(import.meta.url))
+const ses = readFileSync(join(root, "ses.tf"), "utf8")
+const configuration = readFileSync(join(root, "configuration.tf"), "utf8")
+const literal = (value) => value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")
+
+function block(source, header) {
+  const start = source.indexOf(header)
+  assert.notEqual(start, -1, `Missing ${header}`)
+  const opening = source.indexOf("{", start + header.length - (header.endsWith("{") ? 1 : 0))
+  let depth = 0
+  for (let index = opening; index < source.length; index++) {
+    if (source[index] === "{") depth++
+    if (source[index] === "}" && --depth === 0) return source.slice(opening + 1, index)
+  }
+  assert.fail(`Unclosed ${header}`)
+}
+
+test("each API environment uses its own SES configuration sets and event topic", () => {
+  const production = block(configuration, "production_api_values = {")
+  const staging = block(configuration, "staging_api_values = merge(")
+
+  for (const [environment, resource, topic, values] of [
+    ["production", "orbit", "ses_events", production],
+    ["staging", "staging", "ses_events_staging", staging],
+  ]) {
+    for (const stream of ["transactional", "marketing"]) {
+      const key = stream === "transactional" ? "Ses__TransactionalConfigurationSet" : "Ses__MarketingConfigurationSet"
+      assert.match(values, new RegExp(`${literal(key)}\\s*=\\s*aws_sesv2_configuration_set\\.${literal(resource)}\\["${literal(stream)}"\\]\\.configuration_set_name`), environment)
+    }
+    assert.match(values, new RegExp(`Ses__TopicArn\\s*=\\s*aws_sns_topic\\.${literal(topic)}\\.arn`), environment)
+  }
+})
+
+test("SES event destinations and subscriptions stay within the sending environment", () => {
+  for (const [resource, destination, topic, subscription, endpoint] of [
+    ["orbit", "sns", "ses_events", "ses_events_api", "https://api.useorbit.org/api/email/ses-events"],
+    ["staging", "sns_staging", "ses_events_staging", "ses_events_staging_api", "https://api-staging.useorbit.org/api/email/ses-events"],
+  ]) {
+    const event = block(ses, `resource "aws_sesv2_configuration_set_event_destination" "${destination}"`)
+    const hook = block(ses, `resource "aws_sns_topic_subscription" "${subscription}"`)
+    assert.match(event, new RegExp(`for_each\\s*=\\s*aws_sesv2_configuration_set\\.${literal(resource)}`))
+    assert.match(event, new RegExp(`topic_arn\\s*=\\s*aws_sns_topic\\.${literal(topic)}\\.arn`))
+    assert.match(hook, new RegExp(`topic_arn\\s*=\\s*aws_sns_topic\\.${literal(topic)}\\.arn`))
+    assert.ok(hook.includes(`endpoint  = "${endpoint}"`))
+  }
+})
+
+test("each API environment has credentials scoped to its own configuration sets", () => {
+  const productionPolicy = block(ses, 'resource "aws_iam_user_policy" "api_ses"')
+  const stagingPolicy = block(ses, 'resource "aws_iam_user_policy" "api_ses_staging"')
+  const keyParameter = block(ses, 'resource "aws_ssm_parameter" "api_ses_access_key_id"')
+  const secretParameter = block(ses, 'resource "aws_ssm_parameter" "api_ses_secret_access_key"')
+
+  assert.match(productionPolicy, /aws_sesv2_configuration_set\.orbit/)
+  assert.doesNotMatch(productionPolicy, /aws_sesv2_configuration_set\.staging/)
+  assert.match(stagingPolicy, /aws_sesv2_configuration_set\.staging/)
+  assert.doesNotMatch(stagingPolicy, /aws_sesv2_configuration_set\.orbit/)
+  assert.match(keyParameter, /each\.key == "production" \? aws_iam_access_key\.api_ses\.id : aws_iam_access_key\.api_ses_staging\.id/)
+  assert.match(secretParameter, /each\.key == "production" \? aws_iam_access_key\.api_ses\.secret : aws_iam_access_key\.api_ses_staging\.secret/)
+})