diff --git a/README.md b/README.md index 97e49d0..c6d0353 100644 --- a/README.md +++ b/README.md @@ -27,10 +27,13 @@ LiveKit OSS + Coturn, Kafka + kafka-ui, Vault, Dapr sidecar + placement, APISIX `.env.example` single source of truth, `.githooks/pre-commit` formatter, `infra/compose/e2e.yml` ephemeral overlay, `.github/workflows/ci.yml`, and `scripts/seed.sh`. See [docs/roadmap/phase-01-repository-tooling.md](docs/roadmap/phase-01-repository-tooling.md) -for the per-packet history. Phase 02a (Platform Kernel + Multi-Tenancy) has -kicked off; see [Phase 02a Status & Packets](docs/roadmap/phase-02a-kernel-tenancy.md) -for the 11-packet breakdown. Packet 0 — Kickoff has shipped; Packet 1 — -Foundation decisions (ADR-0023 / ADR-0024 / ADR-0028 to Accepted) is next. +for the per-packet history. Phase 02a (Platform Kernel + Multi-Tenancy) is +underway; see [Phase 02a Status & Packets](docs/roadmap/phase-02a-kernel-tenancy.md) +for the 11-packet breakdown. Packets 0 (Kickoff) and 1 (Foundation decisions +— [ADR-0023](docs/decisions/0023-strongly-typed-id-source-generator.md) Vogen, +[ADR-0024](docs/decisions/0024-api-versioning-policy.md) API versioning, +[ADR-0028](docs/decisions/0028-audit-log-partition-management.md) audit +partition mgmt) have shipped; Packet 2 — Shared Kernel core is next. Phase 02c (Hub Foundation, parallel, separate repo) starts once the 02a sockets it depends on are in place. diff --git a/docs/decisions/0002-initial-architecture.md b/docs/decisions/0002-initial-architecture.md index c1921f2..39f6f9b 100644 --- a/docs/decisions/0002-initial-architecture.md +++ b/docs/decisions/0002-initial-architecture.md @@ -76,7 +76,7 @@ pre-implementation, so the migration drag is zero): 2. **PostgreSQL major pinned to 18.x** per [ADR-0031](0031-postgresql-major-version.md). 18 is the longest- runway LTS available (EOL 2030-11), brings native `gen_uuid_v7()` - that the [ADR-0023 draft](README.md#open-adr-drafts) can adopt + that [ADR-0023](0023-strongly-typed-id-source-generator.md) adopts without an extension, and async I/O for sequential scans helps the partitioned `audit_log` ([ADR-0016](0016-audit-log-subsystem.md)) operator queries. RLS diff --git a/docs/decisions/0016-audit-log-subsystem.md b/docs/decisions/0016-audit-log-subsystem.md index 010ac3a..313680a 100644 --- a/docs/decisions/0016-audit-log-subsystem.md +++ b/docs/decisions/0016-audit-log-subsystem.md @@ -292,7 +292,8 @@ Three blocker-level architecture tests are added in Phase 02: ### Negative -- One more table partition to manage per month (`pg_partman` or custom Hangfire job). +- One more table partition to manage per month (managed by a Hangfire recurring job + per [ADR-0028](0028-audit-log-partition-management.md)). - Multi-entity command JSON can be large (>100 KB for bulk updates); truncation policy needed for very large operations. - The PII-redaction obligation (ADR-0026 in Nexora, equivalent rule in LearnStack) applies @@ -314,8 +315,14 @@ Three blocker-level architecture tests are added in Phase 02: `Value` field so JSON is readable. - The `Changes` JSON shape: array of `{ entityType, entityId, field, old, new }` for multi-entity commands; flat object `{ field, old, new }` for single-entity commands. -- Retention purge: `LearnStackJob` running monthly, dropping partitions older than the - tenant's configured retention window (default 7y / 2y by operation class). +- Retention follows the split defined in + [ADR-0028](0028-audit-log-partition-management.md): + partition lifecycle (creating monthly partitions, dropping them only at the + platform-max horizon) is owned by the `learnstack:audit:partition-management` + Hangfire job; per-tenant retention enforcement (default 7y / 2y by operation + class) is a separate row-level delete job + (`learnstack:audit:retention-purge`) operating *inside* the still-attached + partitions — never by partition drop. - PII redaction handler: subscribes to `UserGdprDeletedIntegrationEvent`, runs a parameterised UPDATE on audit rows containing the matching user reference, replaces PII fields with `[REDACTED]` placeholder constant. diff --git a/docs/decisions/0023-strongly-typed-id-source-generator.md b/docs/decisions/0023-strongly-typed-id-source-generator.md new file mode 100644 index 0000000..5c7f568 --- /dev/null +++ b/docs/decisions/0023-strongly-typed-id-source-generator.md @@ -0,0 +1,258 @@ +# ADR-0023: Strongly-Typed ID Source Generator — Vogen + +## Status + +Accepted + +**Date:** 2026-05-20 +**Deciders:** @platform + +## Decision Drivers + +- **Strongly-typed IDs are a hard requirement.** + [Standards 02 § Strongly-Typed Identifiers](../standards/02-backend-coding.md) + forbids raw `Guid` on the public surface of any entity. Every aggregate root and + cross-module reference uses `record struct CourseId(Guid Value) : IStronglyTypedId` + shape; the shape is fixed, the *emitter* is what this ADR picks. +- **The emitter has to produce four artefacts per ID type**, not just the struct: + EF Core value converter, `JsonConverter`, ASP.NET Core minimal-API model binder, and + OpenAPI schema mapping. Hand-rolling four boilerplate files per ID across ~15 modules + with multiple aggregates each is a maintenance crime. +- **Value objects share the same emitter shape.** Standards 02 calls out `Email`, `Slug`, + `LocaleCode` as value objects with invariants. An emitter that handles *both* IDs and + value objects with the same pattern is a leverage point; one that handles only IDs + leaves a second hand-rolled track for value objects. +- **Pre-implementation phase is the cheapest commit window.** Phase 02a's Packet 2 + introduces the first `Entity` and `AuditableEntity` bases, which depend on + the emitter at compile time. Choosing now means every subsequent packet wires the + same generator from the first commit; choosing later means a forced re-emit pass + across every module that already landed an ID type. +- **Provider lock-in budget is zero on the core paths.** ADR-0014 puts every external + port behind an interface (`IEventBus`, `ICacheService`, `IStorageProvider`, …). The + ID emitter is a compile-time build dependency, not a runtime port — it cannot sit + behind an interface — so the chosen library has to be one we are comfortable depending + on for the platform's lifetime, or removable with a one-shot find-and-replace if it + goes unmaintained. +- **PostgreSQL 18 native `gen_uuid_v7()` is available** ([ADR-0031](0031-postgresql-major-version.md)). + The emitter has to play well with DB-side `DEFAULT gen_uuid_v7()` as well as + app-side `Guid.CreateVersion7()` (.NET 9+) — both code paths exist in the codebase + (DB-side for high-volume audit / outbox tables; app-side for aggregates that need + the ID before flush). + +## Considered Options + +1. **Vogen** (chosen). MIT, source generator authored by Steven Giesel; emits a `record + struct` value object (works for both IDs and richer value objects), plus + pre-baked `EFCoreValueConverter`, `SystemTextJsonConverter`, `TypeConverter`, + ASP.NET Core minimal-API model binder, Dapper handler, `INumber`/`IParsable` + conformance, OpenAPI schema customizer. Annotation-based opt-in: + `[ValueObject(...)]` on a partial record struct. +2. **StronglyTypedId** (rejected). MIT, Andrew Lock's library; the original entrant + in this space. Focused exclusively on IDs (no broader value-object support), + smaller surface, older codebase. Active maintenance but the project's velocity + has slowed since 2024. +3. **Custom in-house emitter** (rejected). A Roslyn source generator in + `backend/analyzers/` emitting the four-artefact set per `[Id]`-marked struct. + Maximum control, zero third-party trust, full alignment with LearnStack's + own conventions. + +## Decision + +LearnStack uses **Vogen** as the source generator for strongly-typed IDs **and** +value objects. + +- Every aggregate's ID type is declared as a partial `record struct` + annotated with `[ValueObject(conversions: Conversions.EfCoreValueConverter | + Conversions.SystemTextJson | Conversions.TypeConverter)]`. The + `TypeConverter` flag is what makes ASP.NET Core route-parameter binding + work — Vogen does not ship a separate `AspNetCoreRouteParameter` flag. + OpenAPI schema customisation is wired separately (see Implementation Notes + § OpenAPI). +- The Vogen **build-time** generator (`Vogen` package) is referenced via + `PrivateAssets="all"` on **each project that hosts `[ValueObject<...>]` + declarations** — `LearnStack.SharedKernel` for cross-cutting value objects + (`Email`, `Slug`, `LocaleCode`, `Money`), and **each + `LearnStack.Modules..Domain`** for the module's aggregate-root IDs. The + reference is centralised via `Directory.Build.props` so adding a new module + picks the generator up automatically. The **runtime** assembly + `Vogen.SharedTypes` (which carries the `Conversions` enum and a handful of + helper types the generated code calls into) flows transitively to + consumers — this is a small (~10 KB) MIT dependency, not a heavyweight + runtime. +- Value objects with invariants (`Email`, `Slug`, `LocaleCode`, `Money`, …) + follow the same annotation pattern, with a `Validate` static method + enforcing the invariant. +- The `IStronglyTypedId` marker interface ([Standards 02 + § Strongly-Typed Identifiers](../standards/02-backend-coding.md)) is + implemented by every Vogen-emitted ID struct. The interface stays; Vogen + is just the body. + +The choice covers the four-artefact emission requirement, the value-object case, the +PostgreSQL 18 DB-side UUIDv7 path (Vogen can wrap any `Guid`, including those minted +by `gen_uuid_v7()`), and a `[Description]`/`[ReadOnly]` annotation surface Roslyn +analyzers can read for additional compile-time rules. + +## Context + +### Why Vogen over StronglyTypedId + +StronglyTypedId was the field's first mover and would have worked for the ID case. +Three things pushed the choice: + +- **Value-object coverage.** Standards 02 already lists `Email`, `Slug`, `LocaleCode` + as value objects with invariants. Vogen's `[ValueObject]` annotation generates the + same emitter set for these as for IDs — one generator covers both surfaces. With + StronglyTypedId, IDs use the library and value objects use a hand-rolled path, with + divergent EF/JSON conversion patterns. +- **OpenAPI schema customizer.** Vogen ships a Swashbuckle / Microsoft.OpenApi schema + filter that registers the underlying primitive (`format: uuid`) on every emitted + type. StronglyTypedId requires a manual `MapType(() => new + OpenApiSchema(...))` per ID type in the OpenAPI setup — across 60+ ID types + projected for Phase 02a–08, that is a real cost. +- **Maintenance velocity.** Vogen released 12 versions in 2024 and 3 in early 2025; + StronglyTypedId's release cadence has slowed. Both are MIT, both are forkable in + a worst case, but the active-maintenance signal favours Vogen. + +### Why not custom + +A custom Roslyn source generator would have produced the same artefacts. We +considered it because: + +- The IL it emits is small and well-understood; we could match Vogen's output by hand. +- We have no external pressure to ship Vogen's "Bogus customization", "Dapper + handler", or other peripheral surfaces. +- LearnStack already has a Roslyn analyzer project (`backend/analyzers/` per + ADR-0032's `LearnStackException-DomainExceptionThrow`); adding one more isn't + conceptually new. + +Three things outweighed the appeal: + +- **Vogen has 5+ years of community-found bugs already fixed.** Equality semantics, + EF Core conversion edge cases (nullable navigation), JSON deserialization of `null` + vs `0`, OpenAPI schema for nested generic types — Vogen's issue tracker is the + receipts. Rebuilding that from scratch costs months we should spend on the domain. +- **The maintenance interface is `git pull`, not "find the file we wrote in + 2026".** A custom generator is a forever-owned artefact; an MIT package is owned + externally with a clean exit (fork) if it stalls. +- **Roslyn source generator design has a steep learning curve.** Generator-author + experience (incremental generators, attribute discovery, cancellation tokens, + `IIncrementalGenerator` vs the older `ISourceGenerator`) is non-trivial; the + library represents real expertise we'd otherwise rediscover. + +### What would change our minds + +- Vogen license shift away from MIT. +- Vogen archived / abandoned for > 12 months on .NET 11+ without a community fork + picking it up. +- A LearnStack-specific emission requirement Vogen cannot model via a custom + `Conversions` enum entry or a `[Description]`-style annotation pair — for instance, + a regulatory ID format we needed to enforce at compile time that Vogen's + validation hook could not express. + +### What we explicitly punted on + +- **UUIDv7 source.** Both DB-side (`gen_uuid_v7()`) and app-side + (`Guid.CreateVersion7()`) are valid; the choice between them is per-aggregate (high- + volume insert paths like `audit_log` / `outbox_messages` prefer DB-side, aggregates + that need the ID before flush prefer app-side). This stays a Standards 05 (database) + micro-rule, not an ADR. +- **Vogen-emitted type comparison semantics.** Default record-struct equality is + by-value, which is correct for IDs; we accept Vogen's defaults rather than tuning + them. + +## Consequences + +### Positive + +- Compile-time strongly-typed IDs across the codebase from the first aggregate + onwards; no raw-`Guid` slippage at the boundary. +- EF Core, JSON, OpenAPI, ASP.NET route binding all "just work" per ID type — no + per-ID boilerplate. +- Value objects (`Email`, `Slug`, …) reuse the same annotation pattern; one shape + to teach the team. +- OpenAPI spec generation knows the underlying primitive; the SDK generator (per + ADR-0024) emits clean wrapper types for SDK consumers without manual schema + hints. +- Roslyn-analyzer-friendly: future architecture tests can read Vogen's + `[ValueObject]` attribute to enforce "every aggregate root ID uses Vogen, not + raw `Guid`". + +### Negative + +- One more compile-time dependency on a third-party generator. Bumping Vogen major + versions can break emission; we pin the version in `Directory.Packages.props` and + treat upgrades as deliberate ADR-adjacent changes. +- Source generators slow incremental build slightly. Measured impact in Phase 02a's + scaffold is < 200ms on a clean build; reassess if the codebase grows to a point + where the generator dominates compile time. +- Diagnostics on emitted code reference Vogen-generated source files — IDE + "go to definition" lands on generated `obj/` files; team has to know to + navigate to the partial declaration instead. + +### Neutral + +- The `IStronglyTypedId` interface in `LearnStack.SharedKernel` stays as a + type-system contract; Vogen-generated structs implement it. +- Consumer projects gain a small (~10 KB) **runtime** dependency on + `Vogen.SharedTypes` (which carries the `Conversions` enum and a handful of + helper types referenced by generated code). The Vogen generator package + itself (`Vogen`) stays build-time-only via `PrivateAssets="all"`. + +## Implementation Notes + +- **Package references:** `Directory.Packages.props` pins ``. **Every project that hosts `[ValueObject<>]` declarations** — `LearnStack.SharedKernel` (for cross-cutting value objects: `Email`, `Slug`, `LocaleCode`, `Money`) and **each `LearnStack.Modules..Domain`** (for its aggregate-root IDs) — adds ``. A `Directory.Build.props` rule under `backend/src/Modules/` keeps the per-module addition automatic when a new module is scaffolded. Source generators only run on projects that reference the generator package; transitive references do **not** carry the generator (this is a `PrivateAssets="all"` semantics constraint, not a Vogen quirk). +- **Naming convention (per Standards 02):** ID type names end in `Id` + (`TenantId`, `OrganizationId`, `CourseId`, …); value object types are named + for the concept (`Email`, not `EmailValueObject`). +- **Default conversions enum:** every ID + value object opts into the same + `Conversions` mask: `EfCoreValueConverter | SystemTextJson | TypeConverter`. The + `TypeConverter` member carries ASP.NET Core minimal-API + MVC route-parameter + binding (Vogen does not expose a separate `AspNetCoreRouteParameter` flag). + A `LearnStack.SharedKernel.VogenDefaults` const captures the mask so the + annotation reads `[ValueObject(LearnStackVogenDefaults.IdMask)]`. +- **OpenAPI schema mapping:** Vogen does **not** ship a `Conversions.SwaggerSchemaFilter`-style + flag. Schema customisation is wired one of two ways: (a) an assembly-level + `[VogenDefaults(openApiSchemaCustomizations: ...)]` attribute in + `LearnStack.SharedKernel` so every emitted type advertises its primitive + shape, or (b) a custom `IOpenApiSchemaTransformer` in `LearnStack.Api` + (Microsoft.AspNetCore.OpenApi) that detects Vogen-generated wrappers via the + generated `IVogenValueObject`-marker interface and emits the underlying + primitive (`format: uuid`, `format: int64`, …). Packet 4 picks one when API + conventions wiring lands; both paths are documented in Vogen's upstream docs. +- **EF Core registration:** the `EfCoreValueConverter` Vogen flag **generates** + the converter type per ID; it does not auto-register it. Each module's + `DbContext.OnConfiguring` (or a shared `IModelCustomizer`) calls + `configurationBuilder.Properties().HaveConversion()` + for each Vogen-emitted ID. A `LearnStack.SharedKernel.Infrastructure` + helper `ModelConfigurationBuilder.RegisterVogenIds(Assembly[])` reflects + over the Domain assemblies and applies the registration in one call. +- **Architecture test (lands in Phase 02a Packet 2):** + `Aggregate_Roots_Use_StronglyTypedId` — every type implementing + `IAggregateRoot` has `TId : IStronglyTypedId`, and every such + `TId` carries `[ValueObject]`. Catalogued under + [21-architecture-tests-catalogue.md](../standards/21-architecture-tests-catalogue.md) + when the first ID lands. +- **UUIDv7 minting:** + - DB-side (`gen_uuid_v7()` per ADR-0031) for `audit_log`, `outbox_messages`, + `inbox_messages`, `idempotency_keys` — high-volume append-only tables. + - App-side (`Guid.CreateVersion7()`) for aggregates that need the ID before + `SaveChangesAsync()` to emit domain events / outbox writes referencing the + new aggregate ID. +- **PostgreSQL 18 alignment:** Vogen-emitted `Guid` wrapper types are wire-compatible + with `uuid` columns; the `gen_uuid_v7()` `DEFAULT` is a server-side concern, + invisible to Vogen. + +## Amendments + +_(none yet)_ + +## References + +- [Standards 02 § Strongly-Typed Identifiers](../standards/02-backend-coding.md) +- [ADR-0031 PostgreSQL — Start on 18.x](0031-postgresql-major-version.md) — native + UUIDv7 widens the design space; this ADR commits to Vogen-emitted wrappers on + that primitive. +- [ADR-0032 Exception Handling, Logging, and Observability](0032-exception-handling-logging-and-observability.md) + — establishes the `backend/analyzers/` Roslyn analyzer location; future ID-shape + analyzers live alongside. +- [Vogen on GitHub](https://github.com/SteveDunn/Vogen) — upstream project (MIT). diff --git a/docs/decisions/0024-api-versioning-policy.md b/docs/decisions/0024-api-versioning-policy.md new file mode 100644 index 0000000..faf64f0 --- /dev/null +++ b/docs/decisions/0024-api-versioning-policy.md @@ -0,0 +1,335 @@ +# ADR-0024: API Versioning Policy + +## Status + +Accepted + +**Date:** 2026-05-20 +**Deciders:** @platform + +## Decision Drivers + +- **OpenAPI spec generation starts in Phase 02a Packet 4.** The current + scaffold exposes only the unversioned infrastructure endpoint `GET /healthz`; + before any versioned endpoint lands we owe a written versioning policy so + the contract is stable from the first byte. Late-arriving policies create + retroactive breaking-change classification ambiguity. +- **URL-based versioning is already in the corpus, but the prefix shape is + inconsistent.** [Standards 04 § Versioning](../standards/04-api-design.md) + committed to URL versioning (no header-based versioning). The APISIX + gateway is configured to route `/api/v*/**` (per + [architecture/30](../architecture/30-api-gateway.md) and + `infra/apisix/apisix.yaml`); some older standards/architecture examples + still write `/v1/...` without the `/api/` prefix. This ADR fixes the + inconsistency on `/api/v{N}/` and codifies *the rest* of the policy that + bullets in Standards 04 sketched but never closed: deprecation cadence, + exact header set, OpenAPI marking, SDK-generation implications, the + breaking/non-breaking rule. +- **Tenant-facing SDK consumers are the constraint.** The Hub HTTPS contract + ([ADR-0019](0019-learnstack-hub.md)) is internal mTLS at four endpoints — its + versioning is governed by the Hub repo's own ADR set, not this one. The + externally-exposed contract is the *tenant-facing* `/api/v1/*` surface, hit by + the typed SDK generated from OpenAPI, by tenant-side integrations, by webhook + consumers, and (in Phase 04+) by the public storefront. Their upgrade cadence + determines our deprecation window, not internal engineering velocity. +- **SaaS continuous-deploy vs. SelfHosted release cadence are different.** SaaS + tenants get the latest binary on the next deploy; SelfHosted instances upgrade + on their own clock (potentially quarterly). A deprecation window that works + only for SaaS leaves SelfHosted tenants stuck on dead endpoints; one that + works only for SelfHosted slows everyone else. The policy has to accommodate + both via the **30-day grace** model of ADR-0020. +- **RFC 8594 (`Sunset` header) and RFC 9745 (`Deprecation` header, published + March 2025) are the industry-standard signalling mechanism.** Stripe, + GitHub, Google, Twilio all use this pair. Inventing a proprietary + signalling scheme on a multi-tenant SaaS in 2026 is a self-inflicted wound. + +## Considered Options + +1. **6-month deprecation window + RFC 8594 `Sunset` header + `Deprecation` header + + OpenAPI `deprecated: true` + `x-sunset` extension** (chosen). +2. **3-month deprecation window** (rejected). Faster internal iteration, but + too tight for tenant-side SDK consumers — a tenant on a quarterly release + train cannot reasonably adapt to a Q1 deprecation announcement and ship the + fix before Q2. +3. **12-month deprecation window** (rejected). Safer for the slowest SelfHosted + tenants, but doubles the cost of dual-version maintenance on every breaking + change. With Phase 11+ projected to see real version-2 endpoints, paying + 12-month maintenance debt on each one is more than the gain. +4. **Header-based versioning (`X-API-Version: 1`)** (rejected at the Standards 04 + level already). URL versioning is industry-default for tenant-facing APIs; + it makes routing trivial in APISIX, lets gateways enforce version-specific + rate limits and quotas, and shows up plainly in access logs. + +## Decision + +LearnStack's tenant-facing HTTP API uses **URL-based versioning** under the +`/api/v{N}/` prefix (`/api/v1/*`, `/api/v2/*`, …) with a **6-month deprecation +window** signalled by RFC 9745 + RFC 8594 headers and OpenAPI metadata. The +`/api/` prefix matches the existing APISIX gateway routes (`/api/v*/**` per +[ADR-0015](0015-api-gateway-apisix.md) + [architecture/30](../architecture/30-api-gateway.md)) +and is the only canonical public route shape for versioned API endpoints. + +### The version axis + +- **Major versions in the URL.** A new major version is the only way a breaking + change reaches a tenant. Mainline development continues in `/api/v1`; + `/api/v2` appears only when a breaking change is unavoidable and a new major + has been ADR'd at architecture-doc level. +- **No minor versions in the URL.** All non-breaking changes (additive fields, + new optional query params, new endpoints under an existing resource, looser + validation) ship to the same major. +- **Two adjacent majors coexist.** While `/api/v2` is current, `/api/v1` runs + in parallel for its full deprecation window. Three concurrent majors is not + supported; cutting `/api/v3` requires `/api/v1` to have reached its Sunset + date first. + +### What counts as a breaking change + +| Breaking | Non-breaking | +|---|---| +| Removing a field from a response | Adding a field to a response | +| Renaming a field in a response | Adding a new endpoint | +| Removing a field from a request | Adding an optional query param | +| Changing the type of a field | Loosening a validator (more inputs accepted) | +| Adding a required request field | Adding a new value to an open-set enum (`x-extensible-enum: true`) | +| Removing an enum value from a response | Adding a header to a response | +| **Adding a value to a closed-set enum** (the default — see below) | Re-ordering JSON object keys | +| Tightening a validator (rejecting inputs that used to pass) | Improving an error message under an unchanged code | +| Changing HTTP status codes for an outcome | Changing internal implementation | +| Renaming a path segment | | + +Enums are **closed-set by default**: a consumer that round-trips an unknown +value is allowed to throw, so adding a new value silently breaks consumers. +Opt-in to open-set behaviour by marking the schema `x-extensible-enum: true`, +which tells SDK codegen to emit a forward-compatible "unknown" branch. +Flipping an enum from open to closed is itself a breaking change. + +### Lifecycle of a deprecated endpoint + +1. **T = 0 (announcement).** New major `/api/v2/*` ships. The old `/api/v1/*` + endpoint acquires: + - `Deprecation: @` HTTP response header on every call, + using the sf-date format from RFC 9651 § 3.3.7 as the value (per + [RFC 9745 § 2](https://www.rfc-editor.org/rfc/rfc9745.html)). + - `Sunset: ` HTTP response header pointing 6 months into the + future. The value is an HTTP-date per RFC 9110 § 5.6.7 (e.g. + `Sun, 15 Jan 2027 00:00:00 GMT`); per + [RFC 8594 § 3](https://www.rfc-editor.org/rfc/rfc8594.html#section-3). + - `Link: ; rel="successor-version"` + HTTP response header pointing at the migration guide. + - OpenAPI spec marks the operation `deprecated: true` and adds + `x-sunset: ` and `x-successor: /api/v2/...` extensions for SDK + codegen to surface. +2. **T = 0 → T = 6mo (parallel run).** Both versions accept traffic, are + monitored, and emit identical audit-log entries (same operation key, version + stamped in `metadata.api_version`). Per-tenant usage telemetry surfaces + "tenant X is still on `/api/v1/Y`" so account managers can reach out. +3. **T = 6mo (sunset).** The `/api/v1/*` endpoint returns + `410 Gone` with RFC 7807 Problem Details (problem-type host per + [Standards 04 § Errors](../standards/04-api-design.md)): + ```json + { + "type": "https://errors.learnstack.dev/api-version-sunset", + "title": "API version sunset", + "status": 410, + "detail": "GET /api/v1/courses was sunset on 2027-01-15. Use GET /api/v2/courses.", + "successor": "/api/v2/courses", + "migrationGuide": "https://docs.learnstack.dev/v2/migration" + } + ``` + +### Per-deployment-mode behaviour + +- **SaaS:** the 6-month clock starts on the SaaS release cutting the + `/api/v2`. All SaaS tenants migrate within the window; tenant-managed + integrations that miss the window get `410 Gone`. +- **Dedicated:** identical to SaaS; the dedicated cluster operates on the same + binary cadence. +- **SelfHosted (Online + Air-Gapped):** the 6-month clock starts on the + SelfHosted release that introduces `/api/v2`. Self-hosted operators may + delay upgrading past sunset; their `/api/v1` traffic continues to return + `410 Gone` on the binary that has the new release applied. Operators who + want longer parallel coexistence can stay on the prior binary inside their + 30-day grace; this is a customer-side choice, not a platform commitment. + +### OpenAPI marking + +Every operation in the OpenAPI spec carries: + +```yaml +get: + operationId: getCourse + deprecated: false + x-version-introduced: v1 + responses: + '200': + ... +``` + +Deprecated operations add: + +```yaml +get: + operationId: getCourse + deprecated: true + x-version-introduced: v1 + x-sunset: 2027-01-15T00:00:00Z + x-successor: /api/v2/courses/{id} + x-migration-guide: https://docs.learnstack.dev/v2/migration#getCourse +``` + +The SDK generator reads these to: + +- Mark generated SDK methods `[Obsolete("...")]` with the sunset date. +- Surface the migration-guide URL in the method's XML doc comment. +- Emit a compile-time warning that turns into an error 30 days before sunset. + +## Context + +### Why 6 months and not 3 / 12 + +We measured against three signals: + +- **Tenant-side SDK regeneration cadence.** Tenant integrations are typically + on Node or .NET stacks with quarterly release trains. A 3-month window asks + every tenant to ship two consecutive releases against a breaking change; a + 6-month window comfortably fits one cycle. +- **Dual-maintenance cost.** Every endpoint in a deprecation window has to be + kept correct in both majors, audited in both majors, tested in both majors. + Doubling that cost to 12 months for every breaking change is more dual-version + carrying than the slow-SelfHosted tail warrants. +- **Industry benchmarks.** Stripe → 12 months (very conservative, billing-grade + external surface). GitHub → 12 months (DX-sensitive). Twilio → 6 months + (similar shape to LearnStack — multi-tenant SaaS with SDK consumers). + Auth0 → 6 months (very similar mix of tenant-facing + admin surface). + LearnStack's "tenant-facing API + SDK + internal Hub" mix puts us in the + Twilio/Auth0 cluster, not the Stripe billing cluster. + +### Why URL versioning and not header versioning + +- **APISIX (ADR-0015) routes on URL prefix.** Different version routes can carry + different rate-limit budgets, JWT realm requirements, plugin chains without + any conditional logic. Header-based versioning would push routing decisions + into post-gateway middleware. +- **Caching layer (Valkey via Dapr) keys on URL.** Header-based versioning would + require explicitly varying cache keys on `X-API-Version`; URL versioning is + cache-friendly by construction. +- **Observability is cleaner.** Access logs, OTel traces, audit-log entries + all carry the URL natively; we never have to "lookup the header to find the + version" downstream. + +### Why we accept the cost of dual-version maintenance + +Every breaking change carries a real cost: tests run against both majors, audit +config covers both, RLS policies are version-agnostic but the endpoints that +hit them differ. This cost is the *price* of being a reasonable steward of the +contract; the alternative (no versioning, breaking changes shipped silently) +is not a platform, it's a script we run. + +### What would change our minds + +- A tenant-facing SDK consumer base large enough that 6 months is empirically + insufficient (telemetry: > 5% of tenants still on `/api/v1/X` after 5 months + from announcement on three consecutive breaking changes). +- A regulatory requirement forcing 12-month coexistence (e.g. financial-sector + audit reproducibility). +- An OpenAPI/SDK toolchain change that makes dual-version maintenance cheap + enough to justify a longer window — for instance, if every endpoint were + expressible as a versioned schema with an `If-Match`-style version negotiator + built in. + +### What we explicitly punted on + +- **Per-endpoint deprecation overrides.** All deprecations get 6 months; we do + not allow per-endpoint shorter or longer windows. A "this endpoint needs 3 + months because it's wrong" case is rare enough that the policy refuses to + encode it; revisit if it actually happens. +- **Pre-v1 / `/api/v0`.** No `/api/v0/*` endpoints exist or will exist; + `/api/v1` is the first contract. +- **Internal `/api/internal/*` (Hub) versioning.** That surface uses its own + versioning per [ADR-0019](0019-learnstack-hub.md); this ADR governs only + tenant-facing `/api/v*/*`. + +## Consequences + +### Positive + +- Single, documented breaking-change policy from the first endpoint forward. +- Tenant-side SDK consumers get RFC-standard signalling — they can build their + own automation against `Sunset` / `Deprecation` headers without + LearnStack-specific tooling. +- OpenAPI spec is the contract; SDK generation reads it directly. No + out-of-band documentation lookup. +- APISIX routing and observability stay clean (URL-based). +- 410 Gone with Problem Details means a tenant migration that lapsed gets a + precise, machine-readable error — not an opaque 404. + +### Negative + +- Every major version doubles maintenance temporarily (6 months parallel). + Architecture tests + integration tests run against both surfaces; CI cost + grows linearly with the number of deprecation windows currently open. +- The 6-month commitment ties LearnStack's hands on the slowest-moving tenant. + A tenant whose CTO took a sabbatical mid-window will be locked out at month + 6+1; the customer-success process has to surface this proactively. +- Open-set enum decisions (`x-extensible-enum: true`) become semi-permanent — + flipping an enum from open to closed is itself a breaking change. + +### Neutral + +- SelfHosted tenants opt into their own version cadence by their upgrade + schedule; the platform doesn't try to enforce a global clock. +- The Hub contract is unaffected and governed elsewhere. + +## Implementation Notes + +- **Controller / endpoint registration:** every endpoint sits under `/api/v{N}/` + in the routing table; no version-less endpoints exist. Phase 02a Packet 4 + wires the URL convention via ASP.NET Core route conventions. +- **OpenAPI generation:** Phase 02a Packet 4 also wires Microsoft.OpenApi / + `Microsoft.AspNetCore.OpenApi` to emit `/openapi/v{N}.json` per major and to + read the `x-version-introduced`, `x-sunset`, `x-successor`, `x-migration-guide`, + and `x-extensible-enum` extensions from attribute metadata. +- **Sunset header emission:** an ASP.NET Core middleware + (`ApiVersioningHeadersMiddleware`) reads attribute metadata on the matched + endpoint and appends `Deprecation` / `Sunset` / `Link` headers when the + endpoint is marked `[Deprecated(sunset: "...", successor: "...")]`. +- **Architecture test (lands when the first `/v2` endpoint is added):** + `Every_Deprecated_Endpoint_Has_Sunset_And_Successor` — every controller + action with `[Obsolete]` declares `Sunset` + `Successor`. Catalogued under + [21-architecture-tests-catalogue.md](../standards/21-architecture-tests-catalogue.md). +- **Architecture test (lands in Phase 02a Packet 4):** + `Every_Endpoint_Is_Under_Versioned_Route` — no controller action exists + outside `/api/v{N}/...`. +- **`/healthz` exemption:** `GET /healthz` and `GET /readyz` are unversioned; + they are infrastructure endpoints consumed by the orchestrator, not the + versioned API surface. The architecture test allow-lists them by name. +- **SDK codegen:** the generated typed SDK (`packages/sdk`) ships a class per + major; `LearnStackClient.V1` and (later) `LearnStackClient.V2` coexist for + one cycle. The codegen reads `x-sunset` to emit `[Obsolete]` with the + sunset date. +- **Migration guide URL convention:** `https://docs.learnstack.dev/v{N}/migration` + is the public landing page; the `x-migration-guide` extension narrows to the + per-endpoint anchor (`#getCourse`, …). The actual hosting of `docs.learnstack.dev` + is a deployment concern — for the pre-MVP phase the URL resolves to a 404 + placeholder; the *header pattern* is the contract from Day 1. + +## Amendments + +_(none yet)_ + +## References + +- [Standards 04 § Versioning](../standards/04-api-design.md) +- [ADR-0015 API Gateway with APISIX](0015-api-gateway-apisix.md) — APISIX routes on + URL prefix; URL versioning is gateway-friendly. +- [ADR-0019 LearnStack Hub](0019-learnstack-hub.md) — the four-endpoint internal + `/api/internal/*` surface has its own versioning rules. +- [ADR-0020 Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) + — SaaS / Dedicated / SelfHosted cadence model; the 30-day grace touches + per-deployment-mode behaviour here. +- [RFC 8594 — The Sunset HTTP Header Field](https://www.rfc-editor.org/rfc/rfc8594.html) — Sunset header definition; value is an HTTP-date per RFC 9110 § 5.6.7. +- [RFC 9745 — The Deprecation HTTP Response Header Field](https://www.rfc-editor.org/rfc/rfc9745.html) — Deprecation header definition (March 2025; supersedes `draft-ietf-httpapi-deprecation-header`). +- [RFC 9651 — Structured Field Values for HTTP](https://www.rfc-editor.org/rfc/rfc9651.html) — defines the sf-date format the Deprecation header uses (§ 3.3.7). +- [RFC 9110 — HTTP Semantics § 5.6.7 HTTP-date](https://www.rfc-editor.org/rfc/rfc9110.html#section-5.6.7) — the date format the Sunset header uses. +- [RFC 7807 — Problem Details for HTTP APIs](https://www.rfc-editor.org/rfc/rfc7807.html). diff --git a/docs/decisions/0028-audit-log-partition-management.md b/docs/decisions/0028-audit-log-partition-management.md new file mode 100644 index 0000000..da0f29c --- /dev/null +++ b/docs/decisions/0028-audit-log-partition-management.md @@ -0,0 +1,264 @@ +# ADR-0028: `audit_log` Partition Management — Hangfire Recurring Job + +## Status + +Accepted + +**Date:** 2026-05-20 +**Deciders:** @platform + +## Decision Drivers + +- **`audit_log` is partitioned by month from Day 1.** + [ADR-0016 § audit_log table](0016-audit-log-subsystem.md) commits the table + to monthly partitions for cheap drop-based retention. Partition lifecycle + (creating next month's partition, dropping expired ones) is a *recurring* + operation, not a one-time migration, so it has to be owned by code that runs + on a schedule. +- **Three deployment modes, one binary.** [ADR-0020](0020-triple-deployment-hybrid-license.md) + commits LearnStack to SaaS / Dedicated / SelfHosted from a single codebase. + SelfHostedAirGapped specifically prohibits any runtime dependency the + customer cannot ship inside their air gap; an extra PostgreSQL extension + pushes the customer's DBA workload onto the platform's deployment story. +- **Partition lifecycle is application-domain logic, not DBA tooling.** The + rules ("create next month's partition by the 25th of this month", "drop + partitions older than the longest retention window across all tenants", + "honour the per-tenant `AuditConfig` retention overrides") live in the + audit subsystem's design ([architecture/31-audit-subsystem.md + § Retention](../architecture/31-audit-subsystem.md)). Putting them in a + database extension means moving rules out of the codebase into per-environment + config, which conflicts with the standards-corpus single-source-of-truth + posture. +- **A Hangfire job runner already exists.** The audit retention purge is already + committed to Hangfire ([Standards 18](../standards/18-audit-coverage.md) + § Retention; [architecture/31-audit-subsystem.md + § 8 Retention](../architecture/31-audit-subsystem.md)). Partition + management can ride the same `LearnStackJob`-shaped surface instead of + introducing a second mechanism. +- **PostgreSQL 18** ([ADR-0031](0031-postgresql-major-version.md)) ships with + native declarative partitioning improvements; `CREATE TABLE ... PARTITION OF` + + `DETACH PARTITION CONCURRENTLY` cover the operations a partition manager + needs, without requiring an extension. +- **`pg_partman` is genuinely good at this — but at the cost of an extension + binary** that has to be installed in every environment, version-matched to + PostgreSQL major, and validated for the air-gapped story. + +## Considered Options + +1. **Hangfire recurring job (`learnstack:audit:partition-management`)** + (chosen). A C# job in `LearnStack.Infrastructure.Audit`, scheduled via + Hangfire's `RecurringJob.AddOrUpdate(...)`. Runs daily; creates next month's + partition if missing; drops partitions whose end timestamp + the platform's + max retention window < `now`. Per-tenant retention is enforced by a separate + row-level purge job (`learnstack:audit:retention-purge`) operating *inside* + the still-attached partitions. +2. **`pg_partman` PostgreSQL extension** (rejected). The de facto standard for + declarative time-partitioned tables; battle-tested at scale. Configured via + `partman.create_parent(...)` and a background `partman.run_maintenance()` + call (either from pg_cron or a Hangfire-triggered SQL call). Drops the + custom logic to a config-table lookup. +3. **Manual SQL via EF Core migrations** (rejected outright). Partition lifecycle + is recurring — every month a new migration would have to land. This conflicts + with the "migrations append-only after merge" rule + ([Standards 05](../standards/05-database.md)) and makes the Self-Hosted + upgrade story brittle. + +## Decision + +LearnStack manages `audit_log` partitions via a **Hangfire recurring job** +named `learnstack:audit:partition-management`, living in +`LearnStack.Infrastructure.Audit`. + +- **Schedule:** daily at 02:15 UTC (after the daily retention purge slot but + before the bulk of the next day's audit traffic ramps up). +- **Create-ahead:** the job ensures the **current** month's partition exists + (idempotent), the **next** month's partition exists, and the + **month-after-next** partition exists. The two-month-ahead horizon protects + against a failed run on the 25th of a month not blocking the 1st of the + next month's writes. +- **Drop policy:** the job drops a partition only when its end timestamp + + the **platform's maximum retention window** (10 years for safety — + longer than every per-class retention in + [Standards 18 § Retention](../standards/18-audit-coverage.md)) is in the + past. Per-tenant retention enforcement is **not** done by partition drop; + it is done by row-level delete in the + `learnstack:audit:retention-purge` job operating inside the still-attached + partitions. +- **Partition shape:** `PARTITION BY RANGE (timestamp)` with monthly + partitions `audit_log_YYYY_MM` covering `[YYYY-MM-01, (YYYY-MM+1)-01)`. + This matches the shape defined in + [ADR-0016 § audit_log table](0016-audit-log-subsystem.md) and + [architecture/31](../architecture/31-audit-subsystem.md); this ADR owns + *lifecycle*, not the table layout. +- **Failure mode:** the job is idempotent — re-running creates no duplicate + partitions and drops no live data. If a job run fails, Hangfire retries + the next day; if four consecutive runs fail the job emits a + `learnstack.audit.partition.management.failed` event for operator alerting. +- **No `pg_partman` runtime dependency.** No environment ever needs to + install the extension. + +## Context + +### Why Hangfire over `pg_partman` + +The choice is genuinely close on technical merit; `pg_partman` is more polished +at the SQL surface. Three things tipped the balance toward Hangfire: + +- **Air-gapped deployment story.** SelfHostedAirGapped is a real first-class + mode in [ADR-0020](0020-triple-deployment-hybrid-license.md). Adding a + PostgreSQL extension means the air-gapped customer's release bundle has to + ship the extension binaries matched to their PG major, with their + validation work, with their patching cadence on top. Hangfire is C# code + that ships in the LearnStack binary itself. +- **Single source of truth for retention rules.** Standards 18 already owns + the retention class table (7y / 2y / per-tenant override). Putting partition + rules in `pg_partman` config means two sources for "how long does this row + live": one in code (Standards 18), one in `partman.part_config` (the + extension's config table). Drift is inevitable; choosing one source keeps + the corpus clean. +- **The retention purge job already exists in Hangfire.** Two related jobs in + two different mechanisms (one Hangfire, one `pg_partman` + pg_cron) is more + cognitive load than one mechanism handling both. + +### What `pg_partman` would have bought us + +- **DBA familiarity.** Operators with a `pg_partman` background would have a + shorter on-call ramp. +- **Tested-at-scale operations.** `BEFORE/AFTER` create-partition hooks, + cross-database replication-friendly partition naming, support for + retention by partition rather than row — all robust and battle-tested. +- **Less code to write and own.** A single SQL call (`partman.create_parent`) + per environment vs. a custom Hangfire job + its tests + its monitoring. + +We accept losing those in exchange for the air-gapped + single-source-of-truth +benefits. + +### Why row-level retention purge inside still-attached partitions + +The platform's max retention is 10 years; the longest per-tenant retention +(security events) is 7. Most rows have 2-year retention. If we relied solely on +partition drops, a 2-year-old row would still sit in a partition that won't +drop for 8 more years (because the 7y-retention rows in the same partition pin +it). + +The architecture splits responsibilities: + +- **Partition manager** ensures partitions exist for the writing window and + drops only on the platform-max horizon. +- **Retention purge** issues per-tenant per-class deletes against still-attached + partitions on its own schedule. Bulk deletes inside partitioned tables hit + only the partitions relevant to the `timestamp` range, so the pruning + remains cheap. + +### What would change our minds + +- A measured production scale where Hangfire-driven partition management can't + keep up with audit write volume (i.e. partition creation lagging behind + inbound writes). The threshold is "next month's partition does not exist on + the 1st of that month for any tenant" — if we observe even one such failure + in production, this ADR gets revisited (with `pg_partman` as the front-runner). +- An air-gapped deployment that happens to ship its own DBA-managed Postgres + extension catalogue. If `pg_partman` becomes free in the air-gap world, the + air-gapped argument weakens. +- A separate domain need for `pg_partman` (e.g. partitioning `outbox_messages` + by hour) that's painful to write a Hangfire job for. If the extension lands + for one other reason, the marginal cost of using it here drops. + +### What we explicitly punted on + +- **Sub-monthly partitioning** (daily / hourly). Phase 11 production hardening + may revisit if audit volume exceeds projections; for Phase 02a–10 monthly is + the design. +- **Cold-storage tiering** (moving partitions older than 1 year to slower + storage). Mentioned in [architecture/31 § Phasing](../architecture/31-audit-subsystem.md); + Phase 11 concern. +- **Hub-side audit aggregation.** Hub may run its own audit collation across + tenants for SaaS support; that's a Hub-repo ADR and out of scope here. + +## Consequences + +### Positive + +- One mechanism (Hangfire) owns all audit-related lifecycle: partition create, + partition drop, row-level retention purge. Operator training curve is one + job runner, not two. +- Air-gapped deployment ships the same binary as SaaS. No "make sure + `pg_partman` is at version 5.x" install-time check. +- Retention rules live in one place (Standards 18 + per-tenant `AuditConfig`). +- Failure surface is `IErrorTrackingProvider` ([ADR-0032](0032-exception-handling-logging-and-observability.md)) + by default — same observability rails as everything else in the platform. + +### Negative + +- We write and maintain partition-management code that an industry-standard + extension already provides. Roughly 200-300 lines of C# + tests; non-trivial + but bounded. +- A Hangfire job that fails silently is a worse failure mode than a `pg_partman` + + pg_cron pair, where the database itself complains. Mitigation: the + consecutive-failure event + the architecture test + `Partition_Manager_Job_Is_Registered_AtStartup` listed below. +- Adding sub-monthly partitioning later means rewriting our own scheduler logic; + with `pg_partman` it would have been a config change. + +### Neutral + +- The `audit_log` table layout and the partition-naming convention + (`audit_log_YYYY_MM`) are independent of the partition manager and would + not change if we ever revisit this ADR. +- PostgreSQL 18's native `DETACH PARTITION CONCURRENTLY` is used by either + approach; not a differentiator. + +## Implementation Notes + +- **Job class:** `AuditPartitionManagementJob` in + `LearnStack.Infrastructure.Audit/Jobs/`. Implements a single + `RunAsync(CancellationToken)` method. Registered at startup via + `RecurringJob.AddOrUpdate("learnstack:audit:partition-management", ... , + "15 2 * * *")` (daily 02:15 UTC; cron in 6-field Hangfire format). +- **Schema-aware SQL:** the job uses `Microsoft.EntityFrameworkCore`'s + `ExecuteSqlInterpolatedAsync` with a parameterized partition name; no string + concatenation. The SQL templates live next to the job class. +- **Idempotency:** every CREATE is wrapped in `IF NOT EXISTS`; every DROP is + preceded by a `SELECT pg_class.relname` existence check + a sanity-check + that the partition's bounds match what the job expects. +- **Failure observability:** failed runs hit `IErrorTrackingProvider` + (per [ADR-0032 § IErrorTrackingProvider](0032-exception-handling-logging-and-observability.md)). + After four consecutive failures the job emits a + `learnstack.audit.partition.management.failed` integration event via the + outbox; an operator dashboard subscribes. +- **Architecture test (lands in Phase 02a Packet 9):** + `Partition_Manager_Job_Is_Registered_AtStartup` — asserts the host's + `IServiceCollection` registers `AuditPartitionManagementJob` and that + Hangfire's recurring-job catalogue contains the canonical job id at startup. + Catalogued under + [21-architecture-tests-catalogue.md](../standards/21-architecture-tests-catalogue.md). +- **Tenant-context shape:** the job runs without a tenant context (it's a + platform-admin operation). `ITenantContextAccessor.Current` is null while + the job runs; `TenantContextSpanProcessor` (ADR-0032) tolerates that and + emits the span with `tenant.id = "platform"`. +- **Migration shape:** the first migration creates `audit_log` as a partitioned + parent table with two seed partitions (current and next month). Subsequent + partitions are created by the recurring job — *not* by additional migrations. + +## Amendments + +_(none yet)_ + +## References + +- [ADR-0016 Audit Log Subsystem](0016-audit-log-subsystem.md) — establishes + the `audit_log` table shape and monthly-partition policy. +- [ADR-0020 Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) + — the SelfHostedAirGapped mode that pushes against PostgreSQL extensions. +- [ADR-0031 PostgreSQL — Start on 18.x](0031-postgresql-major-version.md) — + native declarative partitioning surface this ADR builds on. +- [ADR-0032 Exception Handling, Logging, and Observability](0032-exception-handling-logging-and-observability.md) + — the `IErrorTrackingProvider` failure path; `TenantContextSpanProcessor` + null-context tolerance. +- [Standards 18 § Retention](../standards/18-audit-coverage.md) — the per-class + retention table that scopes the row-level purge job; partition drops use the + platform max only. +- [architecture/31 § 8 Retention](../architecture/31-audit-subsystem.md) — + the canonical job-name catalogue the recurring-job id slots into. +- [pg_partman](https://github.com/pgpartman/pg_partman) — rejected alternative; + link kept for future revisit. diff --git a/docs/decisions/0031-postgresql-major-version.md b/docs/decisions/0031-postgresql-major-version.md index 8c95a29..3c400e8 100644 --- a/docs/decisions/0031-postgresql-major-version.md +++ b/docs/decisions/0031-postgresql-major-version.md @@ -19,8 +19,8 @@ major-version choice only; the rest of ADR-0002 stands) versus 16 LTS at 2028-11. Starting on 18 buys an extra two years of upstream patches before any forced major upgrade. - **`gen_uuid_v7()` is native in 18.** LearnStack's - [ADR-0023 (Strongly-typed ID source generator)](README.md#open-adr-drafts) is a - pending draft considering UUIDv7 as the canonical id format + [ADR-0023 (Strongly-typed ID source generator)](0023-strongly-typed-id-source-generator.md) + adopts UUIDv7 as the canonical id format (time-ordered, index-friendly). Postgres 18 ships a built-in `gen_uuid_v7()` SQL function — DB-side DEFAULT values become trivial, the app side keeps the strongly-typed wrapping, no extension is @@ -97,7 +97,7 @@ extension we picked early diverges from the 18 native function. | 18 feature | LearnStack benefit | |------------|--------------------| -| `gen_uuid_v7()` built-in | ADR-0023 draft can pick "DB-side DEFAULT" without committing to an extension | +| `gen_uuid_v7()` built-in | ADR-0023 uses DB-side `DEFAULT gen_uuid_v7()` for high-volume append-only tables without committing to an extension | | Async I/O for sequential scans | `audit_log` partition scans (ADR-0016) — operator query latency | | OAuth authentication | Optional shortcut for Phase 11 break-glass paths (not adopted today) | | Virtual generated columns | Computed columns for `LocalizedMessage`-like derived data (Phase 02a+) | @@ -136,7 +136,7 @@ preview, that preview already runs on 18 — no major upgrade needed. - Longest support runway (EOL 2030-11) — five-year horizon before any forced major upgrade. -- Native UUIDv7 — ADR-0023 design space widens. +- Native UUIDv7 — ADR-0023 adopts it (DB-side + app-side paths). - Async I/O perf — direct benefit to `audit_log` and any future read-heavy partitioned table. - Phase 11 production deployment ships on the modern LTS without a @@ -168,8 +168,8 @@ preview, that preview already runs on 18 — no major upgrade needed. ADR-0002 Amendment 2 references this decision; doc sweep across Standards 12 / Architecture / Standards 20. - **Phase 02a** (Platform kernel): first EF migration targets Postgres - 18; if ADR-0023 picks UUIDv7, evaluate DB-side `gen_uuid_v7()` as the - default-value generator. + 18; ADR-0023 adopts UUIDv7 with DB-side `gen_uuid_v7()` as the + default-value generator for high-volume append-only tables. - **Phase 11** (production hardening): production sizing, backup cadence, replication topology — all written for 18. @@ -178,7 +178,7 @@ preview, that preview already runs on 18 — no major upgrade needed. - [ADR-0002 Initial Architecture](0002-initial-architecture.md) — original PostgreSQL major-version row, now partially superseded. - [ADR-0003 Tenant Isolation Defense in Depth](0003-tenant-isolation-defense-in-depth.md) — RLS pattern unchanged across 16/17/18. - [ADR-0016 Audit Log Subsystem](0016-audit-log-subsystem.md) — partitioned `audit_log` benefits from async I/O. -- [ADR-0023 Strongly-typed ID source generator](README.md#open-adr-drafts) — draft (no dedicated file yet — listed in the decisions index); UUIDv7 native in 18 widens the design space. +- [ADR-0023 Strongly-typed ID source generator](0023-strongly-typed-id-source-generator.md) — adopts UUIDv7; PostgreSQL 18's native `gen_uuid_v7()` powers the DB-side default path. - [Standards 05 — Database](../standards/05-database.md) - [Standards 12 § Database Operations](../standards/12-infrastructure.md) - PostgreSQL 18 release notes: . diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 2a6f068..1253b85 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -37,6 +37,9 @@ Accepted ADRs are not rewritten. A new decision is a new ADR, possibly supersedi | 0020 | [Triple Deployment + Hybrid License](0020-triple-deployment-hybrid-license.md) | SaaS / Dedicated / Self-Hosted from one codebase; phone-home + RSA-signed key + 30-day grace | | 0021 | [Feature-Based Entitlement](0021-feature-based-entitlement.md) | Feature flags + numeric limits per plan; typed `FeatureKeys` / `LimitKeys` registries | | 0022 | [Custom Domain & TLS](0022-custom-domain-tls.md) | Hub-owned custom domain admin; DNS-01 + HTTP-01 + Let's Encrypt; APISIX hot-reload | +| 0023 | [Strongly-Typed ID Source Generator — Vogen](0023-strongly-typed-id-source-generator.md) | Vogen as the source generator for both IDs and value objects; `[ValueObject]` annotation; EF + JSON + ASP.NET + OpenAPI emitters out of the box | +| 0024 | [API Versioning Policy](0024-api-versioning-policy.md) | URL-based `/v{N}/`; 6-month deprecation window; RFC 8594 `Sunset` + `Deprecation` headers; OpenAPI `deprecated` + `x-sunset` extensions; 410 Gone with RFC 7807 on sunset | +| 0028 | [`audit_log` Partition Management — Hangfire Recurring Job](0028-audit-log-partition-management.md) | Daily `learnstack:audit:partition-management` Hangfire job; create-ahead 2 months; drop only on platform-max retention horizon; row-level purge separate; no `pg_partman` dependency | | 0029 | [Object Storage — SeaweedFS](0029-object-storage-seaweedfs.md) | Self-hosted SeaweedFS behind the existing `IStorageProvider` S3 contract; partially supersedes ADR-0002's MinIO row | | 0030 | [Redis-compatible Store — Valkey](0030-redis-compatible-store-valkey.md) | Valkey (Linux Foundation, BSD-3-Clause) for the cache + Dapr state-store backend; RESP-protocol drop-in; partially supersedes ADR-0002's Redis row | | 0031 | [PostgreSQL — Start on 18.x](0031-postgresql-major-version.md) | Pin primary RDBMS major version to PostgreSQL 18; native `gen_uuid_v7()` + async I/O + longest LTS runway; partially supersedes ADR-0002's PostgreSQL row | @@ -70,12 +73,9 @@ table records the phase commitment so reviewers can flag late drafts. | Reserved # | Topic | Target phase (must be Accepted before) | Referenced from | |---|---|---|---| -| 0023 | Strongly-typed ID source generator (Vogen vs StronglyTypedId vs custom; emitter spec) | **Phase 02a** — interceptors and value converters need the generator at compile time | [02-backend-coding.md](../standards/02-backend-coding.md) | -| 0024 | API versioning policy (URL prefix `/v1/` stays the convention; this ADR codifies deprecation cadence, sunset headers, and the rule for breaking changes) | **Phase 02a** — OpenAPI spec + SDK generation start here | [04-technical-architecture.md § API Strategy](../architecture/04-technical-architecture.md), [04-api-design.md § Versioning](../standards/04-api-design.md) | | 0025 | Scoring + completion DSL sandbox engine (CEL vs restricted Lua vs custom; sandbox boundary; allowed function set) | **Phase 05** — `TenantCompletionRule` runtime evaluator lights up here; Phase 08a's assessment scoring depends on it | [ADR-0018](0018-tenant-driven-customization-model.md), [phase-05-education-learning-content.md](../roadmap/phase-05-education-learning-content.md), [phase-08a-assessment-notifications.md](../roadmap/phase-08a-assessment-notifications.md) | | 0026 | Release-tag scheme (`vYYYY.MM.DD.` vs SemVer; SaaS continuous-deploy reconciliation; Self-Hosted release cadence) | **Phase 11** — production hardening checklist owns this | [14-git-workflow.md § Tagging and Releases](../standards/14-git-workflow.md) | | 0027 | Frontend i18n library pick (`next-intl` vs `react-intl` vs `lingui`) | **Phase 04** — the first CMS / page-builder surface ships locale-aware copy | [12-localization.md](../architecture/12-localization.md), [08-localization.md](../standards/08-localization.md) | -| 0028 | `audit_log` monthly partition management (Hangfire job vs `pg_partman` extension; failure-mode comparison) | **Phase 02a** — partition policy is Day 1; the choice can be retrofitted later but must be ADR'd before production load | [ADR-0016](0016-audit-log-subsystem.md), [31-audit-subsystem.md](../architecture/31-audit-subsystem.md) | **Reservation rule:** the numbers above are *reserved but not yet drafted*. When a draft lands, take its reserved number; do not let another ADR claim it. If the diff --git a/docs/roadmap/phase-02a-kernel-tenancy.md b/docs/roadmap/phase-02a-kernel-tenancy.md index 043ddee..89dee84 100644 --- a/docs/roadmap/phase-02a-kernel-tenancy.md +++ b/docs/roadmap/phase-02a-kernel-tenancy.md @@ -16,14 +16,18 @@ > place. No code, no ADR state changes — Packet 0 is a planning slice > that unblocks the rest of the phase by fixing the order. > -> **Packet 1 — Foundation decisions ⏳** -> Move the three Phase-02a-blocking drafts from -> [decisions/README.md § Open ADR Drafts](../decisions/README.md) to -> **Accepted**: ADR-0023 (strongly-typed ID source generator — -> Vogen / StronglyTypedId / custom), ADR-0024 (API versioning policy — -> `/v1/` URL convention, deprecation cadence, sunset headers), ADR-0028 -> (`audit_log` monthly partition management — Hangfire job vs `pg_partman`). -> Decision-only; no code. Unblocks Packets 2+. +> **Packet 1 — Foundation decisions ✅** +> The three Phase-02a-blocking ADRs are now Accepted: +> [ADR-0023](../decisions/0023-strongly-typed-id-source-generator.md) +> picks **Vogen** as the source generator for both IDs and value objects; +> [ADR-0024](../decisions/0024-api-versioning-policy.md) codifies +> URL-based versioning with a **6-month deprecation window** + RFC 8594 +> `Sunset` / `Deprecation` headers + OpenAPI `x-sunset` extensions; +> [ADR-0028](../decisions/0028-audit-log-partition-management.md) picks +> a **Hangfire recurring job** (`learnstack:audit:partition-management`) +> over `pg_partman` to keep the SelfHostedAirGapped story extension-free. +> Decision-only; no code. Standards 02 § Strongly-Typed Identifiers and +> Standards 04 § Versioning cross-link to the new ADRs. > > **Packet 2 — Shared Kernel core ⏳** > `IClock`, `IRandom`, `IGuidFactory` (deterministic-test abstractions per @@ -590,15 +594,15 @@ already in place from 02a. ### ADR commitments that must land in this phase -Per [decisions/README.md § Open ADR Drafts](../decisions/README.md), three drafts -target Phase 02a and **must be Accepted before this phase exits**: +Three ADRs targeted Phase 02a as exit blockers; all three are now Accepted +(Packet 1): -| Reserved # | Topic | Why it blocks Phase 02a | -|---|---|---| -| ADR-0023 | Strongly-typed ID source generator (Vogen vs StronglyTypedId vs custom) | EF interceptors and value converters depend on the emitter at compile time | -| ADR-0024 | API versioning policy (deprecation cadence, sunset headers, `/v1/` convention) | OpenAPI spec + SDK generation start in this phase | -| ADR-0028 | `audit_log` monthly partition management (Hangfire job vs `pg_partman`) | Partition policy ships Day 1 with the audit infrastructure | +| # | Topic | Status | Decision | +|---|---|---|---| +| [ADR-0023](../decisions/0023-strongly-typed-id-source-generator.md) | Strongly-typed ID source generator | **Accepted** (2026-05-20) | Vogen as the emitter for both IDs and value objects | +| [ADR-0024](../decisions/0024-api-versioning-policy.md) | API versioning policy | **Accepted** (2026-05-20) | URL `/v{N}/`, 6-month deprecation window, RFC 8594 `Sunset` + `Deprecation` headers, OpenAPI `x-sunset` extensions | +| [ADR-0028](../decisions/0028-audit-log-partition-management.md) | `audit_log` monthly partition management | **Accepted** (2026-05-20) | Daily Hangfire recurring job (`learnstack:audit:partition-management`); no `pg_partman` runtime dependency | -Each PR that lands one of these ADRs flips the corresponding "exit checklist" -item; the phase is not "done" until all three are Accepted and CI exercises the -chosen implementation. +The remaining exit gates (tenant + organization resolution, isolation tests, +audit pipeline, customization runtime read paths, API conventions, architecture- +test catalogue green) close as Packets 2–10 ship. diff --git a/docs/standards/02-backend-coding.md b/docs/standards/02-backend-coding.md index 1e90361..e149ab6 100644 --- a/docs/standards/02-backend-coding.md +++ b/docs/standards/02-backend-coding.md @@ -1,7 +1,7 @@ # 02 — Backend Coding Standards **Status:** Active -**Derives from:** [ADR 0002 — Initial Architecture](../decisions/0002-initial-architecture.md), [ADR 0006 — Events and Outbox](../decisions/0006-events-and-outbox.md). +**Derives from:** [ADR 0002 — Initial Architecture](../decisions/0002-initial-architecture.md), [ADR 0006 — Events and Outbox](../decisions/0006-events-and-outbox.md), [ADR 0023 — Strongly-Typed ID Source Generator](../decisions/0023-strongly-typed-id-source-generator.md). C# / .NET conventions for LearnStack backend code. @@ -49,11 +49,18 @@ public readonly record struct CourseId(Guid Value) : IStronglyTypedId } ``` -A shared source generator (or analyzer pack) emits: +Per [ADR-0023](../decisions/0023-strongly-typed-id-source-generator.md), the +shared source generator is **[Vogen](https://github.com/SteveDunn/Vogen)**. The +canonical declaration uses Vogen's `[ValueObject(...)]` annotation on a +partial `record struct`; Vogen emits: - EF Core value converter. - `JsonConverter`. - Minimal API model binder. -- OpenAPI schema mapping. +- OpenAPI schema mapping (Swashbuckle / Microsoft.OpenApi schema filter). + +The same annotation covers richer value objects (`Email`, `Slug`, `LocaleCode`, +`Money`) — the emitter shape is identical for IDs and value objects, with the +value-object's invariant captured in a `Validate` static method. ## Nullability diff --git a/docs/standards/04-api-design.md b/docs/standards/04-api-design.md index 435e501..a0bafee 100644 --- a/docs/standards/04-api-design.md +++ b/docs/standards/04-api-design.md @@ -1,7 +1,7 @@ # 04 — API Design Standards **Status:** Active -**Derives from:** [ADR 0002 — Initial Architecture](../decisions/0002-initial-architecture.md), [ADR 0003 — Tenant Isolation Defense in Depth](../decisions/0003-tenant-isolation-defense-in-depth.md). +**Derives from:** [ADR 0002 — Initial Architecture](../decisions/0002-initial-architecture.md), [ADR 0003 — Tenant Isolation Defense in Depth](../decisions/0003-tenant-isolation-defense-in-depth.md), [ADR 0024 — API Versioning Policy](../decisions/0024-api-versioning-policy.md). REST conventions for LearnStack public and admin APIs. @@ -11,31 +11,44 @@ REST conventions for LearnStack public and admin APIs. - Resources are plural nouns (`/courses`, `/users`). - HTTP verbs used idiomatically: `GET` read, `POST` create, `PUT` replace, `PATCH` modify, `DELETE` remove. - JSON, `application/json`, UTF-8. -- URL versioned: `/v1/...`. +- URL versioned: `/api/v1/...` (canonical prefix per [ADR-0024](../decisions/0024-api-versioning-policy.md)). - Bodies use `camelCase`. ## URL Structure ``` -/{version}/{resource}/{id?}/{sub-resource?} +/api/{version}/{resource}/{id?}/{sub-resource?} ``` Examples: -- `GET /v1/courses?cursor=...&limit=20` -- `POST /v1/courses` -- `GET /v1/courses/{id}` -- `PATCH /v1/courses/{id}` -- `POST /v1/courses/{id}/versions` +- `GET /api/v1/courses?cursor=...&limit=20` +- `POST /api/v1/courses` +- `GET /api/v1/courses/{id}` +- `PATCH /api/v1/courses/{id}` +- `POST /api/v1/courses/{id}/versions` -Platform-admin endpoints live under `/v1/platform/...` and require platform-admin scope. +Platform-admin endpoints live under `/api/v1/platform/...` and require platform-admin scope. ## Versioning -- URL-based: `/v1`, `/v2`. Header-based versioning is not used. -- **Non-breaking changes** stay in the same version: additive fields, new optional query params, new endpoints. -- **Breaking changes** require a new major version: removed/renamed fields, behavior changes, removed endpoints. -- Deprecated fields stay one minor release minimum with `Deprecation` header and a `Sunset` date. -- Two adjacent versions coexist; EOL is announced. +The full versioning policy lives in +[ADR-0024 — API Versioning Policy](../decisions/0024-api-versioning-policy.md). +Summary so reviewers don't have to chase a link to know the shape: + +- **URL-based** under `/api/v{N}/` (the only canonical public route shape; + matches APISIX gateway routes per ADR-0015). Header-based versioning is + not used. +- **Two adjacent majors coexist**; deprecation window is **6 months**. +- **`/healthz` and `/readyz` are unversioned** infrastructure endpoints. +- The internal `/api/internal/*` Hub contract has its own versioning per + [ADR-0019](../decisions/0019-learnstack-hub.md) and is not governed by + ADR-0024. + +For the breaking/non-breaking matrix, the exact header set +(`Deprecation` per RFC 9745, `Sunset` per RFC 8594, `Link: +rel="successor-version"`), OpenAPI extensions (`x-sunset` / `x-successor` +/ `x-migration-guide` / `x-extensible-enum`), the 410 Gone Problem Details +shape, and per-deployment-mode behaviour — see ADR-0024 directly. ## Status Codes @@ -68,7 +81,7 @@ All errors use **Problem Details (RFC 7807)**. "status": 400, "code": "validation_failed", "detail": "One or more fields are invalid.", - "instance": "/v1/courses", + "instance": "/api/v1/courses", "correlationId": "01H7F...", "errors": { "title": ["Title is required."], @@ -97,7 +110,7 @@ The tenant is **never** read from a request body or query param at the API edge. 1. Host header → registered domain → tenant id. 2. `tenant_id` claim in the JWT (studio / platform-admin contexts). -3. Explicit override in `/v1/platform/...` endpoints with proper scope. +3. Explicit override in `/api/v1/platform/...` endpoints with proper scope. A request that cannot resolve a tenant returns **404** (not 403, to avoid disclosure). @@ -106,7 +119,7 @@ A request that cannot resolve a tenant returns **404** (not 403, to avoid disclo Cursor pagination by default: ``` -GET /v1/courses?cursor=eyJpZCI6Li4ufQ&limit=20 +GET /api/v1/courses?cursor=eyJpZCI6Li4ufQ&limit=20 ``` Response: @@ -141,7 +154,7 @@ Rules: `POST` operations with external side effects accept an `Idempotency-Key` header: ``` -POST /v1/orders +POST /api/v1/orders Idempotency-Key: 01HX7F... ``` @@ -156,8 +169,8 @@ Rules: Mutable resources expose `ETag` (or `version` field). ``` -GET /v1/courses/{id} → ETag: "7" -PATCH /v1/courses/{id} +GET /api/v1/courses/{id} → ETag: "7" +PATCH /api/v1/courses/{id} If-Match: "7" ``` @@ -187,7 +200,7 @@ Version mismatch returns **409** with `concurrency_conflict`. ## Webhooks (Inbound) -- Endpoint: `/v1/webhooks/{provider}`. +- Endpoint: `/api/v1/webhooks/{provider}`. - Verifies provider signature with a tenant-scoped or platform-scoped secret. - Idempotent: `(provider, event_id)` stored; duplicates ignored. - Returns **200** quickly; heavy work deferred to a job.