Repository navigation
Description
Goal
NEMAR can say how many SU a holder may spend and refuse a spend that would exceed it, atomically.
After this phase there is a versioned owner policy, per-holder pools (accounts, the platform's two budgets, partner clients), an append-only ledger whose totals can never disagree with it, a service that holds, releases, settles, adjusts and refunds, and owner and admin routes and CLI commands to set policy and grant SU.
Routing
- Readiness: ready
- Implementer: Sonnet
- Opus checkpoint: review before merge. Look at: the hold statement's atomicity under concurrency, that no path can write pool totals except through a ledger row, overrun and negative-available handling, integer-only arithmetic, the policy write path (owner only, audited, validated), and that a refused hold writes nothing.
- Depends on: Phase A1: Compute contract and ADR 0118 #2006 ; Unblocks: Phase A3: Job records and the results and records buckets #2008, Phase A4: Tapis client and stand-in #2009, Phase A5: Job lifecycle, webhooks, settlement and results #2010
- Repository and base branch: nemarOrg/nemar-cli, epic branch
feature/issue-#2005-epic-compute-core
Scope
In: migration 0114_compute_ledger.sql; backend/src/services/compute-ledger.ts and compute-policy.ts; admin routes; nemar admin compute policy|pool.
Out: the job table and the job-count guards on a hold (#2008), settle callers (#2010, #2012), automatic renewal and approval-time default grants (Epic D; the policy stores renewal and defaultGrantMilli and nothing here reads them), mail.
Design references
Design section 5 (pools, ledger, life of a charge), ADR 0118, ADR 0034 (no new datasets columns), DISPATCH_OPEN_CLAIM_SQL in backend/src/services/upstream-repull-dispatch.ts (one INSERT-SELECT with the check in its WHERE, meta.changes as the winner test), 0112_upstream_repull_automatic.sql (migration style).
Changes
- Migration
backend/src/db/migrations/0114_compute_ledger.sql, header comment per column, CHECKs that are TRUE or FALSE, no foreign keys, GLOBs of 50 characters or fewer:compute_policy(version INTEGER PRIMARY KEY AUTOINCREMENT, policy_json TEXT NOT NULL CHECK json_valid, set_by INTEGER NOT NULL, note TEXT, created_at), append-only by BEFORE UPDATE and DELETE triggers raising ABORT; the current policy is the highest version, and no row meansDEFAULT_COMPUTE_POLICY.compute_pools(id, holder_kind IN ('account','platform','partner'), holder_ref TEXT, pool IN ('cpu','gpu'), granted_milli, held_milli, charged_milli, frozen, created_at, updated_at, UNIQUE(holder_kind, holder_ref, pool))with each total at least 0; the migration seeds the four platform rows (pipelinesandpeople, each cpu and gpu).compute_ledger(id, kind IN the eight, pool_id, budget_pool_id NULL, granted_delta_milli, held_delta_milli, charged_delta_milli, job_id TEXT NULL, hold_ref INTEGER NULL, reason TEXT NULL fixed word, actor_user_id NULL, idempotency_key TEXT NOT NULL UNIQUE, created_at)append-only by triggers, with CHECKs fixing which deltas each kind may carry (grant: granted+; hold: held+; release: held-; charge: held at most 0, charged at least 0; refund: charged-; adjust: exactly one of granted or charged nonzero; overrun: charged+; expire: granted-), a partial unique index onhold_reffor kinds release and charge (a hold closes once), andbudget_pool_idallowed only for kinds that spend. Decision: an AFTER INSERT trigger oncompute_ledgeradds the three deltas topool_id's row and the held and charged deltas tobudget_pool_id's row. The ledger is therefore the only writer of totals, "only if the insert happened" holds by construction, andchanges()is not needed. (If D1 refuses the trigger, stop and escalate; do not fall back to application-side updates.) compute-ledger.ts(export every SQL as*_SQL):HOLD_INSERT_SQL,holdStatements(db, { poolId, budgetPoolId, amountMilli, jobId, idempotencyKey, extraGuards })returning the one INSERT whose WHERE checks the pool not frozen, pool available at least the amount, budget pool available at least the amount, the key unused, and any caller guard;placeHold(runs it, reads which guard failed on zero changes and returnsinsufficient_balance,suspendedorplatform_budget_exhausted; a replayed key returns the original);releaseHold;settleProvisional(db, { jobId, chargeMilli })(onechargeentry closing the hold, charge capped at the hold, anoverrunentry for any excess in the same batch);settleFinal(anadjustof the charged delta to the final figure, once per job);refundJob;grantPool;adjustPool;expireUnused;availableMilli(row)(granted minus held minus charged, may be negative);ensurePool(holderKind, holderRef, pool)(an idempotent insert; Phase A7: Tenant reconcile, observer and reporting #2012 uses it for thetenantpartner pool).compute-policy.ts:getPolicy(db)(validated, default on no row) andputPolicy(db, ownerId, input, note), which validates with the strict zod schema and inserts the version and an audit row in one batch.backend/src/routes/admin/compute.tsexportingregisterComputeAdminRoutes(admin), registered inroutes/admin/index.ts:GETandPUT /compute/policy(owner, auditcompute.policy.set);GET /compute/pools,GET /compute/pools/by-id/:accountId(admin);POST /compute/pools/by-id/:accountId/grantand/adjust(admin,Idempotency-Keyheader required, reason is a fixed word, auditcompute.pool.grantandcompute.pool.adjust);POST /compute/pools/platform/:key/grantand/adjust(owner). Errors are{ error, code }; an invalid body names fields and never echoes values.- CLI:
src/lib/api/compute.ts,src/commands/admin-compute.ts(computeCommandwithpolicy show|set --file <json> [--note]andpool show|grant|adjust, amounts in SU throughparseSuToMilli, one fresh Idempotency-Key per invocation), registered insrc/commands/admin.tsbesidemailCommand; add the module toMODULE_EXPORTSandPOST_SPLIT_ADDITIONSintest/api-export-surface.unit.test.ts; pin the routes intest/admin-route-inventory.unit.test.ts.
Tests (write these first)
backend/test/compute-ledger-migration.test.tson bun:sqlite and Miniflare D1 (template:upstream-repull-dispatches-migration.test.ts): every CHECK refuses its bad row; update and delete of a ledger or policy row abort; runbun run migrations:d1-check.backend/test/compute-ledger-race.test.ts: on Miniflare D1, two concurrentplaceHoldfor the last SU of one pool; exactly one wins, the other writes no ledger row and returnsinsufficient_balance. Mutation: delete the availability term from the WHERE; the test must fail.- Same file: platform budget exhausted while the person has balance returns
platform_budget_exhausted; a frozen pool returnssuspended; a replayed idempotency key returns the first hold and moves no total. backend/test/compute-ledger.test.ts: after a seeded random sequence of every operation, each pool's three totals equal the sums of its ledger deltas (the trigger invariant); settle then final adjust ends at the final figure; excess use writes anoverrunand negative available; a hold cannot be settled twice. Mutation: drop one delta from the trigger.backend/test/compute-policy-routes.test.tsandcompute-pool-routes.test.tsthroughapp.fetch: a member and an admin are refused on policy; ownerPUTof an invalid body is 400 without echoing values; each write leaves an audit row; a repeated grant key grants once.test/admin-compute-cli.test.ts(templatetest/admin-repull-auto-cli.test.ts):policy setthenshowround trips;pool grant 12.5moves 12500; a member's key exits 1.
Acceptance criteria
- Totals equal ledger sums after the random-sequence test; the race test proves one winner.
- No code path updates
compute_poolstotals directly (grep and a test). -
bun run migrations:d1-checkpasses sequentially. - Routes and CLI module are pinned in the inventory and export-surface tests.
Gates
bun test backend/test/compute-ledger-migration.test.ts backend/test/compute-ledger-race.test.ts backend/test/compute-ledger.test.ts backend/test/compute-policy-routes.test.ts backend/test/compute-pool-routes.test.ts test/admin-compute-cli.test.ts test/admin-route-inventory.unit.test.ts test/api-export-surface.unit.test.tsbun run migrations:d1-checkbun run typecheckbunx biome check src shared backend/srcbun test
Rules that bite here
- A refused hold writes nothing; the rate-limit counters are never a quota.
- Integer thousandths; an
overrunmay make available negative but never a stored total. - The ledger is append-only and jobs or entries never cascade from the account row.
compute_policyis owner-only; D1 rejects SQL bun:sqlite accepts (GLOB length, compound SELECT terms).
Escalate to the owner if
- D1 will not run the triggers.
- Admin-only pool grants look too permissive once the platform budget is considered; the owner may want owner-only grants.
When merged
Update design section 5 with the table and entry names and the trigger decision, and note the new owner commands in the runbook list.
Activity
neuromechanist commentedon Nov 4, 2026
Driver amendment, 2026-11-04, from the phase A1 review (PR #2030) and the Opus advisor. This comment is part of the phase brief; where it differs from the body above, this comment wins.
GET /compute/policywith a passthrough schema (unknown keys accepted at the top level and nested);ComputePolicySchema(strict) is only for checkingpolicy setinput and forputPolicy. Add a test wherepolicy showprints a response that carries unknown keys at the top level and nested.getPolicystays strict on stored rows and fails closed (503) on a row it cannot parse. It never falls back toDEFAULT_COMPUTE_POLICY.neuromechanist commentedon Nov 4, 2026
Merged into the epic branch by #2032 (squash
941ba788), 2026-11-04, after a three-reviewer panel, the Opus checkpoint and its delta; every accepted finding fixed (triage on the pull request). Gates: 973 targeted tests across the ledger, contract and CLI suites, typecheck and the D1 migration check pass (178 objects); the full CI run is the epic gate.What later phases get, as built:
compute_policy,compute_poolsandcompute_ledger(migration 0114) with thirteen triggers so the table itself refuses what the hold statement refuses,compute-ledger.ts(hold, release, settle, final, refund, grant, adjust, expire; the hold key is derived from the job; a replay of a closed hold is refusedhold_closedwithclosedBy; a final after a refund is refusedrefunded; the final settlement always records its key),compute-policy.ts, the admin routes andnemar admin compute policy|pool.Open for the repository owner, needed before phase A5: whether a refund or a charge correction credits the
peoplebudget back (what the code does today, with account charge corrections owner-only as an interim rule) or only moves the person's pool. The schema does not decide it. Still owed from this phase's "When merged" step: the design page (section 5) and the runbook list of owner commands, by a docs pull request.neuromechanist commentedon Nov 5, 2026
A2 follow-up merged into the epic branch by #2044: the people budget now records consumption only (the owner's decision of 2026-11-04, option A). The refund path is deleted; a charge correction that lowers a pool's charge can never name the budget (the service refuses, the route names the budget only on a raise, and the statement itself carries the refusal); only a job's final settlement and the owner's own platform correction can lower the budget. Optional hardening left for the owner: a table trigger refusing a budget-naming lowering and kind refund (migration 0114 is unreleased).