Repository navigation
Description
Goal
Ending an account ends its compute, and ACCESS learns the end user of every job.
After this phase, revoking or deleting an account records the cancellation of every live job and frees unsubmitted holds in the same database batch as the other credential cascades, a retried sweep cancels at Tapis, and NEMAR reports each Slurm job's pseudonymous gateway user to ACCESS exactly once.
Routing
- Readiness: ready
- Implementer: Sonnet
- Opus checkpoint: review before merge. Look at: cascade completeness across both caller sites, the caller-count sentences, that no Tapis or ACCESS call happens inside a batch, and that a report is never sent twice.
- Depends on: Phase A5: Job lifecycle, webhooks, settlement and results #2010 ; Unblocks: Phase A7: Tenant reconcile, observer and reporting #2012
- Repository and base branch: nemarOrg/nemar-cli, epic branch
feature/issue-#2005-epic-compute-core
Scope
In: migration 0119_compute_access_reports.sql, computeReleaseStatements, its two caller sites, the ACCESS reporter and its cron.
Out: suspension of compute and its grant changes (Epic D), OIDC grant revocation (Epic B), the tenant-wide sweep (#2012).
Design references
Design sections 7 (cancellation and revocation) and 9 (attribution to ACCESS). AGENTS.md "Revocation cascades, always". Templates: mailReleaseStatements in services/ticket-assignment.ts; the 30-minute tick in scheduled().
Changes
- Revocation:
computeReleaseStatements(userId)inservices/compute-release.ts(templatemailReleaseStatements): for jobs not yet at Tapis, cancel and release the hold in SQL; for jobs at Tapis, setcancel_reason = 'account_ended'andcancel_requested_at; setfrozen = 1on the account's pools. Every statement must end in a WHERE clause, becausetombstoneUserCascadeappends a guard to each. Add it to the batch offinalizeRevocation(routes/admin/users.ts) and totombstoneUserCascade(db/user-tombstone.ts), which the owner soft delete and the test-account sweep both run, so there are two caller sites, not three. Role demotion and key revocation deliberately do not end compute jobs; say why in the docblock. Update the caller-count sentences inservices/docs-auth.tsandticket-assignment.tsand extendbackend/test/admin-credential-cascade.test.ts. - Migration
0119_compute_access_reports.sql:compute_access_reports(job_id PRIMARY KEY, state IN ('pending','sent','failed'), attempts, next_attempt_at, last_error_code fixed word NULL, sent_at NULL, created_at). Fill Phase A5: Job lifecycle, webhooks, settlement and results #2010's hook: the batch in which a job's record first shows a Slurm id inserts apendingrow (INSERT OR IGNORE, so a replayed read never adds a second). services/access-attributes.ts: POST{ jobid, submittime, xsederesourcename, gatewayuser }to<base>/job_attributes(base fromglobalThis.NEMAR_ACCESS_ATTRIBUTES_URL, else the ACCESS address) withdebugset outside production; key from the secretACCESS_GATEWAY_ATTRIBUTES_KEY, resource name fromACCESS_RESOURCE_NAME, both optional (a row stayspendingwithunconfigured); exponential retry,failedafter a bound. The key header name and body spelling are unverified: isolate them in two constants and say so. The gateway user is the job row'sgateway_user(Phase A3: Job records and the results and records buckets #2008), andnemar-pipelinesfor pipeline jobs.runAccessReportCron(env)sends due rows; wire it into the 30-minute tick's production-only block like Phase A5: Job lifecycle, webhooks, settlement and results #2010's crons, self-fenced withisExactProduction, with atick-prod-onlyentry incron-sweep-wiring.test.ts.
Tests (write these first)
- Cascade: revoking and tombstoning an account mid-run records
account_ended, frees unsubmitted holds, freezes pools, and the retried sweep cancels at Tapis; mutation: remove the statement from either caller. - ACCESS reporter against a stand-in: body fields,
debugoutside production, retry thenfailed, never sent twice for one job. - A replayed status read after the Slurm id first appears inserts no second report row. Mutation: drop
OR IGNORE.
Acceptance criteria
- Revocation and deletion record cancellation of every live job in the same batch as the other cascades, through both caller sites.
- No Tapis or ACCESS call runs inside a database batch.
- Every Slurm job is reported once, with the pseudonymous gateway user, and never with a name or email.
- Cascade and wiring tests are updated and pass.
Gates
bun test backend/test/admin-credential-cascade.test.ts backend/test/compute-lifecycle.test.ts backend/test/cron-sweep-wiring.test.ts backend/test/cron-dev-safety.test.tsbun run migrations:d1-check,bun run typecheck,bunx biome check src shared backend/src,bun test
Rules that bite here
- Revocation cascades, always: the batch, the counts and a retried sweep for the network call, never a Tapis call inside the batch.
- Every release statement ends in a WHERE clause, because
tombstoneUserCascadeappends a guard to each. - The ACCESS key is a Worker secret the owner sets; never put it on a command line or in a job.
- New crons are production-only.
Escalate to the owner if
- Tapis cannot cancel a job the credential did not submit.
- The ACCESS interface refuses the pseudonymous gateway user, or needs fields the design does not list.
When merged
Update the revocation paragraph of design section 7 and the attribution paragraph of section 9 as shipped, and list the reporter cron in the systems inventory runbook.
Activity
neuromechanist commentedon Nov 5, 2026
Merged into the epic branch by #2046 (squash cf34441). Round 1 (Opus checkpoint), round 2 (Sonnet delta) and a rebase after the A2 follow-up converged. Revocation and deletion record the cancellation of every live NEMAR job in the same batch as the other cascades (both caller sites), partner reservations are left to the partner contract, the sweep cancels at Tapis, and every Slurm job is reported to ACCESS at most once with the pseudonymous gateway user (a report may be lost, never doubled). Owner items: the secret is ACCESS_GATEWAY_ATTRIBUTES_KEY plus ACCESS_RESOURCE_NAME (both owner-set Worker secrets, pinned absent from the toml); the ACCESS interface constants (header name, body spelling, time format, debug field, duplicate behaviour) are UNVERIFIED and must be confirmed when the key is issued. Carried to other phases: A7 (list failed report rows, owner re-queue with acknowledgement, the A5 and A8 open points), Epic D (freeze cause, pseudonym for account-less partner jobs, the partner-reservation items).