Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 132 additions & 0 deletions .context/decisions/0030-compute-pages-follow-adr-0010-phase-3.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions .context/decisions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,4 +61,5 @@ Add new entries here as you create ADRs:
- ADR 0027 - [The admin approval page prints the API's review words and computes no verdict](0027-the-admin-approval-page-prints-the-apis-review-words.md) (accepted)
- ADR 0028 - [The BIDS validator runs in the browser from a vendored bundle, in a worker](0028-bids-validator-vendored-bundle-in-a-worker.md) (accepted)
- ADR 0029 - [The admin Mail tab renders email as inert text and sends every reply under one key per draft](0029-admin-mail-tab-untrusted-text-and-reply-keys.md) (accepted)
- ADR 0030 - [Compute pages are ADR 0010's Phase 3, behind a grant of their own, and decide nothing](0030-compute-pages-follow-adr-0010-phase-3.md) (accepted)
- ADR 0031 - [The cross-site form check is the site's own, with one exemption: the authorization-request POST](0031-cross-site-form-check-is-the-sites-own-with-one-exemption.md) (accepted)
32 changes: 32 additions & 0 deletions .context/handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,38 @@

**Last session:** 2026-11-07.

## 2026-11-07: The compute dashboard (website#495, epic website#493 phase E2)

Branch `feature/issue-495-compute-dashboard`, PR into the epic branch. ADR 0030 addendum. Depends on nemar-cli's Epic D branch (phase D2 routes) and its copy PR #2082 (seven keys, reworded `compute_access.granted.body`).

- **Recipe:** `NEMAR_CLI_DIR=<Epic D checkout> PORT=8791 bun scripts/local-backend.mjs --seed-compute`, then `PUBLIC_API_BASE_URL=http://localhost:8791 bun run dev`; cookies for `rich`, `frozen`, `overrun`, `granted` print with the others (`.context/local-backend.md`).
- **Fixtures** (`test/fixtures/compute/*.json`, 13 new files) are captured by `scripts/capture-compute-read-fixtures.mjs` from the real router; a rerun writes the same bytes.
- **Drift gates:** `test/compute-user-contract-drift.test.ts` (`NEMAR_CLI_COMPUTE_STATE`, `_USER_CONTRACT`, `_CONTRACT`) and the account-copy drift test are red in CI until the compute epic and #2082 reach nemar-cli `dev`. Never put them on `staging` earlier.
- **Release gate:** do not promote until the production API serves `GET /compute/me`; a site talking to an API without it shows one "not available" notice.
- **Follow-ups:** a usage window selector, a `?state=` filter, the ADR 0030 hand-off (drop `compute-terms.ts` and `COMPUTE_REQUESTS_ENABLED`), the job page (E3). Backend gaps: nemar-cli#2078 (pool as a usage group), #2079 (job name), #2080 (are new jobs accepted).

## 2026-11-07: Admin Compute tab (website#497, epic website#493 phase E4)

Branch `feature/issue-497-admin-compute-tab`, PR into the epic branch (not `staging`). ADR 0030. Page `/admin/compute`, tab after Mail.

- **Served:** Requests with grant, deny, suspend, revoke by id (Grant needs an unticked-by-default export-review acknowledgement, Revoke a typed word, nothing is focused but Cancel); read-only Pools, Usage (the quarterly counts), Flagged jobs (one flag per read) and Reconciliation (status, unmatched accounting rows, mismatch jobs).
- **Not served, by decision:** policy, move-charge, pool grants and adjustments, job cancel and advance (the lede points to the CLI). **Not served by the backend:** platform SU by day (nemar-cli#2083), the ledger-versus-scheduler difference and its tolerance (#2084); also asked for: a count and an order option (#2085), last decision and standing on the access row (#2086), single-account read (#2088). Copy keys: #2087.
- **Copy:** 156 `compute_admin.*` keys added to nemar-cli `shared/contract/account-copy.ts` (small PR into the Epic D branch) and transcribed here, with new placeholder names the nemar-cli parity test declares. The parity pair is red in nemar-cli CI until the website keys reach `dev`.
- **Gates:** `test/compute-admin-contract-drift.test.ts` is strict in CI (`NEMAR_CLI_COMPUTE_ADMIN_CONTRACT`) and red until the compute epic is on nemar-cli `dev`; locally pass `NEMAR_CLI_COMPUTE_ADMIN_CONTRACT=<checkout>/shared/contract/compute-admin.ts`.
- **Fixtures:** `NEMAR_CLI_DIR=<Epic D checkout> bun scripts/capture-compute-admin-fixtures.mjs` (needs `bun install` at the checkout root and in `backend/`). Local browser world: `.context/local-backend.md`.
- **Shared with E2:** `formatSu` (shortest exact decimal, used for every admin figure), `jobBadge`, `readComputeError` and the job state and source lists come from E2's `compute-format.ts`, `compute-badges.ts`, `compute-api.ts` and `compute-contract.ts`; the admin contract file keeps only admin-specific lists.
- **The badge** is the length of the first page of the `requested` listing (`50+` when more), one more API call per admin render, 2 s deadline, fail-soft.

## 2026-11-06: Compute access card in Settings (website#494, epic website#493 phase E1)

Branch `feature/issue-494-compute-access-card`, PR into the epic branch `feature/issue-493-epic-compute-web` (not `staging`). ADR 0030.

- **Recipe for the next phase:** run the real backend with `NEMAR_CLI_DIR=<nemar-cli Epic D checkout> bun scripts/local-backend.mjs`, then `PUBLIC_API_BASE_URL=http://localhost:8787 bun run dev`; `.context/local-backend.md` has the cookie steps. Fixtures are captured, not written: `scripts/capture-compute-fixtures.mjs` writes `test/fixtures/compute/`.
- **Where the keys come from:** nemar-cli commit `3a869903` (phase D1, squash on `feature/issue-2021-epic-compute-users`): 27 copy keys, the `compute_access` gap block, the `affiliation` row, `COMPUTE_TERMS_VERSION`.
- **Gates:** drift against that checkout passes locally (`NEMAR_CLI_ACCOUNT_COPY=<checkout>/shared/contract/account-copy.ts`, `NEMAR_CLI_COMPUTE_ACCESS_CONTRACT=<checkout>/shared/contract/compute-terms.ts`). In CI both are red until the compute epic is on nemar-cli `dev`; the CI ref stays `dev`. Never put these keys on `staging` before then.
- **When D2 lands:** delete `src/lib/compute-terms.ts` and its drift test, and read `current_terms_version` from `GET /compute/me` in the page frontmatter; `COMPUTE_REQUESTS_ENABLED` flips with the published terms, in a reviewed change.
- **Open:** the terms page (`/policies/compute-terms/`) is a held draft, so the link 404s on staging until docs publish. The active state's body mentions the compute page; the link comes with E2.

## 2026-11-07 - Authorization-request POST exempt from the form check (nemar-cli#2020, ADR 0031)

Branch `feature/issue-2020-authorize-post`, PR to `staging`. Astro's `security.checkOrigin` is off; `src/middleware.ts` runs `crossSiteFormRefusal` (`src/lib/origin-check.ts`), Astro's rule plus one exemption (POST, exactly `/oauth/authorize`, urlencoded). The page refuses a cross-site POST that names a consent field (`readAuthorizeInput`).
Expand Down
113 changes: 113 additions & 0 deletions .context/local-backend.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Running the website against a local nemar-cli backend

Worked on 2026-11-06 (website#494). It runs the real backend, not a stand-in:
`scripts/local-backend.mjs` imports nemar-cli's own Worker entry point, its real migrations over a
`bun:sqlite` database, and its real session issuance, and serves them with `Bun.serve`. The only
additions are the platform's: an `ExecutionContext`, and the backend's documented `X-Test-Bypass`
for its rate limiter (which needs the Cloudflare Cache API that bun lacks).

## Safety

The server listens on `127.0.0.1` only (`Bun.serve({ hostname })`; without it bun binds every
interface). It adds `X-Test-Bypass` to every request and prints a fixed owner key, so it is never to
be reachable from another machine. The script refuses to start under a production-like
`ENVIRONMENT` (anything but `test`, `dev`, `development`, `local`), a `BIND_HOST`, `HOST` or
`BUN_HOST` that is not loopback, or a `WEB_ORIGIN` that is not loopback, and exits if the socket it
got is not `127.0.0.1`. `test/local-backend-guard.test.ts` runs the refusals.

## Steps

1. A checkout of the nemar-cli branch to run (for the compute card, the Epic D branch
`feature/issue-2021-epic-compute-users` or any branch after it). Make your own worktree; do not
touch a shared checkout. Run `bun install` at its root and in `backend/`.
2. In this repo:

```bash
NEMAR_CLI_DIR=/path/to/nemar-cli-checkout bun scripts/local-backend.mjs
```

It seeds an owner and nine `@nemar.test` accounts (complete profiles and an ORCID identity row;
one per compute state, plus `noaffiliation`, `noprofile` and `noorcid`, which a request refuses).
For each it signs in through the real `POST /auth/code/request` and `POST /auth/code/verify`
(the dev backend echoes the code for `@nemar.test`), prints the session cookie, and writes them
to `.local-backend/cookies.json` (git-ignored). The database is in memory: a restart resets it.
3. In another terminal:

```bash
PUBLIC_API_BASE_URL=http://localhost:8787 bun run dev
```

The middleware verifies a real cookie against `${PUBLIC_API_BASE_URL}/auth/me`, and the browser's
calls go through the `/api/v1` proxy to the same backend. Localhost is an allowed Origin there.
4. In the browser, on `http://localhost:4321`, set the cookie from the script's output and open
`/settings`: `document.cookie = "nemar_session=<value>; path=/"`. (The development login mock
never reaches the backend; pasting the cookie is what does.) A request made from the card is a
real write; grant it from a terminal with the owner key the script prints:

```bash
curl -s -X POST -H "Authorization: Bearer <owner key>" \
http://localhost:8787/admin/compute/access/grant/localrequested
```

## Seeding the compute dashboard (`--seed-compute`)

```bash
NEMAR_CLI_DIR=/path/to/nemar-cli-checkout bun scripts/local-backend.mjs --seed-compute
```

Adds, with the backend's own test helpers and ledger services (never a stand-in): `localrich` (active; a CPU and a GPU grant, charges on several days of the last 30 on both pools, one administrator's correction naming no job, an open hold, 40 jobs over the ten states and four front doors with enough finished ones for a second page, and queue samples with figures, a real zero and a thin sample), `localfrozen` (a frozen pool), `localoverrun` (a negative available) and `localgranted` (granted through the REAL admin routes, access then a pool grant: no jobs, no charges). Their cookies print with the others. `PORT=<n>` moves the backend (and `PUBLIC_API_BASE_URL` with it). The loopback guards are unchanged.

There is no unverified persona: signing in by emailed code verifies the email. `bun run dev` without `PUBLIC_API_BASE_URL` has the `@nemar.cpending` mock for that screen. The database refuses a markup-bearing `app_id` (a CHECK on `compute_jobs`), so the "hostile text" case is covered by unit tests, not by a seeded row.

A second browser profile on the same host shares cookies; use `http://127.0.0.1:4321` beside `http://localhost:4321` to keep two sessions apart.

## What differs from production

- `ENVIRONMENT` is `test`: the backend takes compute requests whether or not the terms are
published, and no mail is sent (no mail key is configured). The website shows the request form
because `PUBLIC_API_BASE_URL` is not the production API (`computeRequestsOpen`).
- The CLI keys card in Settings still shows the `astro dev` mock list (it is gated on the dev flag).
- Under `bun run dev` with no `PUBLIC_API_BASE_URL`, the `@nemar.*` personas of `src/lib/auth-dev.ts`
need no backend at all (`@nemar.cnone` and the other compute personas render each state).

## If it breaks

The script fails loudly at the first thing it cannot do (a missing `NEMAR_CLI_DIR`, a missing
`backend/node_modules`, a migration the checkout does not have). Do not substitute a mock backend:
fix the checkout, or escalate with the error.

## The admin Compute tab (website#497)

The script also seeds the admin Compute tab's world: sign in as `owner` (an owner) or `adminself`
(a non-owner administrator whose own compute request is open) and open `/admin/compute`. It seeds
a queue of people who asked (varied countries, one purpose full of markup, a service account, an
account with no name or affiliation), sets a non-zero owner default allocation through the real
`PUT /admin/compute/policy`, and posts two scheduler accounting rows no job claims through the real
observer route under a really minted run token.

`LOCAL_BACKEND_JOBS=1` runs on the backend's lifecycle fixture (Miniflare's R2 and the Tapis
stand-in, no tenant) and also makes jobs through the real services, flagged with the same
`UPDATE compute_jobs SET flag_... = 1` the backend's own reporting test uses. No real path
produces a flag without a live tenant.

Worked on 2026-11-07. Differences from production: the compute switch reads `off` here (it needs the
exact production environment), so Reconciliation shows its paused banner; no mail is sent, so a
decision answers `email_sent: false` and the row says the person could not be emailed. Browser
windows cannot be narrower than about 600 px in the Chrome tools; check 375 px in devtools.

## The job page (website#496)

`--seed-compute-jobs` (with `--seed-compute` and `LOCAL_BACKEND_JOBS=1`) makes the job page's jobs for
the account `localjobs` (sign in with the `jobs` cookie) through the real services and the Tapis
stand-in: a finished job (three outputs, a log), a running one, one to cancel, a failed job with a
hostile log, one with a log over 1 MiB, one with 1,002 outputs (a second page of links), one whose
results expire in two days, one whose results are past the 30 days (the real 410), a running job of
another account (the 404 check) and a job that advances by itself every 20 seconds (queued, running,
finished). Ids are printed and written to `.local-backend/jobs.json`. The seed runs the backend with
`ENVIRONMENT=development` (compute only runs in `production` or `development`); mail stays fenced.
The seed takes about 40 seconds, and the watch job starts advancing when it ends: open it at once.
Result links point at the local object store on `127.0.0.1`, which is why `safeResultHref` accepts
`http:` on loopback. Not reachable here: a `stale: true` job (needs the stand-in down at read time;
covered by the captured fixture) and a real 5xx.

`bun run dev` (Vite) 500s a request URL that ends in `.txt`, such as a results cursor `?after=out/f.txt`; this is dev-server only (a deployed build was checked), so the seed names its outputs `.dat`.
33 changes: 33 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -119,12 +119,38 @@ jobs:
# test/oidc-authorize-contract-drift.test.ts is a SEVENTH, against
# shared/contract/oidc-authorize.ts (the authorization endpoint's routes, answer kinds, codes
# and reasons) and shared/oidc.ts beside it (the page's path). Same `dev` pairing, and strict.
# test/compute-terms-contract-drift.test.ts is an EIGHTH, against shared/contract/compute-terms.ts:
# the compute terms version a request must name (website ADR 0030, interim until phase D2's
# GET /compute/me serves it). Same `dev` pairing, strict, and red until the compute epic
# reaches nemar-cli `dev`; the ref stays `dev`.
# test/compute-user-contract-drift.test.ts is a NINTH, against shared/compute-state.ts and
# shared/contract/compute-user.ts and compute.ts: the job states, front doors, usage groups,
# page bounds and read-refusal words the /compute page branches on (phase E2). Same `dev`
# pairing, strict, and red until the compute epic reaches nemar-cli `dev`.
# test/compute-admin-contract-drift.test.ts is a TENTH: the compute state machine, the decision
# words and the bounds the admin Compute tab branches on (compute-access-state.ts,
# compute-state.ts, contract/compute-access.ts, contract/compute-admin.ts). Same `dev` pairing,
# and strict: red on a phase PR until the compute epic reaches nemar-cli `dev`.
# test/compute-job-contract-drift.test.ts is an ELEVENTH, against shared/compute-state.ts,
# shared/contract/compute-submit.ts and compute-responses.ts: the failure codes, the refusal
# statuses, the truncated-log header, the bounds and the cancel outcomes the job page
# (phase E3) branches on. Same `dev` pairing, strict, and red until the compute epic reaches
# nemar-cli `dev`.
- uses: actions/checkout@v4
with:
repository: nemarOrg/nemar-cli
path: nemar-cli-contract
sparse-checkout: |
/shared/compute-access-state.ts
/shared/compute-state.ts
/shared/contract/account-copy.ts
/shared/contract/compute-access.ts
/shared/contract/compute-admin.ts
/shared/contract/compute-responses.ts
/shared/contract/compute-submit.ts
/shared/contract/compute-terms.ts
/shared/contract/compute-user.ts
/shared/contract/compute.ts
/shared/contract/docs-auth.ts
/shared/contract/mail-admin.ts
/shared/contract/oidc-authorize.ts
Expand All @@ -139,6 +165,13 @@ jobs:
run: bun run test
env:
NEMAR_CLI_ACCOUNT_COPY: nemar-cli-contract/shared/contract/account-copy.ts
NEMAR_CLI_COMPUTE_ACCESS_CONTRACT: nemar-cli-contract/shared/contract/compute-terms.ts
NEMAR_CLI_COMPUTE_CONTRACT: nemar-cli-contract/shared/contract/compute.ts
NEMAR_CLI_COMPUTE_RESPONSES_CONTRACT: nemar-cli-contract/shared/contract/compute-responses.ts
NEMAR_CLI_COMPUTE_SUBMIT_CONTRACT: nemar-cli-contract/shared/contract/compute-submit.ts
NEMAR_CLI_COMPUTE_STATE: nemar-cli-contract/shared/compute-state.ts
NEMAR_CLI_COMPUTE_USER_CONTRACT: nemar-cli-contract/shared/contract/compute-user.ts
NEMAR_CLI_COMPUTE_ADMIN_CONTRACT: nemar-cli-contract/shared/contract/compute-admin.ts
NEMAR_CLI_DOCS_CONTRACT: nemar-cli-contract/shared/contract/docs-auth.ts
NEMAR_CLI_MAIL_ADMIN_CONTRACT: nemar-cli-contract/shared/contract/mail-admin.ts
NEMAR_CLI_OIDC_AUTHORIZE_CONTRACT: nemar-cli-contract/shared/contract/oidc-authorize.ts
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,4 @@ playwright-report/
# hand-created locally; point NEMAR_CLI_ACCOUNT_COPY at a real checkout
# instead.
nemar-cli-contract/
.local-backend/
Loading
Loading