Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .context/decisions/0030-compute-pages-follow-adr-0010-phase-3.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,21 @@ The captured fixtures in `test/fixtures/compute/` gain a `GET /compute/me` captu
- **Shared copy.** Seven keys were added to nemar-cli's contract and transcribed (`compute_page.*`, `compute_access.granted.link`), and `compute_access.granted.body` no longer promises submission from the website: jobs are submitted with the command line.
- **Still open from the hand-off above:** replacing `compute-terms.ts` and `COMPUTE_REQUESTS_ENABLED` with `access.current_terms_version` and `access.requests_open` is a follow-up; `/compute` already reads `requests_open` (falling back to `terms_published`) for its own "request" link.

## Addendum: the job page (phase E3, website#496, 2026-11-07)

`/compute/jobs/<id>` reads `GET /compute/me/jobs/:id` and, on request, the log, the result links and the cancel (nemar-cli Epic D, phase D3). The decisions that are expensive to reverse:

- **Bearer result links are minted only by a click and live only in the DOM.** The server-rendered page carries no link and calls neither the log nor the results client (a source scan fails the build otherwise). "Get download links" and "Renew links" are the only callers; a link reaches an `href` only through `safeResultHref` (`https:`, or `http:` on loopback for the local backend), with `rel="noopener noreferrer"` and no referrer, and is never written to storage, the URL, the console, an attribute other than `href` or the clipboard. A page restored from the back-forward cache clears the list, and rows are replaced when the links' own expiry passes. The backend's `notice` about keeping them private is shown as served, not retyped.
- **The log is inert text, and hostile characters are shown, not removed.** ANSI formatting is stripped as formatting; everything else goes through `displaySafeBody` (ADR 0029), so a bare carriage return becomes a line break (nothing can be overwritten) and a control, override, isolate or zero-width character is written out (`\x1b`, `\u202e`). A sequence that never ends is not stripped and swallows nothing after it. Lines are capped at 2,000 characters. The backend serves the FIRST 1 MiB with `X-Nemar-Truncated: true`, so a long log shows its beginning and says so (nemar-cli#2094 asks for a tail). The proxy forwards that one header and its constant has one reader.
- **The page claims only what the record carries.** The job has three times (created, started, ended) and no per-stage times (nemar-cli#2097), so the timeline shows a time on three steps and a step it cannot vouch for reads "not known", never "done"; Archiving is never claimed for a failed or canceled job. No sentence says whose fault a failure was (the user view serves no attribution) or promises a refund. The 17 failure codes, the cancel outcomes, the refusal statuses, the header and the bounds are transcribed and drift-tested (`test/compute-job-contract-drift.test.ts`); the wording is website-owned (`compute-copy.ts`, `compute-failure.ts`), and the cancel dialog repeats the CLI's own charge rule.
- **Polling is cheap by construction.** The first read is 10 seconds after load; an unchanged answer doubles the delay to 60 (the backend refreshes a live job from the cluster at most once a minute, so asking faster adds load and no freshness), a change resets it, a fifth error in a row stops it, a hidden tab pauses it and a visible one refreshes at once, an ended or unknown state stops it for good, and it stops after two hours with a Resume button. The schedule is pure (`nextPoll`, an injected clock); `compute-job-live.ts` is the one job module that reads a clock or holds a timer, which `test/compute-dashboard-ui.test.ts` names as its single exception.
- **The retention date is derived until the server states it.** `ended_at` plus `limits.results_days` gives "kept until", with the backend's own strict `now > ended + days` rule; the 410 stays authoritative and the button stays until the backend says so (nemar-cli#2096 asks for the date). A page never promises a charge beyond the dialog's sentence and the refreshed record.
- **One 404, never an oracle.** An absent, foreign or malformed id is one page with status 404 (a malformed id without calling the API), the headers `Cache-Control: private, no-store` and `Referrer-Policy: no-referrer` are set before any return, and `/compute` joins `/oauth` in `routeBypassesEdgeCache`.

Dev-server note: `bun run dev` (Vite) answers a URL that ends in `.txt` (here, a results cursor that is a path ending in `.txt`, passed as `?after=`) as a static file lookup and 500s. It was checked against a deployed build and the effect is dev-only, so there is no code change; the local seed uses `.dat` outputs.

Review round (same PR): the job read and the cancel carry named deadlines longer than the backend work behind them (`COMPUTE_JOB_READ_TIMEOUT_MS` 12 s, `COMPUTE_CANCEL_TIMEOUT_MS` 20 s, against a cluster read of about 8 s), so a slow cluster shows the stored record with `stale: true` or the cancel's own outcome instead of a false error. What the page holds of the result links is a pure state machine (`compute-job-results-state.ts`), tested for the rules that matter: rows are dropped after `expires_at`, a back-forward-cache restore drops every link and brings the "Get download links" button back (it is derived from the job, not from what was shown last), and a 410 never returns the button. A plain-http result link is accepted only when the page itself is on a loopback host. Logs write out the three implicit direction marks too (an option of `displaySafeBody`; mail rendering is unchanged).

## Addendum: the dataset page dialog (phase E5, website#498, 2026-11-07)

The dataset page's "Compute instructions" dialog now describes `nemar compute` instead of an S3 prefix and a Neuroscience Gateway flow that never shipped. The decisions that are expensive to reverse:
Expand All @@ -96,5 +111,5 @@ Receipts for this addendum: website#498, #493; nemar-cli#2021, #2024, #2090, #20

## Receipts

- website#493 (epic), website#494 (phase E1), website#495 (phase E2), website#498 (phase E5); nemar-cli#2021 (Epic D), #2071 (phase D1, commit 3a869903).
- website#493 (epic), website#494 (phase E1), website#495 (phase E2), website#496 (phase E3), website#498 (phase E5); nemar-cli#2094 to #2097; nemar-cli#2021 (Epic D), #2071 (phase D1, commit 3a869903).
- ADRs 0006, 0007, 0010, 0014, 0015, 0029; nemar-cli ADRs 0040, 0042, 0118.
17 changes: 17 additions & 0 deletions .context/local-backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,3 +94,20 @@ Worked on 2026-11-07. Differences from production: the compute switch reads `off
exact production environment), so Reconciliation shows its paused banner; no mail is sent, so a
decision answers `email_sent: false` and the row says the person could not be emailed. Browser
windows cannot be narrower than about 600 px in the Chrome tools; check 375 px in devtools.

## The job page (website#496)

`--seed-compute-jobs` (with `--seed-compute` and `LOCAL_BACKEND_JOBS=1`) makes the job page's jobs for
the account `localjobs` (sign in with the `jobs` cookie) through the real services and the Tapis
stand-in: a finished job (three outputs, a log), a running one, one to cancel, a failed job with a
hostile log, one with a log over 1 MiB, one with 1,002 outputs (a second page of links), one whose
results expire in two days, one whose results are past the 30 days (the real 410), a running job of
another account (the 404 check) and a job that advances by itself every 20 seconds (queued, running,
finished). Ids are printed and written to `.local-backend/jobs.json`. The seed runs the backend with
`ENVIRONMENT=development` (compute only runs in `production` or `development`); mail stays fenced.
The seed takes about 40 seconds, and the watch job starts advancing when it ends: open it at once.
Result links point at the local object store on `127.0.0.1`, which is why `safeResultHref` accepts
`http:` on loopback. Not reachable here: a `stale: true` job (needs the stand-in down at read time;
covered by the captured fixture) and a real 5xx.

`bun run dev` (Vite) 500s a request URL that ends in `.txt`, such as a results cursor `?after=out/f.txt`; this is dev-server only (a deployed build was checked), so the seed names its outputs `.dat`.
9 changes: 9 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,11 @@ jobs:
# shared/contract/compute-user.ts and compute.ts: the job states, front doors, usage groups,
# page bounds and read-refusal words the /compute page branches on (phase E2). Same `dev`
# pairing, strict, and red until the compute epic reaches nemar-cli `dev`.
# test/compute-job-contract-drift.test.ts is an ELEVENTH, against shared/compute-state.ts,
# shared/contract/compute-submit.ts and compute-responses.ts: the failure codes, the refusal
# statuses, the truncated-log header, the bounds and the cancel outcomes the job page
# (phase E3) branches on. Same `dev` pairing, strict, and red until the compute epic reaches
# nemar-cli `dev`.
# test/compute-admin-contract-drift.test.ts is a TENTH: the compute state machine, the decision
# words and the bounds the admin Compute tab branches on (compute-access-state.ts,
# compute-state.ts, contract/compute-access.ts, contract/compute-admin.ts). Same `dev` pairing,
Expand All @@ -141,6 +146,8 @@ jobs:
/shared/contract/account-copy.ts
/shared/contract/compute-access.ts
/shared/contract/compute-admin.ts
/shared/contract/compute-responses.ts
/shared/contract/compute-submit.ts
/shared/contract/compute-terms.ts
/shared/contract/compute-user.ts
/shared/contract/compute.ts
Expand All @@ -160,6 +167,8 @@ jobs:
NEMAR_CLI_ACCOUNT_COPY: nemar-cli-contract/shared/contract/account-copy.ts
NEMAR_CLI_COMPUTE_ACCESS_CONTRACT: nemar-cli-contract/shared/contract/compute-terms.ts
NEMAR_CLI_COMPUTE_CONTRACT: nemar-cli-contract/shared/contract/compute.ts
NEMAR_CLI_COMPUTE_RESPONSES_CONTRACT: nemar-cli-contract/shared/contract/compute-responses.ts
NEMAR_CLI_COMPUTE_SUBMIT_CONTRACT: nemar-cli-contract/shared/contract/compute-submit.ts
NEMAR_CLI_COMPUTE_STATE: nemar-cli-contract/shared/compute-state.ts
NEMAR_CLI_COMPUTE_USER_CONTRACT: nemar-cli-contract/shared/contract/compute-user.ts
NEMAR_CLI_COMPUTE_ADMIN_CONTRACT: nemar-cli-contract/shared/contract/compute-admin.ts
Expand Down
5 changes: 4 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ src/
upload.astro upload/success.astro upload flow (dropzone + BIDS pre-check + full BIDS validation in a worker + direct-to-storage PUTs)
settings.astro account: name/email/ORCID/GitHub/profile self-service; CLI keys card; compute access card
compute/index.astro compute dashboard (app host, ADR 0030): access, balances, usage chart per pool, jobs with Load more, queue waits; each read and panel fails alone
compute/jobs/[id].astro one compute job (app host, ADR 0030 E3): timeline, resources and charge, failure sentence, log and result links on request (bearer links minted only by a click), cancel; private no-store, one 404 for absent, foreign or malformed ids; the page the results-ready mail links to
admin/publication-requests.astro admin-only (role=admin; 404s for others)
admin/mail.astro admin/mail/{[id],queues,away}.astro admin Mail tab: tickets, queues, away windows (ADR 0029); email is escaped text only
admin/compute.astro admin Compute tab (website#497, ADR 0030): Requests (grant, deny, suspend, revoke by id, each through ComputeDecisionDialog) and read-only Pools, Usage, Flagged jobs, Reconciliation; a section the backend does not serve says so and links its nemar-cli issue; nothing owner-only or cost-bearing is offered
Expand All @@ -98,6 +99,8 @@ src/
compute-view.ts compute-labels.ts compute-badges.ts pure view models (panelsFor, access, balances, queue, jobRowView, usage), the page's short labels (sentences live in account-copy.ts), job-state badges
compute-format.ts compute-chart.ts integer milli-SU formatting (`suFixed`: the CLI's three-decimal digits; `formatSu`: the shortest exact decimal, the same rule as nemar-cli's `formatSu`) and the usage chart geometry (nice ceiling from the stacked total, hand-rolled SVG, ADR 0007)
compute-jobs-dom.ts builds appended job rows with createElement/textContent only
compute-job-view.ts compute-logs.ts compute-failure.ts compute-copy.ts the job page's pure modules (E3): timeline, canCancel, retention window, results rows, cancel outcome words and the polling schedule (`nextPoll`, no clock); `prepareLog` (ANSI stripped, hostile characters written out, long lines capped); one sentence per failure code; the website-owned sentences
compute-job-dom.ts compute-job-page.ts compute-job-live.ts the job page's browser side: nodes from the same view models (createElement and textContent only), the controller (poll, log, results, cancel), and the ONE job module that reads a clock or holds a timer
compute-admin-api.ts admin Compute client (browser-safe): access list and decisions by id, pools, jobs by flag, unmatched accounting rows, access report, status, the badge count; { error, code } reader, decision failure classes, no POST retry
compute-admin-contract.ts hand copy of the closed lists the Compute tab branches on (drift-tested, test/compute-admin-contract-drift.test.ts, strict via NEMAR_CLI_COMPUTE_ADMIN_CONTRACT)
compute-admin-view.ts pure view models for the Compute tab: request row, allowed buttons, decision and failure sentences, pool and job rows, parameters, pager
Expand Down Expand Up @@ -128,7 +131,7 @@ src/
eeg-viewer.css global signal-viewer styles (runtime-built DOM), imported by BidsTree and the embed
test/
fixtures/ qa-aggregates, qa-file-dataqual, qa-hed-summary (Phase 3)
fixtures/compute/ captured from the real nemar-cli routes, never hand-written: `NEMAR_CLI_DIR=<Epic D checkout> bun scripts/capture-compute-fixtures.mjs` (access, E1) and `bun scripts/capture-compute-read-fixtures.mjs` (me, usage, jobs, errors; E2); `fixtures/compute/cli-help/` is the real `nemar ... --help-all` text the dialog's commands are checked against: `NEMAR_CLI_DIR=<Epic D checkout> bun scripts/capture-compute-cli-help.mjs` (E5)
fixtures/compute/ captured from the real nemar-cli routes, never hand-written: `NEMAR_CLI_DIR=<Epic D checkout> bun scripts/capture-compute-fixtures.mjs` (access, E1), `bun scripts/capture-compute-read-fixtures.mjs` (me, usage, jobs, errors; E2) and `bun scripts/capture-compute-job-fixtures.mjs` (job, log, results, cancel, refusals; E3, on the backend's lifecycle fixture and Tapis stand-in); `fixtures/compute/cli-help/` is the real `nemar ... --help-all` text the dialog's commands are checked against: `NEMAR_CLI_DIR=<Epic D checkout> bun scripts/capture-compute-cli-help.mjs` (E5)
public/ static logos + brain hero assets (og/ cards are generated, gitignored)
public/brand/ GENERATED press logo kit (SVG, PDF, transparent PNG; color, reverse, black, white); rebuild with `bun scripts/generate-brand-kit.mjs`, never hand-edit
public/vendor/bids-validator/<ver>/ GENERATED worker bundle of the BIDS validator + SHA256SUMS + provenance + licenses (ADR 0028); never hand-edited
Expand Down
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,16 @@ The release pull request from `staging` to `main` moves those entries under the

### Added

- **The compute job page** (website#496, epic website#493 phase E3; nemar-cli epic #2021 phase D3, website ADR 0030 addendum).
`/compute/jobs/<id>` shows one job: its state on a timeline that claims only what the record carries (three times), its resources and charge (the same three-decimal digits as `nemar compute status`), a plain sentence for each of the 17 failure codes, its log on request, its result links on request (renewable), and a cancel for a live job.
A live job refreshes itself every 10 seconds, backing off to 60 while nothing changes, pausing in a hidden tab and stopping at an ended state.
The log is shown as inert text with hostile characters written out (`\x1b`, `\u202e`) rather than silently dropped; a log cut at 1 MiB says so and points at `nemar compute logs` (the end of a long log cannot be read yet: nemar-cli#2094).
Result links are bearer links: viewing the page mints none, only the button does, and they are never stored, logged or copied.
The page is the one the "results ready" email links to, answers the same 404 for an absent, foreign or malformed id, and is never edge-cached (`/compute` joins `/oauth` in `routeBypassesEdgeCache`).
The API proxy forwards one more response header, `X-Nemar-Truncated`.
Backend gaps filed: nemar-cli#2094 (log tail), #2095 (links as attachments), #2096 (state when results are deleted), #2097 (per-stage times).
`scripts/capture-compute-job-fixtures.mjs` captures nine fixture files from the real router, and `scripts/local-backend.mjs --seed-compute-jobs` seeds jobs through the real services and the Tapis stand-in.

- **The `/compute` dashboard** (website#495, epic website#493 phase E2; nemar-cli epic #2021 phase D2, website ADR 0030 addendum).
A signed-in person sees their compute access, CPU and GPU balances (granted, held, charged, available, in the same three-decimal service units `nemar compute balance` prints), a daily usage chart per pool stacked by partition (hand-rolled SVG, themed by new `--chart-*` tokens, with the numbers in a table beside it), their jobs from every front door with "Load more", and the platform's recent queue waits ("Not enough data" with the count when fewer than five jobs started).
A person without active access sees why and how to ask, not an error; a suspended or ended account keeps its history read-only; one failed read shows one notice that says the figures are missing, not zero.
Expand Down
Loading
Loading