Skip to content

Phase A4: Tapis client and stand-in #2009

Closed

Description

@neuromechanist

Goal

The Worker can talk to a Tapis tenant, and everything about that conversation is tested against a stand-in before a tenant exists.
After this phase there is a typed Tapis client with two interchangeable credential strategies, a stand-in server with failure modes, the dark switch computeEnabled, and an admin status view the owner can use to check credentials once secrets are set.

Routing

Scope

In: backend/src/services/tapis/; backend/test/helpers/tapis-standin.ts; computeEnabled; GET /admin/compute/status and nemar admin compute status.
Out: orchestration, webhooks and cron (#2010), the person-level Tapis username and token minting for people (Epic D), any real tenant call, any secret value.

Design references

Design sections 3 (Tapis identity, the compromised-partner caution), 7 (states, webhooks), 12 (questions 1, 2, 10, 13, 16). Implementation notes section 5 (Tapis facts). Template for the token cache: backend/src/services/github-auth.ts; for errors: HttpError and isRetryable in services/retry.ts; for the base URL override: GITHUB_API() in services/github/shared.ts; for the switch and its pin: autoRepullEnabled in services/repull-switches.ts and backend/test/repull-switches-flag.test.ts. Tapis OpenAPI files: JobsAPI.yaml in tapis-project/openapi-jobs, service/resources/openapi_v3.yml in tapis-project/tokens-api and tapis-project/authenticator.

Changes

  • backend/src/services/tapis/config.ts: tapisConfig(env) returns { strategy: "password" | "service", baseUrl, tenantId, ... } or { unconfigured: true }. Optional bindings in types/bindings.ts: TAPIS_BASE_URL, TAPIS_TENANT_ID, TAPIS_AUTH_STRATEGY, TAPIS_USERNAME, TAPIS_PASSWORD, TAPIS_SERVICE_TOKEN. They are Worker secrets or owner-set vars, never in the toml (a tenant name is not public); the toml comment block lists the names only. TAPIS_BASE(env) returns globalThis.NEMAR_TAPIS_API_URL if set, else env.TAPIS_BASE_URL.
  • tapis/auth.ts: password strategy POST /v3/oauth2/tokens with { username, password, grant_type: "password" }; service strategy POST /v3/tokens with the service token in X-Tapis-Token and { account_type: "user", token_tenant_id, token_username, access_token_ttl }. Both answer result.access_token.access_token with expires_in. Module-scope cache keyed by strategy and username, refresh lead, one shared in-flight promise, a failed mint is dropped; invalidateTapisToken after a 401 and exactly one retry. Dev and production never share a credential because each Worker reads its own secrets.
  • tapis/schemas.ts: zod from the OpenAPI schemas (ReqSubmitJob, Job, JobListDTO, the response envelope { status, message, version, result }, history events, output listing, subscription). .passthrough() on responses, strict on what NEMAR sends; a response that fails its schema is invalid_response, never partially trusted.
  • tapis/client.ts: submitJob, getJob, getJobHistory, cancelJob, searchJobsByTag, listJobs({ listType, limit, skip }), subscribeWebhook(uuid, address, ttlMinutes), listOutputs, downloadOutput(uuid, path, maxBytes). All paths are under /v3/jobs/ (/submit, /{uuid}, /{uuid}/history, /{uuid}/cancel, /search, /list, /subscribe/{uuid}, /{uuid}/output/list/{path}, /{uuid}/output/download/{path}). Every call has a 15-second timeout and a bounded body read; failures are a TapisError with kind unconfigured | unauthorized | not_found | rejected | unavailable | invalid_response wrapping HttpError; reads retry through isRetryable, submitJob never retries inside the client (the caller searches by tag first). GPUs have no field on ReqSubmitJob; the request builder passes them as a parameterSet.schedulerOptions entry and the partition as execSystemLogicalQueue. Say so in a comment.
  • backend/test/helpers/tapis-standin.ts: Bun.serve built from the request and response examples in the OpenAPI files, in-memory jobs with advance(uuid, status, condition?), tag search, subscription capture and fireWebhook(uuid) (POSTs the recorded address with a CloudEvents-shaped body), a request log with headers, and failure modes: forced status per route, hang, non-JSON body, 401 then accept after refresh, down. Header comment: no response was recorded from a real tenant. Add if (import.meta.main) so it can be started by hand.
  • backend/src/services/compute-switches.ts: computeEnabled(env) is (env.ENVIRONMENT === "production" || env.ENVIRONMENT === "development") && env.COMPUTE_ENABLED === "1", so staging (the dev Worker) can run compute against the dev tenant with its own credentials while every compute cron stays production-only; in development a job advances by its webhook, the read-through refresh and nemar admin compute jobs advance (Phase A5: Job lifecycle, webhooks, settlement and results #2010); computePipelinesDispatchEnabled(env) is computeEnabled(env) && env.COMPUTE_PIPELINES_DISPATCH_ENABLED === "1". Both names are absent from every table of wrangler-sccn.toml.
  • GET /admin/compute/status (admin) in routes/admin/compute.ts and nemar admin compute status [--json]: both switches as the Worker reads them, the strategy, whether each of Tapis, the two R2 bindings with the signing secrets, and the ACCESS key is configured (booleans, never values), and the result of one token mint and one health call, as fixed words.

Tests (write these first)

  1. backend/test/compute-switches-flag.test.ts (template repull-switches-flag.test.ts): walks every table of the toml and fails if either switch name appears; computeEnabled is false for "true", " 1", unset, and an ENVIRONMENT that is unset or anything other than exactly production or development, and true for exactly "1" in those two. Mutation: make the check === "true"; it must fail.
  2. backend/test/tapis-client.test.ts against the stand-in: each call sends X-Tapis-Token and the documented body; a 401 refreshes the token once and retries once, a second 401 is unauthorized; a hang ends as unavailable within the timeout; a non-JSON or schema-failing body is invalid_response; concurrent calls mint one token.
  3. Same file, both strategies: password sends the password grant body, service sends the service token header and the owner username; the same client function works under either.
  4. submitJob builds the documented request (tag, no archive system fields, schedulerOptions for GPUs); a test asserts the exact JSON the stand-in received.
  5. backend/test/compute-status-route.test.ts and a CLI test: an unconfigured Worker reports unconfigured; no secret value or tenant URL appears in any output.

Acceptance criteria

  • Neither strategy needs a code change to switch; selection is configuration.
  • No tenant name, hostname or credential in the diff or the logs; the status view prints booleans and fixed words.
  • The PR body lists what is unverified: which strategy the operator grants, whether listType=ALL_JOBS returns every person's jobs, the POST search body shape, the service-token minting permission, the webhook retry policy, and the schedulerOptions spelling for GPUs.
  • The stand-in file says no real response was recorded.

Gates

  • bun test backend/test/compute-switches-flag.test.ts backend/test/tapis-client.test.ts backend/test/compute-status-route.test.ts test/admin-route-inventory.unit.test.ts test/api-export-surface.unit.test.ts
  • bun run typecheck, bunx biome check src shared backend/src, bun test

Rules that bite here

  • Switches are dark by default and pinned absent from the toml; production means exactly production.
  • Tokens are secrets: never logged, never in an error message, never in a test snapshot.
  • A hung upstream must not hang a Worker request: every call has a timeout.
  • Never mock the client: the stand-in is a real HTTP server and the client under test is the production one.

Escalate to the owner if

  • The OpenAPI files and the stand-in examples disagree on a field NEMAR depends on.
  • A new Worker secret name is needed beyond the list above.

When merged

Add the verified-or-unverified list to design section 12 as "assumed in code", and the secret names to the owner actions.

Activity

  1. neuromechanist commented on Nov 4, 2026

    @neuromechanist
    ContributorAuthor

    Driver amendment, 2026-11-04, from the phase A1 review (PR #2030) and the Opus advisor. This comment is part of the phase brief; where it differs from the body above, this comment wins.

    • Security question to settle from the vendored OpenAPI: Tapis's job record includes parameterSet with the environment variables. The launch specification, including job_token, travels in the environment variable NEMAR_LAUNCH_SPEC, so whoever can read the Tapis job record can read the token. Record in the client's notes exactly which job fields and which roles can read parameterSet, and say so in the PR body. Who the Tapis owner of a CLI or web job is stays the repository owner's decision (see the amendment on Phase A5: Job lifecycle, webhooks, settlement and results #2010); do not decide it here.
  2. neuromechanist commented on Nov 4, 2026

    @neuromechanist
    ContributorAuthor

    Owner decision, 2026-11-04 (recorded by the driver). This comment is part of the phase brief; where it differs from the body or an earlier comment above, this comment wins.

    The Tapis owner question is settled: there is a single NEMAR tenant and NEMAR owns every job, so the client's owner stays a parameter (a NEMAR service identity from configuration, never a person). Record in the PR body which Tapis roles can read a job's parameterSet, as already asked, but the person-readable-token concern is closed: no person has Tapis access.

  3. neuromechanist commented on Nov 4, 2026

    @neuromechanist
    ContributorAuthor

    Merged into the epic branch by #2038 (squash c3e815b). Round 1 (two reviewers) and round 2 (delta) converged; 20 round-1 items and four round-2 Low findings fixed; 357 targeted tests pass; every guard mutated. Nothing was verified against a real Tapis tenant: the unverified list is in the PR body. Owner decision pending: the secret name ACCESS_GATEWAY_API_KEY is provisional. The amendment for the next phase is on #2010.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or enhancementhpc

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions