Skip to content

Harmonize agent instructions into the docs repo #1336

Closed
7 of 7 issues completed
Closed

Description

@neuromechanist

The idea

AGENTS.md is 633 lines and it is explicitly a map, not a manual: almost every section ends in a pointer at something under .context/. Those pointees are the actual reference material, and today they are readable only by whoever has this checkout open. Content in nemarOrg/docs is different in kind: it is one git pull away for any agent or person in any of our repos, and it is also published at docs.nemar.org, so it is reachable by URL with no checkout at all.

So the proposal is to move what belongs in a published reference out of this repo's agent files and into nemarOrg/docs, and leave AGENTS.md as a short set of hard rules plus pointers at docs pages. The same question applies to every other repo's AGENTS.md (website, docs, nemar-observability, nemar-tools); this repo is the biggest and should go first, and whatever pattern it settles becomes the pattern for the rest.

Related: nemarOrg/docs#28 is the first piece of this in practice. Nobody could answer "which hostname serves what" from any published page, so it was answered by guessing a URL, and the guess shipped in OpenScience-Collective/osa. The facts existed, in four repos' configuration.

What makes this non-trivial

Not everything can move, and the sorting is the real work. Three buckets:

Publishable reference. Belongs on docs.nemar.org. The dataset ID bands, the release pipeline, the user flow and account kinds, the S3 layout, the Zarr serving contract, the CLI command surface. Much of this is ALREADY on the docs site in some form, which means we currently maintain two copies and they can drift; harmonizing removes a copy rather than adding a page.

Admin-gated runbook. Belongs under docs /admin/*, which is gated by NEMAR's own admin-only session (admin or owner role) through a Cloudflare Pages Function. edge-gated by Cloudflare Access was wrong when this epic was written and is what phase 0 existed to fix: the Access application on that Pages project covers preview deployments only, so every admin page answered 200 to anyone on production. See nemarOrg/docs ADR 0002 and nemar-cli ADR 0056. Verify a deploy with bun run probe:gate against production, never a preview. Disaster recovery is already there. The candidates here are parts of .context/systems-inventory.md and .context/validated_workflows.md.

Must stay in this repo. The docs repo's own AGENTS.md says it plainly: keep genuinely internal material out of the docs site, even behind the gate. That covers the webhook contracts, the blast-radius fences (dev D1 holds real users, the nemarDatasets org is shared with prod, the live nm000103-nm000107 datasets), and anything that is an instruction to an agent editing THIS checkout rather than a fact about the platform.

ADRs stay here regardless. They are binding decisions about this codebase, test/adr-index.unit.test.ts enforces that the index and the files on disk agree, and the never-delete-only-supersede rule depends on them being version-controlled next to the code they bind. This repo is public, so docs pages can link to them at their GitHub paths.

Reframed by ADR 0057 (2026-09-13)

Phase 0 built the admin gate and, in doing so, broke two of this epic's premises.

The gate controls who is SERVED, not who can read. nemarOrg/docs is public. The rule meant to
cover that gap says keep genuinely internal material in nemar-cli, and nemar-cli is public too.
So the original phase 3, "move the operational runbooks behind the gate", would have moved them from
one public repository to another and called the result gated.

"One git pull away" assumed repository access is how agents retrieve docs. It does not have to
be. The docs URL serves that, and serving it that way is what makes a public/private source split
possible at all.

So the epic is now: make docs.nemar.org the canonical retrieval surface for humans and agents,
with the public/gated split enforced at the source rather than only at serving.

Calibration, which is part of the decision: no credential value is in any of the public material,
nothing needs rotating, and this is not an incident. NEMAR is an open project and its public
documentation stays public on purpose. The bar is publish good examples, not a recipe for
attacking the platform
— a single fact about where something runs is fine, an assembled map of
host aliases, state paths, cron windows and deploy mechanism is the artifact that moves. Historical
exposure in git is accepted because operational detail ages out; that reasoning would not cover a
key.

Phases

Phase Issue
0 #1338 Gate the admin docs behind a NEMAR admin session — done, verified 27/27 in production
1 #1339 Inventory, the classification rule, and the docs-link drift guard
2 #1377 The agent retrieval surface: per-page markdown, llms.txt, bot and cache controls
3 #1341 Private admin source, the credential exchange, and the move out of public repos
4 #1340 Move the publishable reference into docs
5 #1342 Rewrite AGENTS.md to hard rules, docs URLs, and escalation
6 #1378 Apply the pattern to the other repos

Two orderings are load-bearing. Phase 2 precedes phase 5, because pointing agents at docs URLs
is only honest once those URLs are retrievable as markdown. Phase 3 precedes phase 4, so the
public/private split is settled before anything moves.

Retrieval and access

An admin or service account (ADR 0048) retrieves gated docs by exchanging an API key for a
short-lived docs-scoped token, never by presenting the key to the docs edge. That keeps ADR 0056's
separation working in both directions.

Losing read access must not cost a contributor the ability to report a problem: a non-admin who hits
an admin-adjacent issue opens it anyway and tags the admin team. No GitHub team exists in
nemarOrg today
, so one has to be created before phase 5 can point at it.

Not in scope

Rewriting the docs site's information architecture, and moving the ADRs.

Activity

  1. added
    documentationImprovements or additions to documentation
    P1High priority - important features, significant bugs
    on Sep 10, 2026
  2. neuromechanist commented on Sep 10, 2026

    @neuromechanist
    ContributorAuthor

    Revised scope after investigation, 2026-09-09

    Two things came out of checking the assumptions in this issue before starting, and both change the plan.

    1. The docs /admin/* gate does not exist. The docs repo's AGENTS.md says access control is applied at the edge by Cloudflare Access on docs.nemar.org/admin/*. It is not. All 12 pages under /admin/ answer 200 anonymously to any user agent, and all 12 are listed in the public sitemap, so they are crawlable rather than merely reachable.

    Nothing confidential is behind it, so this is not a leak: the 4,731 lines there are procedures plus identifiers, never credential values (secret NAMES and setup steps in github-app-setup.md; the UUIDs in restoration-guide.md are DataLad dataset ids next to public Zenodo DOIs, and one already appears in nemar-cli's public scripts/nemar-restore-dataset.sh). Both repos are public, so the text was already published.

    It is worse than a leak in one specific way: it is a false safety claim that invites one. Anyone reading that file believes the section is protected and writes accordingly. This epic's original phase 3 would have been the first victim, moving systems-inventory.md (hosts, paths, crons) behind a control that is not there.

    2. The gate will be NEMAR's own ORCID-backed login, not Cloudflare Access. Owner's call: one identity system for everything rather than a second one for the docs site.

    The constraint that makes this real work rather than a config flip: WEB_SESSION_COOKIE_DOMAIN = "app.nemar.org" (backend/wrangler-sccn.toml:148), scoped that narrowly on purpose so the session never rides along with data.nemar.org byte-range fetches or api.nemar.org search. A cookie scoped to one host cannot authenticate another, so docs.nemar.org needs a signed cross-host handoff, not cookie sharing. Widening the cookie to .nemar.org is not an option; it would undo the reason the scope exists.

    Phases

    Phase Scope Repos
    0 Gate docs.nemar.org/admin/* behind the ORCID-backed session with an admin-role check, verified by an anonymous fetch afterwards docs, nemar-cli backend
    1 ADR fixing the bucket rule, an exhaustive inventory of every agent-facing file, and the drift-guard test nemar-cli
    2 Move the publishable reference into the docs pages that already cover each topic nemar-cli, docs
    3 Move the operational runbooks behind the now-real gate nemar-cli, docs
    4 Rewrite AGENTS.md to hard rules, the ADR index pointer, and docs pointers nemar-cli

    The guard lands in phase 1, before any content moves, so every pointer added in phases 2 to 4 is validated as it lands.

    Settled, not open

    .rules/* (six files, 585 lines) stays in this repo untouched. Those are standards for editing this checkout, not facts about the platform, which is exactly the must-stay bucket. The drift guard reads them only to check any docs URLs they cite.

    Integration branch for this epic is dev, not main.

  3. neuromechanist commented on Sep 14, 2026

    @neuromechanist
    ContributorAuthor

    Merged to dev as e5ccda05, with per-phase history preserved.

    What this epic set out to do, and did

    docs.nemar.org is now the canonical surface for anything about the NEMAR platform. Repositories keep the rules that bind someone editing them and point at URLs for everything else. Twenty-three pull requests across seven repositories.

    phase issue outcome
    0 #1338 Admin docs gated behind NEMAR's own admin-only session
    1 #1339 Inventory, the classification rule, the docs-link drift guard
    2 #1377 Every docs page gets a Markdown mirror
    3 #1341 nemarOrg/docs private at source; runbooks moved in; nemar admin docs
    4 #1340 The four real documentation gaps closed before anything was deleted
    5 #1342 AGENTS.md 669 to 313 lines
    6 #1378 The pattern applied to six sibling repositories

    The premise changed twice, and both corrections were the valuable part

    Phase 0 found that Cloudflare Access was never protecting the admin docs. The application covered preview deployments only, so all twelve pages answered 200 anonymously and sat in the public sitemap. Nothing confidential leaked, but the claim invited someone to put a real secret behind a gate that was not there. A false safety claim is worse than a known gap.

    That broke the epic's own design. The original plan moved internal material into nemar-cli because the docs repo was public. But nemar-cli is public too, so the rule was a surface rule wearing the language of a secrecy control. ADR 0057 and ADR 0059 replaced it: one docs repo, private at source, public at the URL.

    Three defects found along the way, none of them this epic's subject

    • A rate-limit bypass on every authentication route. The API is mounted at / and /nemar, middleware sees the full path, so every AUTH_PATHS entry was unthrottled through the prefix. Long pre-existing.
    • A non-hermetic test that went red repo-wide. import-weekly-summary-sweep compared a SQL timestamp against an injected clock and passed only while the wall clock sat in the same ISO week as a hardcoded literal. Fixed on dev in Fix the weekly-summary audit-gate tests, red on every branch since 2026-09-14 #1385 because it blocked every branch.
    • nemar-tools was stale and is archived; its Zenodo flow had been superseded by EZID per ADR 0007.

    What the seam review caught before it shipped further

    Commissioned to hunt for content deleted in one repository that never landed in the other. That risk was clean -- all four runbook moves verified by structural inventory, zero loss. The real damage was elsewhere:

    • The docs session credential was written to the CLI debug log in plaintext, on the line after the long-lived key was correctly redacted, while two shipped comments claimed it could not happen.
    • unit-pure, a required check, failed on every fork pull request, because the private docs checkout needs a secret forks never receive. Exactly what ADR 0057 says must not happen.
    • Key revocation did not cascade into docs sessions, though the code explicitly warned a fifth credential path would need it.
    • Three accident-prevention rules were deleted by the AGENTS.md rewrite, including "revocation cascades", removed by the same change that violated it.

    The lesson worth keeping: checking the thing you are worried about is not the same as checking. The losses were findable only by inventorying every claim against three buckets -- survives here, exists on the docs site, or nowhere -- not by reading the diff.

    Calibration, recorded because it shaped the work

    This is an open-access project. The restrictions exist to protect resources, not knowledge. When the Zarr verification rules were restored they went on a public page, not behind the gate, because they explain results a data consumer can observe and disclose nothing that costs anything. Gating them by reflex would have been over-restriction.

    Follow-ups, not blockers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priority - important features, significant bugsagentsdocumentationImprovements or additions to documentationepic

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions