Repository navigation
Reorganize AGENTS.md into a map, add a systems inventory - #1107
Merged
Merged
Conversation
AGENTS.md named hosts and crons in passing but never documented the Zarr conversion host at all, and pointed at the wrong hallu-sync.sh path: cron runs the ~/.local/share clone, not the /data/qumulo one, and the two were 300 commits apart. Records what was verified on the live hosts: the three Hallu crons with their real paths and logs, the Zarr state directory on local NVMe, and the two-half deploy for the converter -- the Python driver self-updates from origin/main, hallu-zarr.sh is hand-placed and needs an atomic rename. Also records why merging a converter change is not deploying it: setup() runs once per process and the drain loop holds the lock until the queue empties, so a long run pins the driver and the biosigio version to whatever main was at process start. An instance found running for 8d19h was converting on the #90 driver while main was at #101. Drops two hardcoded counts from the .context index that had already gone stale (21 ADRs, 28 documents).
590 lines had accumulated into a manual: an exhaustive CLI command list that duplicates --help, five validated-workflow recipes that are a strict subset of .context/validated_workflows.md (which holds eight), and full release mechanics. A reader could not find the load-bearing rules in it. Now it opens with a where-to-look table, states the hard rules in full (live datasets, dev D1 sharing prod users and the GitHub org, never hand-bump, work in a worktree, ID bands and the exemplar fleet), then names each part of the system in a line or two with a pointer. Nothing was dropped without a home: the recipes were already covered, commands defer to --help, and hosts/crons/deploys move to the new .context/systems-inventory.md. Adds ADR 0027 and 0028 to the load-bearing list and fixes the auth route path (auth-web.ts, not a routes/auth/ directory). 347 lines.
📦 Package PreviewYour package is ready to test! bun install -g nemar-cli@PR1107Version: 🤖 Updated on each push. |
Review of #1107 found the Zarr claim wrong in a way that would mislead anyone debugging a derivatives-tree recording: 'never derivatives, sourcedata, or code' reads as a description of the bucket, when it only describes what becomes a NEW store. Non-raw stores published before #98 are still served until the authorized purge finishes. Also cross-references ADR 0023, since 'rebuilt from scratch' invited exactly the prefix-wipe reading that ADR removed. ADR 0027's Consequences said the converter did not yet apply the exclusion. True when written, not now: #98 landed it with the compute_clean_orphans guard the ADR said it would need, and #99 landed the purge. Appended a dated implementation-status section rather than editing the original text, and corrected its Receipts measurement, which counted 200 of 754 datasets because GET /datasets caps limit at 200. Restores specifics the reorg dropped with no home: the ADR 0022 pointer and migration 0066 for web auth, the seed-dev-db.sql origin of TEST_ADMIN_API_KEY, nm099999's lazy creation, epic #837's severed couplings, and #655/#794 on the backup section. Left out deliberately: the e2e 10-step enumeration, since src/lib/e2e-test.ts is the authority and duplicating it is what this PR removes.
Merged
5 tasks done
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AGENTS.md had grown to 590 lines and become a manual rather than an entry point. It also carried a wrong path: the documented
hallu-sync.shlocation is a stale clone that cron does not run, and the Zarr conversion host was not documented at all.AGENTS.md (590 -> 347 lines) now opens with a where-to-look table, states the hard rules in full, then names each part of the system in a line or two with a pointer to the detail.
Nothing was dropped without a home:
.context/validated_workflows.mdalready held all five, plus three morenemar <group> --help, which is authoritative; AGENTS.md keeps a group-level table.context/systems-inventory.md.context/release-safety-playbook.mdbackend/src/routes/auth-web.tsSafety content was kept in full and made more prominent under a single
Hard rulesheading: live datasetsnm000103-nm000107, dev D1 sharing production users and thenemarDatasetsorg, production-only-by-default crons, never hand-bumping the version, and the dataset ID bands with the exemplar fleet. Added the worktree rule, which was not written down anywhere.New:
.context/systems-inventory.mdVerified against the live hosts, not written from memory. Covers cloud services and their prod/staging split, the two GitHub orgs, SDSC Hallu, the test machines, and disaster recovery.
Three things it records that were not documented anywhere:
~/.local/share/nemar-cli/scripts/hallu-sync.sh. A second clone at/data/qumulo/openneuro/nemar-cliis not what cron executes, and the two are 300 commits apart. AGENTS.md named the stale one.30 * * * *cron, the state directory on local NVMe, the queue, the venv, the driver clone, and the two-half deploy: the Python driver self-updates fromorigin/main, whilehallu-zarr.shis hand-placed and needs an atomic rename because bash reads a script incrementally.setup()runs once per process, and the drain loop holds the lock until the queue empties, so a long run pins both the driver and the biosigio version to whateverorigin/mainwas at process start. An instance found running for 8d19h was converting on the#90driver with biosigio 1.2.2 whileorigin/mainhad reached#101and biosigio 1.2.4.Also drops two hardcoded counts from the
.context/index that had already gone stale.Test plan
bun test test/adr-index.unit.test.tspasses (8 pass); its docstring depends on theSTART HEREsection, which is preservedbackend/src/routes/auth/toauth-web.tsnemar doctor's tool list checked againstsrc/commands/doctor.tssrc/commands/.context/validated_workflows.mdis a strict superset of the recipes removed