feat: strict cache mode and CHTYPES_CACHE_UNUSABLE (#486, part 2)

.github/workflows/v1.yml

@@ -694,11 +694,13 @@ jobs:
     # within its own line (public issue #481). Then 8 processes install one
     # build into one fresh cache at once, each binding alone and all four
     # mixed, with a watcher for torn, vanished or replaced entries (public
-    # issue #482). Then the read-only commands must create nothing, and a
-    # cache one uid writes at umask 022 must be read by another uid, `nobody`
-    # through `sudo -n` (public issue #486): the hosted runner has sudo, and
-    # the CLIs are installed under one world-readable prefix so `nobody` can
-    # run them. It fails if any cell fails, and prints the tables.
+    # issue #482). Then the read-only commands must create nothing, a cache
+    # one uid writes at umask 022 must be read by another uid, `nobody`
+    # through `sudo -n`, and every cache fault must get the same answer from
+    # every reader in the default mode and with --strict (public issue #486):
+    # the hosted runner has sudo, and the CLIs are installed under one
+    # world-readable prefix so `nobody` can run them. It fails if any cell
+    # fails, and prints the tables.
     runs-on: ubuntu-latest
     timeout-minutes: 30
     steps:
@@ -747,16 +749,21 @@ jobs:
           for b in go python ts rust; do sudo -n -u nobody "$X/bin/cli-$b" --version; done
       - name: "selftest — the interop driver's own comparison and table logic"
         run: python3 scripts/fetch-v1/cache-interop.py --selftest
-      - name: "4 x 4: each binding writes a cache, every binding reads it; the 0.x rows; writer x reader x line (#481); concurrent installs (#482); read-only commands and cross-uid reads (#486)"
+      # /opt/chtypes/v1 is one of the CLIs' default system dirs (spec/fetch-v1/constants.json);
+      # the fault rows fill it with a readable build to prove the default mode answers from it
+      # and strict mode never falls through to it, and leave it empty again.
+      - name: "4 x 4: each binding writes a cache, every binding reads it; the 0.x rows; writer x reader x line (#481); concurrent installs (#482); read-only commands, cross-uid reads and writer x reader x fault x mode (#486)"
         env:
           X: /tmp/cli-prefix
         run: |
+          sudo install -d -m 0755 -o "$(id -u)" -g "$(id -g)" /opt/chtypes/v1
           python3 scripts/fetch-v1/cache-interop.py \
             --cli "go=$X/bin/cli-go" \
             --cli "python=$X/bin/cli-python" \
             --cli "ts=$X/bin/cli-ts" \
             --cli "rust=$X/bin/cli-rust" \
-            --other-user nobody
+            --other-user nobody \
+            --system-dir /opt/chtypes/v1
 
   v1-refcheck:
     name: "v1-refcheck — reference verifiers agree with the fixture verdicts (report only)"

docs/guides/fetch-v1.md

@@ -18,6 +18,24 @@ A standard **OCI image layout** rooted at `${XDG_CACHE_HOME:-~/.cache}/chtypes/v
 - **Modes follow the umask.** Every directory and file the fetch layer creates is created with mode 0777 or 0666, and the process umask alone takes bits away, the temporary file or directory renamed into place included. At the usual umask 022 a cache one uid writes is readable by every other uid (a CI user and a container's nonroot user, for example), and readable is never writable: everything in the cache was downloaded anonymously and is signature-verified, and the library is re-hashed against its record at load. umask 077 keeps a private cache, and umask 002 with a setgid root gives a group-writable one. No binding has a mode option; the umask is the one knob (public issue #486).
 - The cache **can be pre-seeded** with `oras copy -r --to-oci-layout` (ORAS 1.3.4 and `--platform` are `measured` compatible with this layout, per the delivery side's design). A pre-seeded layout has entries in `index.json` with no corresponding `unpacked/` directory yet; the first request for one verifies it against its signature exactly as a freshly downloaded layer would, then unpacks it. Losing a pre-seed's `index.json` entry to a losing race costs only `oras` interop, never correctness — verification never trusts `index.json` content, only digests it then independently checks.
 
+### The cache faults
+
+Every fault the fetch layer can meet in a cache root is one of five kinds, and each binding answers each kind the same way (public issue #486):
+
+| kind                                                                                                                          | example                                   | default mode                                                                 | strict mode                                                                                |
+| ----------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
+| K0, a missing root                                                                                                            | a fresh volume path not yet created       | absent; a fetch creates it                                                   | the same: a missing root is the empty cache, not a fault                                   |
+| K1, an unusable root: the root or `unpacked/sha256/` exists but cannot be listed, or is not a directory                       | a mode 000 mount; the path is a file      | absent, plus a warning naming the path                                       | `CHTYPES_CACHE_UNUSABLE`, reason `unreadable_root` or `not_a_directory`                    |
+| K2, an unreadable entry: an entry directory or its `verified.json` exists, and opening it fails with anything but "not found" | an entry another uid wrote 0700 or 0600   | absent, plus a warning naming the path                                       | `CHTYPES_CACHE_UNUSABLE`, reason `unreadable_entry`                                        |
+| K3, an unacceptable record                                                                                                    | a garbage record; another schema          | absent: re-verified from the blobs and replaced, as below                    | the same, and only when there are no blobs to re-verify from, reason `unacceptable_record` |
+| K4, a 0.x layout: no `oci-layout`, no `unpacked/`, and a `<minor>/manifest.json`                                              | `CHTYPES_CACHE` pointed at a 0.x registry | absent, with the 0.x hint on the not-installed answer ("Upgrading from 0.x") | `CHTYPES_CACHE_UNUSABLE`, reason `layout_0x`                                               |
+
+- **A missing `oci-layout` is not K4 by itself.** Python writes none, and an `oras` pre-seed does. K4 needs the positive 0.x shape, and a root with `unpacked/sha256/` is a 1.x cache whoever wrote it.
+- **The default mode keeps "unreadable is absent"**, so a deployment that works today works the same. It gains one warning per K1 or K2 path, per call: `<path> could not be read (<errno>); treated as not installed. Set CHTYPES_CACHE_STRICT=1 to make this an error.` The warning is in `Resolved.warnings` when a build is returned, and in the not-installed answer's message when none is; the CLI's `list` and `verify` print it on stderr.
+- **Strict mode** is opt-in: Go `FetchOptions.StrictCache`, Python `FetchOptions(strict_cache=)`, TypeScript `FetchOptions.strictCache`, Rust `FetchOptions::strict_cache`, else `CHTYPES_CACHE_STRICT=1` (`0` is off), else off, and `--strict` on every CLI command. It checks the cache before any system directory is consulted, so a fault in the cache is raised and nothing falls through. A system directory that does not exist is skipped, as a default list is; one that exists but is unusable (K1 or K2) is an error too. It applies to `resolve_installed`, `list_installed`, `verify_installed` and `ensure` alike, and an online `ensure` checks the cache before it fetches into it. `chtypes where --strict` checks the root before printing it, and `chtypes verify --strict` that verified no build exits with `CHTYPES_ARTIFACT_MISSING`, so a mounted cache can be health-checked. Strict by default for an explicit cache is a later, separate decision.
+- **`CHTYPES_CACHE_UNUSABLE` (exit status 9, §8)** names the exact path, the reason and the errno name (`EACCES`), and its message is the same sentence in every binding: `<path> is unusable as a cache: <reason> (<errno>)`. Its sixth reason, `unwritable`, applies in every mode: a write the fetch layer needed (a directory, a temporary file, a rename) failed under the cache. That replaces what each binding raised before: Go a misuse error, Python a raw `OSError`, TypeScript a misuse error, Rust `CHTYPES_SOURCE_UNREACHABLE`.
+- The `v1-cache-interop` job faults every binding's cache seven ways (none, the root 000, `unpacked/sha256` 000, an entry 000, a record 000, a garbage record without blobs, a 0.x registry) and requires every binding, reading as another uid, to give the same answer in both modes, with and without a system directory holding the build.
+
 ### The `verified.json` record (schema 1)
 
 The schema is `spec/fetch-v1/schema/verified.schema.json`. The record is a JSON object with exactly these members, every one present, with `null` where shown:
@@ -60,7 +78,7 @@ The schema is `spec/fetch-v1/schema/verified.schema.json`. The record is a JSON
 Nothing about a 0.x cache may crash 1.0, including when something points the 1.0 root at it:
 
 - A `CHTYPES_CACHE` that names a 0.x registry directory (no `oci-layout`, no `index.json`) is an empty, foreign layout. A fetch proceeds normally and installs beside the old files without touching them. `--offline` and `resolve_installed` answer the ordinary not-installed code. Nothing is read as verified. (`upgrade-0x-registry-*`.)
-- That not-installed answer names the directory, in every binding and in the registry's own miss alike: `<root> holds a 0.x registry (<minor>/manifest.json); chtypes 1.x uses an OCI layout at ${XDG_CACHE_HOME:-~/.cache}/chtypes/v1 — point CHTYPES_CACHE at an empty or 1.x directory.` A `chtypes verify` of the directory prints the same sentence. The shape is positive: no `oci-layout`, no `unpacked/`, and at least one `<minor>/manifest.json` (the first by name). A missing `oci-layout` alone is not the shape, because a Python-written 1.x cache has none, and a directory with `unpacked/` is a 1.x cache whoever wrote it (public issue #486).
+- That not-installed answer names the directory, in every binding and in the registry's own miss alike: `<root> holds a 0.x registry (<minor>/manifest.json); chtypes 1.x uses an OCI layout at ${XDG_CACHE_HOME:-~/.cache}/chtypes/v1 — point CHTYPES_CACHE at an empty or 1.x directory.` A `chtypes verify` of the directory prints the same sentence. The shape is positive: no `oci-layout`, no `unpacked/`, and at least one `<minor>/manifest.json` (the first by name). A missing `oci-layout` alone is not the shape, because a Python-written 1.x cache has none, and a directory with `unpacked/` is a 1.x cache whoever wrote it (public issue #486). In strict mode the directory is `CHTYPES_CACHE_UNUSABLE` with reason `layout_0x` instead, for an online fetch too (§1, the cache faults).
 - An `unpacked/sha256/<hex>/` directory holding a 0.x-style `manifest.json` and no `verified.json` reads as absent. It is re-verified from blobs when they exist and reported not installed when they do not. (`upgrade-0x-unpacked-*`.)
 
 ## 2. Sources
@@ -137,7 +155,7 @@ Every one of the checks above runs **in this order**, and a failure at any step
 
 <!-- BEGIN GENERATED: fetch-v1 constants -->
 
-Generated from `spec/fetch-v1/constants.json` (sha256:`bf19d75118f2d10a1aac28e7e4e44022d59d36d7bbea2f9c7b0c327dcc5a73fc`) by `scripts/fetch-v1/gen-constants.py`; do not hand-edit between the markers. Covers this guide's §7 and §8, plus a quick reference for the media types, predicate types and default trust key introduced in earlier sections.
+Generated from `spec/fetch-v1/constants.json` (sha256:`163dbaa1e5173aa65de7d46f1eac27c02f862e96b5c8160a23fee8cfabd77a2a`) by `scripts/fetch-v1/gen-constants.py`; do not hand-edit between the markers. Covers this guide's §7 and §8, plus a quick reference for the media types, predicate types and default trust key introduced in earlier sections.
 
 ## Reference
 
@@ -191,12 +209,14 @@ A digest 404: `next_base_then_retry_on_last`. A tag 404: `next_base_then_unpubli
 | `CHTYPES_SOURCE_FORBIDDEN`      | 6           |
 | `CHTYPES_SOURCE_INCOMPATIBLE`   | 7           |
 | `CHTYPES_ARTIFACT_INCOMPATIBLE` | 8           |
+| `CHTYPES_CACHE_UNUSABLE`        | 9           |
 
-The six codes above the line in `spec/fetch-v1/constants.json` carry v0's own real, measured exit status (every binding's CLI agreed on 2026-10-01: the four "verification failed" codes all exit 1, `CHTYPES_ARTIFACT_UNPUBLISHED` exits 4, `CHTYPES_SOURCE_UNREACHABLE` exits 3). The four new v1-only codes have no v0 precedent and are assigned here, decided by lane 0A:
+The six codes above the line in `spec/fetch-v1/constants.json` carry v0's own real, measured exit status (every binding's CLI agreed on 2026-10-01: the four "verification failed" codes all exit 1, `CHTYPES_ARTIFACT_UNPUBLISHED` exits 4, `CHTYPES_SOURCE_UNREACHABLE` exits 3). The new v1-only codes have no v0 precedent and are assigned here: the first four by lane 0A, the last by public issue #486:
 
 - `CHTYPES_SOURCE_UNAUTHORIZED` and `CHTYPES_SOURCE_FORBIDDEN` — the fetch layer, a 401 or 403 from a source (§2, §5.1 of this guide).
 - `CHTYPES_SOURCE_INCOMPATIBLE` — the fetch layer, an unrecognized manifest or layer media type during resolve (§3).
 - `CHTYPES_ARTIFACT_INCOMPATIBLE` — the **FFI/loader layer**, never the fetch layer: a wrong `chs_abi_version`, an `abi_fingerprint` mismatch against the loaded library, a described symbol the library does not export, or a glibc floor the host does not meet (§9, the seam). It is reserved here, ahead of that lane landing, so the shared error vocabulary and exit-code table stay in one place while the ABI v1 design is still in flux. No v1 fetch-layer conformance case raises it — the fetch layer "never dlopens, checks glibc, or reads `chs_*` symbols" (§1.3 of the v1 fetch-layer plan).
+- `CHTYPES_CACHE_UNUSABLE` — the fetch layer, a cache directory or entry it could not read or write, or that strict mode refuses: it names the path and a reason (§1, the cache faults). It is the one code for the local filesystem, never the network's `CHTYPES_SOURCE_UNREACHABLE`, so a retry loop never retries a permission error.
 
 <!-- END GENERATED: fetch-v1 constants -->
 
@@ -332,7 +352,7 @@ The `v1` workflow's `v1-staging` job is the first fetch over real HTTPS through
 ### Known gaps (report these against this guide, not against a case's `ok`/`code`)
 
 - **`download-token-sent`, `manifest-accept-header`:** `cases.schema.json` has no field for "this header must have been sent" — only `requests.none_matching` (absence). Both cases are enforced by `server.py` itself refusing (400/second 401) when the required header is missing or wrong, which is why they have no declarative `http_script` response sequence doing the real work; see `server.py`'s own header comment.
-- **`spelling-refused`/`spelling-refused-v-prefix`:** refused before any network call, by a client-side regex check, which is not one of the ten shared error codes (`errors`, above) — `expect.code` is `null` for these two; `expect.requests.max: 0` is the real assertion.
+- **`spelling-refused`/`spelling-refused-v-prefix`:** refused before any network call, by a client-side regex check, which is not one of the shared error codes (`errors`, above) — `expect.code` is `null` for these two; `expect.requests.max: 0` is the real assertion.
 - **The `hint` field of every bundle this repository signs is our own keyid** (the `sha256-first16hex` scheme every binding already computes for v0), never a raw-key encoding. Nothing in layout-v2 spec §4.2 fixes the hint's content beyond "a public-key hint" — none of this repository's four hand-rolled verifiers select a key BY the hint (they try every trusted key), so the format is this generator's own choice, recorded here so a reference-verifier integration (`scripts/fetch-v1/refcheck/`) keys its own trust mapping by the same string.
 - **`scripts/fetch-v1/refcheck/` is a representative sample, not an exhaustive cross-check.** It verifies a handful of known-good bundles (one per predicate type this repository signs anything with) against sigstore-go, `@sigstore/verify` and `cosign verify-blob-attestation`, report-only. The plan's broader vision — cross-checking every fixture's trust verdict against all three reference verifiers — is future work, not this lane's acceptance bar.
 - **sigstore-go: the delivery side's design doc cites version "1.3.0"; that version does not exist on the Go module proxy as of 2026-10-01** (the newest published tag is v1.2.2, which `refcheck/go` pins and which verifies this repository's bundles). Recorded here rather than silently substituted.