diff --git a/README.md b/README.md
index a38efd6f117..d20644f2460 100644
--- a/README.md
+++ b/README.md
@@ -91,6 +91,8 @@ decision and its measurements live (`docs/maxusai/`).
 |---|---|---|---|
 | **MLX admission** | weights against free device memory (and, since v0.34.1, a system-memory bound on integrated GPUs) | weights + KV priced at the requested `num_ctx` + a per-architecture headroom; an explicit rung that does not fit is refused, an automatic one is clamped | ADR 0034 |
 | **MLX memory ceiling** | none | `OLLAMA_MLX_MEMORY_LIMIT` and a cache limit, set per runner from the admitted budget | runner knobs |
+| **nvfp4 global scales** | — | held in MLX's `m × 2688` representation and divided back out by every wrapper that applies the scale itself, which is not the identity in float32 (17 of 31b's 191 vision scales move one ulp) | ADR 0039 (proposed) |
+| **model identity in a record** | a tag | the manifest digest: the library re-published `gemma4:*-nvfp4` with bf16 vision towers under unchanged tags and config blobs | ADR 0038 (proposed) |
 | **gemma4 image chunk vs. generation batch (GGUF)** | the batch follows `num_ctx` (1024 above 4096), so a top-rung gemma4 image (up to 1120 tokens) is decoded in two pieces, bidirectional only within each | a gemma4 vision runner starts from the batch rung that holds its image ceiling (2048 at 1120) and steps down only when it does not fit | ADR 0036 |
 | **gemma4 on MLX** | upstream's own vision and audio tower with a fixed per-checkpoint soft-token set, no per-request budget | vision through upstream's `MediaModel` with a per-request budget seam; audio not shipped | ADR 0021 |
 | **media prompts on MLX** | — | prefill chunks span-aligned around image blocks; a late image is refused | ADR 0014 |
diff --git a/docs/maxusai/adr/0038-a-model-is-identified-by-its-manifest-digest.md b/docs/maxusai/adr/0038-a-model-is-identified-by-its-manifest-digest.md
new file mode 100644
index 00000000000..046055625a4
--- /dev/null
+++ b/docs/maxusai/adr/0038-a-model-is-identified-by-its-manifest-digest.md
@@ -0,0 +1,81 @@
+# ADR 0038: a measured model is identified by its manifest digest, not its tag
+
+- **Status:** proposed 2026-09-19. Awaiting Glenn.
+- **Date:** 2026-09-19
+- **Deciders:** MaxusAI fork maintainers
+
+## Context
+
+`gemma4:31b-nvfp4` in this store and `gemma4:31b-nvfp4` in the library are not the same
+model. Measured 2026-09-18 by diffing the manifests: **194 of 1248 layers differ, every
+one of them on the vision path**, each about 3.5× larger upstream — `nvfp4` (4 bits plus
+scales) widened to `bf16`. The library re-published the tag with an unquantised vision
+tower. Our copy was pulled 2026-08-17; the re-publish is after that date.
+
+The same move hit `gemma4:26b-nvfp4` (190 of 1072 layers) and `gemma4:12b-nvfp4` (its six
+vision and audio projections), and `qwen3.5:0.8b-mlx` was re-published with its **tensors
+renamed** (473 of 481 layers). 39 of the store's 54 tags are byte-identical to the
+registry; 10 are local builds the registry never served.
+
+Three properties make this invisible:
+
+- the **config blob digest is unchanged** in every case, so `ollama show` reports the
+  same parameters, template and licence for both artifacts;
+- the pull is idempotent by tag, not by content, so `ollama pull` on a host that already
+  has the old copy *replaces* it without a word;
+- nothing in the store records where a blob came from or when, beyond file mtimes.
+
+The consequence is not hypothetical. Every gemma4 vision result in this repo — the #312
+encoder investigation, `x/mlxrunner/testdata`'s vision goldens, the fine-text tiers, the
+OCRBench ladder's first row — was measured against the **quantised** tower. A host that
+pulls that tag today gets the bf16 one and will not reproduce any of them, and the old
+artifact **cannot be fetched back**: the registry no longer serves it under any name.
+
+This also decides what a cross-host comparison means. The Metal host's OCRBench number
+and the CUDA host's are comparable only if both ran the same tower precision, and neither
+number carried anything that could answer that.
+
+## Decision
+
+**A record that names a model names its manifest digest.** The tag is how a human finds
+the model; the digest is what the number describes.
+
+1. Any document, issue comment or score file that reports a measurement states the
+   manifest digest of each model measured. `vision-suite/store_audit.py --digests` prints
+   them.
+2. Before trusting a comparison between two hosts, run `store_audit.py` on both. A tag
+   present on both hosts is not evidence they hold the same weights.
+3. Benchmark and campaign pulls go to a **separate store** (a directory mounted at
+   `/root/.ollama`), never the production store, so a pull cannot overwrite an artifact
+   an existing measurement was taken against.
+4. Do not `ollama pull` a tag this repo has measured into the production store. If a
+   newer artifact is wanted, pull it beside the old one and record both digests.
+5. The store is the archive. A measured artifact that is deleted locally is gone, because
+   the registry serves only the current content of a tag.
+
+## Options considered
+
+- **Cite tags, as before.** Cheapest, and what every record did until now. Rejected: the
+  failure is silent, the artifact is unrecoverable, and the first symptom is an
+  irreproducible result on someone else's host.
+- **Pin by the model's config digest.** Rejected on the evidence: the config digest was
+  identical across the re-publish in all four cases, so it identifies the model's
+  definition, not its weights.
+- **Snapshot every measured model to the array.** Complete, and about 19 GB per gemma4
+  nvfp4 tag. Worth doing for artifacts a published result depends on; not required by
+  this ADR, which only makes the identity recordable.
+- **Ask upstream to publish immutable tags.** Out of our hands, and nothing goes to
+  `ollama/ollama` by standing instruction.
+
+## Consequences
+
+- A number without a digest is now an incomplete record. Existing documents are not
+  retro-fitted wholesale; the digest is added when a document is next touched, and
+  `docs/maxusai/ocrbench-quantisation-ladder.md` carries them from the start.
+- The vision suite and preflight do not yet stamp digests into their score files. They
+  should, and that is the natural follow-up — the harness already has the store path.
+- Re-pulling to "get the latest" becomes a deliberate act with a recorded before and
+  after, not routine hygiene.
+- This says nothing about which artifact is *better*. The bf16 tower may well be the
+  better model; see the OCRBench ladder, where it is 39 % faster per image on CUDA and
+  scores within noise.
diff --git a/docs/maxusai/adr/0039-nvfp4-global-scales-are-stored-as-the-checkpoint-multiplier.md b/docs/maxusai/adr/0039-nvfp4-global-scales-are-stored-as-the-checkpoint-multiplier.md
new file mode 100644
index 00000000000..d36b35425ce
--- /dev/null
+++ b/docs/maxusai/adr/0039-nvfp4-global-scales-are-stored-as-the-checkpoint-multiplier.md
@@ -0,0 +1,100 @@
+# ADR 0039: nvfp4 global scales are stored as the checkpoint multiplier
+
+- **Status:** proposed 2026-09-19. Awaiting Glenn. Arises from
+  [#312](https://github.com/MaxusAI/ollama/issues/312) and gates part of
+  [#287](https://github.com/MaxusAI/ollama/pull/287).
+- **Date:** 2026-09-19
+- **Deciders:** MaxusAI fork maintainers
+
+## Context
+
+An nvfp4 checkpoint carries a per-tensor multiplier `m` (ModelOpt's `weight_scale_2`).
+MLX wants that scale in its own representation, `m × 448 × 6` — `Nvfp4MaxProduct`, 2688 —
+because its kernels fold the maxima of the E4M3 and E2M1 ranges into the scale.
+
+The v0.34 fold made that representation the **stored** form.
+`ToMLXGlobalScale` (`x/mlxrunner/model/quant.go`) converts once at load:
+
+```go
+return mlx.MulScalar(flat, mlx.Nvfp4MaxProduct)   // holds f32(m × 2688)
+```
+
+and every wrapper that applies the scale itself divides it back out, in `scaleAndCast`
+(`x/mlxrunner/mlx/ops_extra.go`):
+
+```go
+return Mul(out, DivScalar(scale, Nvfp4MaxProduct)).AsType(out.DType())
+```
+
+**`f32(f32(m × 2688) / 2688)` is not `m`.** Checked against the checkpoints in #312: it
+returns `m` exactly for both of 12b's vision global scales and misses by one float32 ulp
+for **17 of 31b's 191**. One ulp in a per-tensor scale flips bf16 roundings of that
+layer's output, and 27 encoder layers grow it into something the golden test sees: on
+MLX-CUDA the 31b vision encoder moved `max sampled Δ 0.0898 → 0.1094` and
+`norm_mean 96.999 → 97.056` across the fold. A two-line control on `8a7ba949` that
+restored the old arithmetic reproduced the pre-fold line **to every digit**, so the round
+trip is the whole of the CUDA-side move.
+
+Three wrappers apply the scale themselves, and all three take the divide:
+
+| wrapper | why it applies the scale itself | reached by |
+|---|---|---|
+| `QuantizedMatmul` | `mlx_quantized_matmul` has no global-scale argument | every dense nvfp4 linear — the whole vision tower and the dense language model |
+| `Dequantize` | MLX's argument accepts only a scalar; callers pass per-expert banks | weight inspection, and #287's prefill dequantisation |
+| `GatherQMM` off Metal, with `rhsIndices` | the CUDA path scales gathered rows in the wrapper | nvfp4 MoE experts |
+
+Two paths hand the scale to MLX and genuinely need the `× 2688` form: `GatherQMM` on
+Metal (or without `rhsIndices`), which passes `gs` into `mlx_gather_qmm`, and the fused
+`QQMM` path, whose identity scale is `Nvfp4MaxProduct` itself.
+
+So the stored representation is chosen for the two consumers that want it, and the three
+that do not want it pay a lossy round trip on every call.
+
+## Decision
+
+**Store `m`. Convert to MLX's representation at the call sites that hand the scale to
+MLX, and nowhere else.**
+
+- `ToMLXGlobalScale` keeps the checkpoint multiplier (cast to float32, flattened) and
+  stops multiplying by `Nvfp4MaxProduct`.
+- `scaleAndCast` multiplies by the scale as given and stops dividing.
+- `GatherQMM`'s native branch and `QQMM` multiply by `Nvfp4MaxProduct` where they build
+  the argument MLX consumes; the identity scale stays as it is.
+- A test asserts the round trip: for every global scale in a served checkpoint, the value
+  a wrapper applies is bit-identical to the checkpoint's `m`.
+
+## Options considered
+
+- **Leave it.** The error is one ulp and every bound the golden test checks still passes.
+  Rejected: it is a silent, permanent divergence from the reference implementation on
+  every dense nvfp4 linear, it made a pin move look like an encoder regression once
+  already (#312), and it costs nothing to remove.
+- **Restore the pre-fold arithmetic everywhere** (drop `× 2688` entirely). This is the
+  control used in #312 and it does reproduce the old numbers, but it breaks the MoE paths
+  that hand the scale to MLX. Rejected.
+- **Store both forms.** Explicit, and two arrays per scale plus a rule about which to
+  reach for. Rejected as more surface than the problem warrants: the conversion is one
+  multiply at two call sites.
+- **Keep the MLX form and make `scaleAndCast` exact** by folding `1/2688` into the
+  multiply as a double. Rejected: it fixes the arithmetic at the point of use while
+  leaving a stored value that is not what any checkpoint says, so the next wrapper added
+  inherits the trap.
+
+## Consequences
+
+- On CUDA the vision encoder returns to bit-parity with the pre-fold build and with
+  `mlx-vlm`, which is what the goldens in `x/mlxrunner/testdata` were generated against.
+  The 31b golden expectations move back (`max Δ 0.1094 → 0.0898`) and should be re-taken
+  in the same change.
+- **#287 inherits the fix.** Its prefill path dequantises nvfp4 weights to bf16 through
+  `Dequantize`, so today it would bake the ulp error into the dequantised weights for the
+  whole prefill. With this decision, dequantised prefill and quantised decode apply the
+  same scale, which removes one objection to that PR. It does not address the others: the
+  PR still changes output deterministically and costs +6.8 GiB peak, and it stays held on
+  those grounds.
+- Metal is unaffected in behaviour: its `GatherQMM` still receives `m × 2688`. Its 26b and
+  31b encoder numbers move for the unrelated reason in #312 (mlx#3912), and this change
+  does not interact with that.
+- One risk, and it is the reason for the test: a call site that hands the scale to MLX and
+  is missed would silently apply a scale 2688× too small. The failure is loud in output
+  quality, not in a crash.
diff --git a/docs/maxusai/ocrbench-quantisation-ladder.md b/docs/maxusai/ocrbench-quantisation-ladder.md
new file mode 100644
index 00000000000..c71321da56f
--- /dev/null
+++ b/docs/maxusai/ocrbench-quantisation-ladder.md
@@ -0,0 +1,368 @@
+# OCRBench across the quantisation ladder: gemma4:31b
+
+**Status: open, numbers landing.** The `mlx-cuda` and `llama.cpp` rows are measured on the
+CUDA host; the `mlx-metal` section is empty on purpose — it is for the Metal session to
+fill from its own host. Nothing here mixes the two platforms into one table
+([ADR 0012]; a missing same-platform baseline is stated, never substituted).
+
+Glenn's ask, 2026-09-18: the Metal host scored an OCRBench slice on the 0.34.0 fixed
+kernel — 175 correct of 200, 0.875 — so run the same slice on `mlx-cuda` and on the
+llama.cpp GGUF path across the ladder (`nvfp4`, `mxfp8`, `bf16` on MLX; `q4_K_M`,
+`q8_0`, `bf16` on GGUF) and compare.
+
+---
+
+## Read this first: a published tag is not a fixed artifact
+
+`gemma4:31b-nvfp4` in this store and `gemma4:31b-nvfp4` in the library today are
+**different models**. Same name, same config blob, 1248 layers of which 194 carry
+different weights — all of them on the vision path, and each about 3.5× larger upstream,
+which is exactly `nvfp4` (4 bits plus scales) widening to `bf16`. The library
+re-published the tag with an **unquantised vision tower**; our copy still has the
+quantised one.
+
+`ollama show` reports neither the layer set nor a content digest, so nothing surfaces
+the swap. The first symptom would have been a measurement that did not reproduce on a
+host that pulled later.
+
+Audit of this store against the registry, from `vision-suite/store_audit.py`:
+
+| tag | local manifest | registry manifest | layers changed | where | size ratio | reading |
+|---|---|---|---|---|---|---|
+| `gemma4:12b-nvfp4` | `117d0d84cf2a` | `ded7a2735003` | 6 / 737 | vision 2, other 2, audio 1, blob 1 | 2.4× | — |
+| `gemma4:26b-nvfp4` | `c8656f50f0a6` | `f0fc7e0ae494` | 190 / 1072 | vision 189, blob 1 | 3.55× | nvfp4 → bf16 |
+| `gemma4:31b-nvfp4` | `637cc0ff1570` | `a22a363052da` | 194 / 1248 | vision 191, other 2, blob 1 | 3.52× | nvfp4 → bf16 |
+| `qwen3.5:0.8b-mlx` | `6a48dd9c06e3` | `12f40a7f8473` | 473 / 481 | language 284, vision 171, audio 18 | — | tensors renamed |
+
+39 of the store's 54 tags are byte-identical to the registry, 4 changed, 10 are local
+builds the registry never served. The three gemma4 `nvfp4` tags all moved the same way:
+the vision path is no longer quantised. On `qwen3.5:0.8b-mlx` the tensor *names* changed
+too, so almost every layer differs and no size ratio is meaningful.
+
+**What this costs us.** Every gemma4 vision measurement in this repo that names
+`gemma4:*-nvfp4` — the #312 encoder work, the vision goldens in
+`x/mlxrunner/testdata`, the fine-text tiers, this ladder — was taken against a
+**quantised tower**. A host that pulls that tag today gets a bf16 tower and will not
+reproduce them. Production is unaffected: `:11497` serves from the store as it stands,
+and nothing re-pulls on its own.
+
+**What to do about it.**
+
+1. Cite the manifest digest, not the tag, in anything that records a number.
+   `python3 vision-suite/store_audit.py --digests gemma4:31b` prints them. This is now
+   [ADR 0038](adr/0038-a-model-is-identified-by-its-manifest-digest.md) and SPEC
+   `vision-harness-reuse` H15.
+2. Run the audit before trusting a cross-host comparison — the other host may hold a
+   different artifact under the same name. That is the first thing the Metal session
+   should run below.
+3. Do not `ollama pull` a tag this repo measures without checking the audit first. A
+   pull is silent and there is no local history to roll back to.
+
+There is a second reading, and it is the interesting one: upstream moved gemma4's
+vision path off `nvfp4` at some point after we pulled. That is the same direction as
+[#312](https://github.com/MaxusAI/ollama/issues/312), where the Metal `fp_qmm_t` bug
+(`K % 32 == 16`, which the tower's `mlp.down_proj` at K = 4304 hits) corrupted exactly
+those weights. Whether upstream did it for that reason or for quality, a bf16 tower
+sidesteps the whole class of problem.
+
+---
+
+## Which checkpoint produced the 0.860, and when the Metal number is comparable
+
+The `mlx-cuda` 0.860 came from **this store's `gemma4:31b-nvfp4`, manifest
+`sha256:637cc0ff15709212de4aa694be67e3af5e80533ca538c86ad69c53c511e40840`, pulled
+2026-08-17** (blob mtimes; the manifest was written 00:57 that morning). Its vision tower
+is **4-bit**: `model.vision_tower.encoder.layers.*.mlp.down_proj.linear.weight` is
+2.79 MB per layer, which is nvfp4 for a [1152 × 4304] tensor. bf16 would be 9.92 MB, and
+that is what the library serves under the same tag today
+(`sha256:a22a363052da…`, the separate row in the table below).
+
+**So the Metal 0.875 is comparable to the 0.860 only if the Metal host also holds a
+4-bit tower.** The check is one command on that host:
+
+```bash
+python3 docs/maxusai/vision-suite/store_audit.py gemma4
+```
+
+- `gemma4:31b-nvfp4` listed as **changed, vision ~191, 3.5× — nvfp4 → bf16**: the host
+  holds the 4-bit-tower artifact, the same class as ours, and the numbers are comparable.
+  `--digests` then gives the exact digest to record; if it is `637cc0ff1570…` it is
+  byte-identical to ours.
+- `gemma4:31b-nvfp4` **not listed** (identical to the registry): the host holds the
+  bf16-tower artifact, and its number belongs on the `nvfp4 / bf16 tower` row instead —
+  ours scored 0.850 there, not 0.860.
+
+A host that pulled after the re-publish cannot get the 4-bit tower back: the registry
+serves only the current content, and no other tag carries it. If the Metal host still has
+it, that copy is an archive worth keeping — see [ADR 0038](adr/0038-a-model-is-identified-by-its-manifest-digest.md).
+
+## What is measured
+
+The repo's own [`vision-suite/extbench.py`](vision-suite/extbench.py), the same tool and
+the same scoring the Metal number came from: OCRBench's `echo840/OCRBench` test split,
+rows 0–200, gold answer contained in the prediction (lmms-eval's `ocrbench` semantics,
+including its whitespace-stripped handwritten-maths case).
+
+```bash
+LIMIT=200 OFFSET=0 THINK=false NUM_CTX=8192 NUM_PREDICT=512 SLEEP=1 TIMEOUT=1800 \
+  python3 docs/maxusai/vision-suite/extbench.py http://127.0.0.1:11521 \
+    <tag> gemma4:31b-nvfp4 ocrbench
+```
+
+| setting | value | why |
+|---|---|---|
+| items | 200, offset 0 | the slice the Metal run used — see the warning below |
+| think | off | OCRBench answers are a word or a phrase |
+| sampling | temperature 0, `apply_sampling=False` | the arms must differ by weights, not by sampling |
+| `num_ctx` | 8192, every arm | one image and a short question; identical across arms so no arm truncates |
+| endpoint | `/api/generate` | what the published slices used |
+| loaded models | one at a time, unloaded between arms | GPU0 is shared; a resident model skews the next arm |
+| engine | the deployed build `0.34.1-dynres-16-g16649e8` | GGUF arms therefore decode the image in one batch (ADR 0036) |
+
+Each arm runs in a probe container on port 11521 against GPU0 with a 16 GiB overhead
+reserve, never against `:11497`.
+
+**What rows 0–200 actually contain.** The dataset is ordered by task, so this slice is
+**not a sample of OCRBench** — it is four of its ten categories, 50 items each:
+
+| Regular Text Recognition | Irregular Text Recognition | Artistic Text Recognition | Handwriting Recognition |
+|---|---|---|---|
+| 50 | 50 | 50 | 50 |
+
+Digit-string, non-semantic text, scene-text VQA, doc-oriented VQA, key-information
+extraction and handwritten maths are **absent**. So every number here — ours and the
+Metal 0.875 — is an accuracy on OCRBench's *text-recognition half*, and must not be
+called an OCRBench score or compared with a published one (those are out of 1000 across
+all ten categories). It is still the right slice for this question, because the arms are
+being compared with each other on identical items. Widening to the full 1000, or to a
+stratified 20-per-category slice, is the obvious follow-up and costs about 80 minutes per
+arm at these rates.
+
+**The ladder varies one thing at a time — but only within an engine.** The library's
+three MLX tags (`31b-nvfp4`, `31b-mxfp8`, `31b-mlx-bf16`) carry an **identical bf16
+vision tower** and differ only in the language model. The three GGUF tags
+(`31b-it-q4_K_M`, `31b-it-q8_0`, `31b-it-bf16`) carry an **F16 tower** in all three and
+differ only in the language model — verified from the GGUF tensor types: 191 `v.*`
+tensors at F16 and 165 at F32 in both `q4_K_M` and `q8_0`. Our local
+`gemma4:31b-nvfp4`, the one the Metal comparison is against, is the exception: its tower
+is nvfp4 too. It is reported as its own row, because it differs from the library's
+`nvfp4` in the tower and not in the language model.
+
+---
+
+## Results — `mlx-cuda` (RTX PRO 6000 Blackwell, sm_120)
+
+<!-- GENERATED by vision-suite/ocrbench_table.py; re-render, do not hand-edit. -->
+
+| arm | engine | model | correct / scored | accuracy | ±1 s.e. | mean s/item | median | prompt_eval |
+|---|---|---|---|---|---|---|---|---|
+| nvfp4 / nvfp4 tower (run 1) | mlx-cuda | `gemma4:31b-nvfp4` | 172 / 200 | **0.860** | 0.025 | 2.3 | 2.0 | 1115 |
+| nvfp4 / nvfp4 tower (run 2) | mlx-cuda | `gemma4:31b-nvfp4` | 172 / 200 | **0.860** | 0.025 | 2.2 | 2.0 | 1115 |
+| nvfp4 / bf16 tower (run 1) | mlx-cuda | `gemma4:31b-nvfp4` (library) | 170 / 200 | **0.850** | 0.025 | 1.4 | 1.4 | 1115 |
+| nvfp4 / bf16 tower (run 2) | mlx-cuda | `gemma4:31b-nvfp4` (library) | 170 / 200 | **0.850** | 0.025 | 1.5 | 1.5 | 1115 |
+| mxfp8 / bf16 tower (run 1) | mlx-cuda | `gemma4:31b-mxfp8` | 169 / 200 | **0.845** | 0.026 | 1.3 | 1.2 | 1115 |
+| mxfp8 / bf16 tower (run 2) | mlx-cuda | `gemma4:31b-mxfp8` | 169 / 200 | **0.845** | 0.026 | 1.3 | 1.2 | 1115 |
+| bf16 | mlx-cuda | `gemma4:31b-mlx-bf16` | **not run** — refused admission, see below | | | | | |
+
+**Repeats**
+
+| arm | runs | accuracies | items that flipped |
+|---|---|---|---|
+| nvfp4 / nvfp4 tower | 2 | 0.860, 0.860 | 0 |
+| nvfp4 / bf16 tower | 2 | 0.850, 0.850 | 0 |
+| mxfp8 / bf16 tower | 2 | 0.845, 0.845 | 0 |
+
+Not one item changed verdict between runs on any MLX arm either. On this workload — one
+image, a short answer, temperature 0, no drafting under a grammar — the engine's
+run-to-run movement does not appear at all, which is a stronger control than
+[ADR 0012] §4's caveat led us to expect.
+
+**Paired**
+
+| A | B | A | B | b (A only) | c (B only) | p | resolved |
+|---|---|---|---|---|---|---|---|
+| nvfp4 / nvfp4 tower | nvfp4 / bf16 tower | 0.860 | 0.850 | 3 | 1 | 0.625 | no |
+| nvfp4 / nvfp4 tower | mxfp8 / bf16 tower | 0.860 | 0.845 | 4 | 1 | 0.375 | no |
+| nvfp4 / bf16 tower | mxfp8 / bf16 tower | 0.850 | 0.845 | 1 | 0 | 1.000 | no |
+
+**The MLX bf16 arm did not run: the scheduler refused it, and the reserve is why.**
+Verbatim from the runner, 2026-09-19 07:28:
+
+```
+gpu memory id=0 library=CUDA available="62.7 GiB" free="79.2 GiB" minimum="457.0 MiB" overhead="16.0 GiB"
+Load failed error="model requires 75.0 GiB (weights 59.1 GiB + KV cache 1.4 GiB at num_ctx 8192
++ 14.5 GiB headroom) but only 62.7 GiB are available (after 16.4 GiB overhead)"
+```
+
+Other tenants held 15.8 GiB of the card, and the 16 GiB reserve ([ADR 0034](adr/0034-mlx-admission-prices-the-context-rung.md)
+prices MLX admission as weights + KV + a per-architecture headroom) takes the budget to
+62.7 GiB against a 75.0 GiB ask — short by 12.3 GiB. Lowering `num_ctx` does not close it:
+KV is 1.4 GiB of the 75. It was not forced, because dropping the reserve on a shared card
+to fit one benchmark arm trades other people's work for a row in this table.
+
+It is not impossible, only not now: with the card otherwise idle the same ask is 75.0 GiB
+against about 78.6 GiB available **with the reserve intact**, so the arm runs whenever
+GPU0 is quiet. The GGUF bf16 arm did run, at the same 58 GiB of weights, because
+llama.cpp's admission prices weights + KV without MLX's headroom term — which is worth
+noting on its own: two engines, the same model size, one admitted and one refused.
+
+**Quantising the vision tower costs 39 % of the time per image and buys nothing here.**
+The two `nvfp4` rows share a language model and differ only in the tower: ours is nvfp4,
+the library's is bf16, and the bf16 one answers in 1.4 s against 2.3 s on identical
+images at an identical 1115-token mean prefill. The bf16 tower is 3.5× the bytes, so this
+is not weight bandwidth — the tower is about half a gigabyte either way. It is kernel
+throughput: the image encode is compute-bound, and MLX's quantised matmul on sm_120 runs
+at roughly half the rate of the dense bf16 path, which is what
+[`sm120-mixed-input-gemm`](../../x/mlxrunner/bench/qqmm) measured directly. Accuracy does
+not move with it: three items separate the two towers, p = 0.625.
+
+Language-model precision does not move accuracy either — `nvfp4` and `mxfp8` over the
+same bf16 tower differ on a single item out of 200, at 1.4 s against 1.3 s.
+
+**By question type** (first run of each arm)
+
+| question type | n | nvfp4 / nvfp4 tower | nvfp4 / bf16 tower | mxfp8 / bf16 tower |
+|---|---|---|---|---|
+| Artistic Text Recognition | 50 | 49/50 | 49/50 | 48/50 |
+| Handwriting Recognition | 50 | 34/50 | 33/50 | 33/50 |
+| Irregular Text Recognition | 50 | 40/50 | 39/50 | 39/50 |
+| Regular Text Recognition | 50 | 49/50 | 49/50 | 49/50 |
+
+## Results — llama.cpp GGUF (same host, same GPU)
+
+| arm | engine | model | correct / scored | accuracy | ±1 s.e. | mean s/item | median | prompt_eval |
+|---|---|---|---|---|---|---|---|---|
+| q4_K_M (run 1) | llama.cpp | `gemma4:31b-it-q4_K_M` | 171 / 200 | **0.855** | 0.025 | 5.0 | 4.9 | 1115 |
+| q4_K_M (run 2) | llama.cpp | `gemma4:31b-it-q4_K_M` | 171 / 200 | **0.855** | 0.025 | 4.9 | 4.8 | 1115 |
+| q8_0 (run 1) | llama.cpp | `gemma4:31b-it-q8_0` | 170 / 200 | **0.850** | 0.025 | 5.1 | 5.0 | 1115 |
+| q8_0 (run 2) | llama.cpp | `gemma4:31b-it-q8_0` | 170 / 200 | **0.850** | 0.025 | 5.2 | 4.8 | 1115 |
+| bf16 (run 1) | llama.cpp | `gemma4:31b-it-bf16` | 171 / 200 | **0.855** | 0.025 | 5.0 | 4.9 | 1115 |
+| bf16 (run 2) | llama.cpp | `gemma4:31b-it-bf16` | 171 / 200 | **0.855** | 0.025 | 4.8 | 4.8 | 1115 |
+
+All 61 layers of the bf16 model sit on the GPU (58 GiB of weights, 1.8 GiB of KV at
+`num_ctx` 8192 — gemma4's sliding-window layers keep the cache small), leaving 19 GB free
+on a card shared with other tenants, so no arm here ran partly on CPU.
+
+**Repeats — the GGUF path is item-level deterministic**
+
+| arm | runs | accuracies | items that flipped |
+|---|---|---|---|
+| q4_K_M | 2 | 0.855, 0.855 | 0 |
+| q8_0 | 2 | 0.850, 0.850 | 0 |
+| bf16 | 2 | 0.855, 0.855 | 0 |
+
+Not one of 200 items changed verdict between runs on either arm. That is the positive
+control the rest of the ladder needs: on this engine a difference between arms is the
+weights, not run noise.
+
+**Paired, the arms measured so far** (same items)
+
+| A | B | A | B | b (A only) | c (B only) | p | resolved |
+|---|---|---|---|---|---|---|---|
+| nvfp4 (mlx-cuda) | q4_K_M (llama.cpp) | 0.860 | 0.855 | 3 | 2 | 1.000 | no |
+| q4_K_M | q8_0 | 0.855 | 0.850 | 1 | 0 | 1.000 | no |
+| q4_K_M | bf16 | 0.855 | 0.855 | 2 | 2 | 1.000 | no |
+| q8_0 | bf16 | 0.850 | 0.855 | 1 | 2 | 1.000 | no |
+
+**The GGUF ladder is flat from 4 bits to bf16.** `q4_K_M` and `bf16` score the same 171
+of 200 and differ on four items, two each way; `q8_0` sits one item below both. Nothing
+resolves, and the seconds per item are the same across all three — this workload is
+prefill-bound with an eight-token answer, so tripling the weights costs no clock. Whatever
+4-bit quantisation does to this model, it is not visible in text recognition, which is the
+same conclusion the Qwen2.5-VL campaign reached at 32B. Against MLX, five items separate the engines, three one way and two the
+other; where both are right or both are wrong they agree exactly, so the engines are not
+reading these images differently, they differ on a handful of hard ones in both
+directions. MLX answers in 2.3 s against llama.cpp's 5.0 s at an identical mean
+prompt_eval of 1115 tokens — the same image budget, so that gap is engine speed, not a
+different amount of image.
+
+**By question type** (first run of each arm), from `ocrbench_table.py --categories`:
+
+| question type | n | mlx-cuda nvfp4 | q4_K_M | q8_0 | bf16 |
+|---|---|---|---|---|---|
+| Artistic Text Recognition | 50 | 49/50 | 49/50 | 48/50 | 48/50 |
+| Handwriting Recognition | 50 | 34/50 | 33/50 | 33/50 | 34/50 |
+| Irregular Text Recognition | 50 | 40/50 | 40/50 | 40/50 | 40/50 |
+| Regular Text Recognition | 50 | 49/50 | 49/50 | 49/50 | 49/50 |
+
+Regular and artistic text are at ceiling for every arm. All the headroom is in
+handwriting (16–17 misses) and irregular text (10 misses, the same count on all three),
+and no arm is more than one item better than another in any category. If a quantisation
+cost exists at 31b, this slice does not show it in text recognition.
+
+## Harness note (2026-09-19)
+
+Two arms died mid-run when a DNS blip made the row fetch fail, so `extbench.py` now
+caches the row slice under `extimgs/<bench>/rows_<offset>_<limit>.json` and retries HTTP
+with backoff. A slice is a fixed set of items: fetching it once per host, rather than
+once per arm, removes the dependency and makes a re-run answer the same questions.
+`REFRESH_ROWS=1` re-fetches. A partial fetch is never cached.
+
+## Results — `mlx-metal` (for the Metal session)
+
+The Metal host's own numbers go here. Preliminary, from Glenn, 2026-09-18: **175 / 200,
+0.875**, described as "0.34.0 fixed kernel" — the artifact it was measured against is
+not yet identified, which the audit above makes load-bearing.
+
+| arm | engine | model | manifest digest | correct / scored | accuracy | mean s/item |
+|---|---|---|---|---|---|---|
+| nvfp4 | mlx-metal | `gemma4:31b-nvfp4` | | | | |
+| mxfp8 | mlx-metal | `gemma4:31b-mxfp8` | | | | |
+| bf16 | mlx-metal | `gemma4:31b-mlx-bf16` | | | | |
+
+**To add yours:**
+
+1. `python3 docs/maxusai/vision-suite/store_audit.py gemma4` on that host, and paste the
+   result. It says whether your `gemma4:31b-nvfp4` is the quantised-tower artifact this
+   ladder's first row uses or the library's current bf16-tower one. Without that line the
+   number cannot be placed on the ladder.
+2. Run the command in "What is measured" per arm, one model loaded at a time.
+3. Render with
+   `python3 docs/maxusai/vision-suite/ocrbench_table.py --engine mlx-metal "nvfp4=<tag>"`
+   and paste the table into this section. Same generator, so the columns line up.
+4. State the build the runner came from, as the CUDA rows do.
+5. Label the number for what it is: rows 0–200 are OCRBench's four text-recognition
+   categories, not the benchmark. `--categories` prints the per-type split.
+
+---
+
+## How to read these numbers
+
+- **A 200-item slice resolves about 5 points, not 1.** At p ≈ 0.87 the binomial standard
+  error is 0.024, so 0.860 and 0.875 — 3 items — are the same number. Do not rank arms on
+  the accuracy column.
+- **The arms answer the same items, so compare them paired.** The generator prints the
+  discordant counts and an exact McNemar p for every pair. That resolves differences the
+  marginals cannot, and it is the only column that supports a claim like "q8 beats q4".
+- **MLX is not bit-reproducible run to run** (despite [ADR 0012] §4), so every MLX arm is run at least twice and the repeat table
+  reports the spread and how many items changed verdict. A gap smaller than an arm's own
+  repeat spread is not a finding.
+- **`mlx-cuda` and `mlx-metal` never share a table.** Different kernels, different
+  silicon, and — per the audit above — possibly different weights.
+- **Speed here is not a throughput claim.** `SLEEP=1` between requests and a shared GPU
+  make the seconds column a sanity check, not a benchmark.
+
+## Provenance
+
+- Score files: `docs/maxusai/vision-suite/ext_ocr31b_*_ocrbench.json` (one record per
+  item: prompt, prediction, gold, verdict, timings), gitignored, kept on the array.
+- Runner logs: `preflight-runs/ocrladder*.log` and `ocrbench-runner.log`.
+- Manifest digests of the CUDA arms:
+
+  | model | manifest digest |
+  |---|---|
+  | `gemma4:31b-nvfp4` (this store, quantised tower) | `sha256:637cc0ff15709212de4aa694be67e3af5e80533ca538c86ad69c53c511e40840` |
+  | `gemma4:31b-it-q4_K_M` | `sha256:6316f0629137b426c9d9b853ffc4c8209589f30ee39aebede6285096c0ff47e7` |
+  | `gemma4:31b-it-q8_0` | `sha256:53dd8459790f8795177444daa9e33f417e03c0d1cdedb80b6c73898603d20aef` |
+
+  The pulled library tags are kept in a separate store on the array
+  (`claude-scratch/ocrbench-store`) so that pulling them cannot overwrite the artifacts
+  the fork's existing measurements were taken against:
+
+  | model (library, pulled 2026-09-19) | manifest digest |
+  |---|---|
+  | `gemma4:31b-nvfp4` (bf16 tower) | `sha256:a22a363052da770302019d5afab29bd967f9318e4377bddd297421189ea09d7d` |
+  | `gemma4:31b-mxfp8` | `sha256:9740f018f0d6bb64b439b6d701017bac85675511d519eae18992b7ed09d0dc6a` |
+  | `gemma4:31b-it-bf16` | `sha256:236d76ae08745dbc143c31b9271b0f25750885199aa6039d0fc0113171606e6d` |
+
+[ADR 0012]: adr/0012-benchmark-report-templates.md
diff --git a/docs/maxusai/spec/vision-harness-reuse.md b/docs/maxusai/spec/vision-harness-reuse.md
index 5bb77c62df6..5d03606e4aa 100644
--- a/docs/maxusai/spec/vision-harness-reuse.md
+++ b/docs/maxusai/spec/vision-harness-reuse.md
@@ -164,6 +164,24 @@ fragments were WRONG about the recommended shape — they showed
 actually uses named coordinate fields. A hand-typed example asserts a wire
 format nothing enforces; an emitted one cannot drift.
 
+**H15 — A reported model is named by its manifest digest, and a slice is named
+by what it contains.** Two hosts holding the same tag do not hold the same
+weights: `gemma4:31b-nvfp4` was re-published with a bf16 vision tower while this
+store kept the nvfp4 one — 194 layers different, the config blob identical, and
+`ollama show` silent about all of it (ADR 0038). A number attributed to a tag
+therefore attributes nothing. `vision-suite/store_audit.py --digests` prints the
+digest to cite, and the same tool run on both hosts is what makes a cross-host
+comparison mean anything.
+
+The second half is the benchmark's own scope. External sets are ordered by
+task, so a `LIMIT`/`OFFSET` window is a stratum, not a sample: OCRBench rows
+0–200 are four of its ten categories — regular, irregular, artistic and
+handwriting recognition — and contain no VQA, key-information extraction,
+digit strings or handwritten maths. A slice result names its rows and its
+categories, and is never called by the benchmark's name alone or set beside a
+published score for the whole set. `ocrbench_table.py --categories` prints the
+split from the cached rows.
+
 **H13 — Report footers derive provenance from the score files, and a MIXED
 footer blocks publication.** T1 and T2 print host(s) and build(s) collected
 from the H11 fields of every file they render. A file that loaded but carries
@@ -389,3 +407,4 @@ something previously hidden:
 | H11 | `host` / `server_version` on every score block, written unconditionally by `client.generate()`; absence marks a pre-2026-08-20 cell |
 | H10 | `client.RETRY_BACKOFF` = 5/15/30s with `_retries` recorded per cell; `test_client.py::TestTransportRetry` asserts a 400 calls `urlopen` exactly once while a 503 retries. `client.evict_others()` polls `/api/ps` until the eviction is observable and returns what it could not evict; `run_engine_compare.sh` calls it before each model when `RESTART_CMD` is absent, `COLD_START=0` opts out |
 | H8 | **Nothing enforces this.** It is a reading habit, and it is the one that would have prevented all three incidents |
+| H15 | `store_audit.py` diffs the store against the registry and `--digests` prints the manifest digest to cite (`test_store_audit.py`); `ocrbench_table.py --categories` prints a slice's question-type split from the cached rows (`test_ocrbench_table.py::TestCategories`) |
diff --git a/docs/maxusai/vision-benchmark-survey.md b/docs/maxusai/vision-benchmark-survey.md
index 1202cc9e4b3..ba2040b69ea 100644
--- a/docs/maxusai/vision-benchmark-survey.md
+++ b/docs/maxusai/vision-benchmark-survey.md
@@ -570,8 +570,18 @@ LIMIT=200 THINK=false SLEEP=1 \
 
 **OCRBench is the right first external benchmark**: 1,000 items at 0.07 GB, ungated,
 judge-free contains-match scoring, and it is the *only* rerunnable benchmark for which one of
-our models has published both-think-modes numbers (nemotron3: 88.3 off / 86.6 on). A 200-item
-slice takes minutes and yields a number comparable to a model card. Add `countbenchqa` (491
+our models has published both-think-modes numbers (nemotron3: 88.3 off / 86.6 on).
+
+> **A 200-item slice is not comparable to a model card, and this paragraph used to say it
+> was.** Measured 2026-09-19: the set is ordered by task, so rows 0–200 are four of its ten
+> categories — regular, irregular, artistic and handwriting recognition, 50 each — with no
+> VQA, key-information extraction, digit strings or handwritten maths. A slice score is an
+> accuracy on the categories it covers and nothing more; published OCRBench numbers are out
+> of 1,000 across all ten. Use the full set, or a stratified window, before setting a number
+> beside a model card, and name the rows either way (SPEC `vision-harness-reuse` H15).
+> `ocrbench_table.py --categories` prints the split. The first ladder run on this slice is
+> [ocrbench-quantisation-ladder.md](ocrbench-quantisation-ladder.md): gemma4:31b across six
+> quantisations, where every arm lands between 0.845 and 0.860 and no pair resolves. Add `countbenchqa` (491
 items, 0.02 GB) when you want a counting check, and `refcoco` when you want to cross-validate
 the bbox scorer against external ground truth.
 
diff --git a/docs/maxusai/vision-suite/README.md b/docs/maxusai/vision-suite/README.md
index 4264d7ece2e..19e9c9e6cf5 100644
--- a/docs/maxusai/vision-suite/README.md
+++ b/docs/maxusai/vision-suite/README.md
@@ -40,6 +40,18 @@ first is image accounting and the second is generation length.
   `client.generate()`; nothing else builds a payload (SPEC H1 / ADR 0028). Carries
   endpoint choice, sampling, the vision-budget options, the context-overflow 400
   translation, `thinking` normalisation across both envelopes, and persistence.
+- `store_audit.py [--store PATH] [--digests] [match ...]` — does this store still hold
+  what the library serves under that tag? A published tag is mutable: `gemma4:31b-nvfp4`
+  was re-published with a bf16 vision tower while our copy keeps the nvfp4 one, same name
+  and same config blob, and `ollama show` surfaces neither. Prints the tags whose layers
+  moved, which tensor groups they are in, and the size ratio (≈3.55× is nvfp4 → bf16).
+  `--digests` prints the local manifest digest to cite in a record instead of the tag.
+  See [../ocrbench-quantisation-ladder.md](../ocrbench-quantisation-ladder.md).
+- `ocrbench_table.py <label=tag[,tag]> ...` — renders an OCRBench ladder from
+  `extbench.py` score files: the arms, the repeat spread, and every pair compared on the
+  items both answered. A 200-item slice carries a ±0.024 standard error, so the paired
+  discordant counts and the exact McNemar p are what resolve two arms, not the accuracy
+  column.
 - `gen_geometry.py` — renders the SPEC C13–C18 geometry set (14 sizes) into
   `visimgs/geom/`, with per-geometry ground truth scaled from fractional shape
   coordinates. Never writes into `visimgs/`.
diff --git a/docs/maxusai/vision-suite/extbench.py b/docs/maxusai/vision-suite/extbench.py
index cd4b6312816..f2c191ce56e 100644
--- a/docs/maxusai/vision-suite/extbench.py
+++ b/docs/maxusai/vision-suite/extbench.py
@@ -12,12 +12,13 @@
   chartqa       lmms-lab-encoder/ChartQA test relaxed accuracy (+-5% numeric)
   refcoco       lmms-lab-encoder/RefCOCO val  dialect-aware bbox IoU
 
-Env: LIMIT (default 50), OFFSET (0), THINK=on|false (false), ENDPOINT=generate|chat
+Env: LIMIT (default 50), OFFSET (0), REFRESH_ROWS=1 (re-fetch the cached row
+slice), THINK=on|false (false), ENDPOINT=generate|chat
 (generate), NUM_PREDICT, NUM_CTX (16384), TIMEOUT (900), SLEEP (0 — seconds between
 requests, to yield the GPU on a shared host).
 
 Writes ext_<tag>_<bench>.json (per-item records + summary) beside the script and
-caches images under extimgs/<bench>/.
+caches images and the row slice under extimgs/<bench>/.
 
 The refcoco scorer reuses the dialect logic of vision_suite.py: it searches
 pixel / norm-1000 / norm-0-1 spaces and xyxy / yxyx orders per item and keeps the
@@ -52,13 +53,36 @@
 BBOX_KEYS = ("bbox", "bbox_2d", "box_2d", "box")
 
 
-def http_json(url, timeout=60):
-    req = urllib.request.Request(url, headers={"User-Agent": "maxusai-extbench/1"})
-    return json.load(urllib.request.urlopen(req, timeout=timeout))
+def http_json(url, timeout=60, attempts=4):
+    """GET with retries: a transient DNS or gateway blip must not end a 20-minute arm."""
+    last = None
+    for i in range(attempts):
+        try:
+            req = urllib.request.Request(url, headers={"User-Agent": "maxusai-extbench/1"})
+            return json.load(urllib.request.urlopen(req, timeout=timeout))
+        except Exception as e:                                        # noqa: BLE001
+            last = e
+            if i + 1 < attempts:
+                time.sleep(2 ** i)
+    raise last
+
+
+def rows_cache_path(bench, offset, limit):
+    return os.path.join(DIR, "extimgs", bench, f"rows_{offset}_{limit}.json")
 
 
 def fetch_rows(bench, offset, limit):
-    """HF datasets-server /rows, paged at its 100-row maximum."""
+    """HF datasets-server /rows, paged at its 100-row maximum, cached on disk.
+
+    A slice is a fixed set of items, so it is fetched once and reused: re-fetching per
+    arm re-asks a remote service for an answer that must not change between arms, and
+    makes every arm depend on that service still resolving. Set REFRESH_ROWS=1 to
+    re-fetch (and to notice a dataset that moved under a pinned offset).
+    """
+    path = rows_cache_path(bench, offset, limit)
+    if os.path.exists(path) and os.environ.get("REFRESH_ROWS", "") != "1":
+        with open(path) as f:
+            return json.load(f)
     spec = BENCHES[bench]
     out = []
     while len(out) < limit:
@@ -70,6 +94,12 @@ def fetch_rows(bench, offset, limit):
         if not rows:
             break
         out.extend(r["row"] for r in rows)
+    if len(out) == limit:                    # only a complete slice is worth caching
+        os.makedirs(os.path.dirname(path), exist_ok=True)
+        tmp = path + ".tmp"
+        with open(tmp, "w") as f:
+            json.dump(out, f)
+        os.replace(tmp, path)
     return out
 
 
diff --git a/docs/maxusai/vision-suite/ocrbench_table.py b/docs/maxusai/vision-suite/ocrbench_table.py
new file mode 100644
index 00000000000..083b682b009
--- /dev/null
+++ b/docs/maxusai/vision-suite/ocrbench_table.py
@@ -0,0 +1,188 @@
+#!/usr/bin/env python3
+"""Render an OCRBench ladder from `extbench.py` score files (ADR 0012: tables come from generators).
+
+    python3 ocrbench_table.py "nvfp4=ocr31b_nvfp4,ocr31b_nvfp4_r2" "q4_K_M=ocr31b_q4km"
+    python3 ocrbench_table.py --dir . --engine mlx-cuda "bf16=ocr31b_bf16"
+    python3 ocrbench_table.py --categories "q4=ocr31b_q4km" "q8=ocr31b_q8"
+
+Each argument is `label=tag[,tag...]`; several tags are repeats of one arm. Reads
+`ext_<tag>_ocrbench.json` and writes three tables: the arms, the repeat spread, and
+every pair compared on the items both arms answered.
+
+Why paired: a 200-item slice carries a binomial standard error near 2.4 points at
+p = 0.87, so two arms 3 items apart are indistinguishable from their accuracies alone.
+The arms answer the *same* items, so the discordant pairs (b, c) carry the signal and
+an exact McNemar test resolves differences the marginals cannot. A difference that does
+not clear it is not a difference; report it as unresolved rather than as a ranking.
+"""
+import argparse
+import json
+import math
+import os
+import sys
+from itertools import combinations
+
+
+def load(directory, tag):
+    with open(os.path.join(directory, f"ext_{tag}_ocrbench.json")) as f:
+        d = json.load(f)
+    rec = [r for r in d["records"] if "error" not in r]
+    return {
+        "summary": d["summary"],
+        "ok": {r["i"]: bool(r["ok"]) for r in rec},
+        "secs": [r.get("secs") or 0 for r in rec],
+        "prompt_eval": [r.get("prompt_eval_count") or 0 for r in rec],
+    }
+
+
+def stderr(correct, n):
+    """Binomial standard error of the slice accuracy."""
+    if not n:
+        return 0.0
+    p = correct / n
+    return math.sqrt(p * (1 - p) / n)
+
+
+def mcnemar_p(b, c):
+    """Exact two-sided McNemar: the discordant pairs under a fair coin."""
+    n = b + c
+    if n == 0:
+        return 1.0
+    k = min(b, c)
+    return min(1.0, 2 * sum(math.comb(n, i) for i in range(k + 1)) / (2 ** n))
+
+
+def question_types(directory, offset, limit, bench="ocrbench"):
+    """{row index: question_type} from extbench's cached row slice; {} when it is absent.
+
+    Reads the cache rather than the dataset server: the breakdown must describe the
+    same rows the arms answered, and a run that scored 200 items already has them.
+    """
+    path = os.path.join(directory, "extimgs", bench, f"rows_{offset}_{limit}.json")
+    if not os.path.exists(path):
+        return {}
+    with open(path) as f:
+        rows = json.load(f)
+    return {offset + i: r.get("question_type") or "unlabelled" for i, r in enumerate(rows)}
+
+
+def category_table(arms, types):
+    """Correct-per-type for each arm: where a ladder's misses actually live."""
+    labels = list(arms)
+    rows = ["| question type | n | " + " | ".join(labels) + " |",
+            "|---|---|" + "---|" * len(labels)]
+    for name in sorted(set(types.values())):
+        idx = [i for i, t in types.items() if t == name]
+        cells = []
+        for label in labels:
+            ok = arms[label][0][1]["ok"]
+            got = [ok[i] for i in idx if i in ok]
+            cells.append(f"{sum(got)}/{len(got)}" if got else "—")
+        rows.append(f"| {name} | {len(idx)} | " + " | ".join(cells) + " |")
+    return "\n".join(rows)
+
+
+def mean(xs):
+    return sum(xs) / len(xs) if xs else 0.0
+
+
+def median(xs):
+    return sorted(xs)[len(xs) // 2] if xs else 0.0
+
+
+def arm_table(arms, engine):
+    rows = ["| arm | engine | model | correct / scored | accuracy | ±1 s.e. | mean s/item | median | prompt_eval |",
+            "|---|---|---|---|---|---|---|---|---|"]
+    for label, runs in arms.items():
+        for i, (tag, r) in enumerate(runs):
+            s = r["summary"]
+            n, c = s["scored"], s["correct"]
+            name = label if len(runs) == 1 else f"{label} (run {i + 1})"
+            rows.append(f"| {name} | {engine} | `{s['model']}` | {c} / {n} | **{c / n:.3f}** | "
+                        f"{stderr(c, n):.3f} | {mean(r['secs']):.1f} | {median(r['secs']):.1f} | "
+                        f"{mean(r['prompt_eval']):.0f} |")
+    return "\n".join(rows)
+
+
+def repeat_table(arms):
+    """Repeats of one arm: the spread, and how many items changed verdict between runs."""
+    multi = {l: r for l, r in arms.items() if len(r) > 1}
+    if not multi:
+        return ""
+    rows = ["| arm | runs | accuracies | items that flipped |", "|---|---|---|---|"]
+    for label, runs in multi.items():
+        accs = [f"{r['summary']['correct'] / r['summary']['scored']:.3f}" for _, r in runs]
+        base = runs[0][1]["ok"]
+        flips = sum(sum(1 for i, v in r["ok"].items() if i in base and v != base[i])
+                    for _, r in runs[1:])
+        rows.append(f"| {label} | {len(runs)} | {', '.join(accs)} | {flips} |")
+    return "\n".join(rows)
+
+
+def paired_table(arms):
+    """Every pair on the items both answered: b, c and the exact McNemar p."""
+    first = {l: r[0][1] for l, r in arms.items()}
+    rows = ["| A | B | A | B | b (A only) | c (B only) | p | resolved |",
+            "|---|---|---|---|---|---|---|---|"]
+    for (la, ra), (lb, rb) in combinations(first.items(), 2):
+        shared = sorted(set(ra["ok"]) & set(rb["ok"]))
+        if not shared:
+            continue
+        b = sum(1 for i in shared if ra["ok"][i] and not rb["ok"][i])
+        c = sum(1 for i in shared if rb["ok"][i] and not ra["ok"][i])
+        p = mcnemar_p(b, c)
+        rows.append(f"| {la} | {lb} | {sum(ra['ok'][i] for i in shared) / len(shared):.3f} | "
+                    f"{sum(rb['ok'][i] for i in shared) / len(shared):.3f} | {b} | {c} | "
+                    f"{p:.3f} | {'**yes**' if p < 0.05 else 'no'} |")
+    return "\n".join(rows)
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("arms", nargs="+", help="label=tag[,tag...]")
+    ap.add_argument("--dir", default=os.path.dirname(os.path.abspath(__file__)))
+    ap.add_argument("--categories", action="store_true",
+                    help="also break the first run of each arm down by OCRBench question type")
+    ap.add_argument("--engine", default="mlx-cuda",
+                    help="engine label for the table; mlx-cuda and mlx-metal numbers never share one")
+    a = ap.parse_args(argv)
+
+    arms, missing = {}, []
+    for spec in a.arms:
+        label, _, tags = spec.partition("=")
+        runs = []
+        for tag in tags.split(","):
+            try:
+                runs.append((tag, load(a.dir, tag)))
+            except FileNotFoundError:
+                missing.append(tag)
+        if runs:
+            arms[label] = runs
+    if not arms:
+        sys.exit(f"no score files found (looked for {', '.join(missing)} in {a.dir})")
+
+    s = arms[next(iter(arms))][0][1]["summary"]
+    print(arm_table(arms, a.engine))
+    print(f"\nOCRBench `{s['dataset']}` [{s['split']}] rows {s['offset']}–{s['offset'] + s['requested']}, "
+          f"think {s['think_env']}, endpoint {s['endpoint']}, contains-match scoring (lmms-eval semantics).")
+    rep = repeat_table(arms)
+    if rep:
+        print("\n**Repeats**\n")
+        print(rep)
+    if len(arms) > 1:
+        print("\n**Paired on the same items** (first run of each arm)\n")
+        print(paired_table(arms))
+    if a.categories:
+        types = question_types(a.dir, s["offset"], s["requested"])
+        if types:
+            print("\n**By OCRBench question type** (first run of each arm)\n")
+            print(category_table(arms, types))
+        else:
+            print("\n<!-- no cached row slice in extimgs/ocrbench; run an arm first -->")
+    if missing:
+        print(f"\nMissing score files: {', '.join('`ext_%s_ocrbench.json`' % m for m in missing)}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/docs/maxusai/vision-suite/store_audit.py b/docs/maxusai/vision-suite/store_audit.py
new file mode 100644
index 00000000000..a94672028a9
--- /dev/null
+++ b/docs/maxusai/vision-suite/store_audit.py
@@ -0,0 +1,195 @@
+#!/usr/bin/env python3
+"""Does the local ollama store still hold what the library serves under that tag?
+
+    python3 store_audit.py                      # every tag in the default store
+    python3 store_audit.py gemma4 qwen3.5       # only tags matching these substrings
+    python3 store_audit.py --store /var/lib/... # a store somewhere else
+    python3 store_audit.py --digests            # tag -> local manifest digest, for citing in a record
+
+A published tag is not a fixed artifact. `gemma4:31b-nvfp4` was re-published with a
+bf16 vision tower while our copy keeps the nvfp4 one: same name, same config blob,
+1248 layers of which 194 are different weights. `ollama show` reports neither the
+layer set nor a content digest, so nothing surfaces the swap — the first symptom is a
+measurement that will not reproduce, on a host that pulled later.
+
+This walks the store's manifests, asks the registry for the same tags, and prints the
+ones that moved, with the tensor groups affected and the local->registry size ratio
+(~3.55x is nvfp4 -> bf16, ~2x is fp8 -> bf16). Tags the registry does not serve under
+that name are listed separately: those are local builds, and nothing upstream can
+change them.
+
+Cite the local manifest digest (`--digests`) in any record that names a model, so the
+measurement identifies its artifact and not just a mutable label.
+"""
+import argparse
+import collections
+import hashlib
+import json
+import os
+import sys
+import urllib.error
+import urllib.request
+
+DEFAULT_REGISTRY = "https://registry.ollama.ai"
+MANIFEST_ACCEPT = "application/vnd.docker.distribution.manifest.v2+json"
+
+# Tensor-name fragments that put a layer on the vision (or audio) path rather than the
+# language model. MLX-format models carry one layer per tensor, so the split is exact;
+# GGUF models are a single `model` layer and only ever compare whole.
+VISION_HINTS = ("vision_tower", "vision_embedder", "embed_vision", "multi_modal",
+                "vision_model", "mm.", "v.")
+AUDIO_HINTS = ("embed_audio", "audio_tower", "a.")
+
+
+def default_store():
+    return os.environ.get("OLLAMA_MODELS") or os.path.expanduser("~/.ollama/models")
+
+
+def classify(name):
+    """Which part of the model a tensor layer belongs to."""
+    if not name:
+        return "blob"
+    if any(h in name for h in VISION_HINTS):
+        return "vision"
+    if any(h in name for h in AUDIO_HINTS):
+        return "audio"
+    if "language_model" in name or name.startswith("model.") or name.startswith("blk."):
+        return "language"
+    return "other"
+
+
+def manifest_digest(raw):
+    """The digest ollama would address this manifest by: sha256 over its bytes."""
+    return "sha256:" + hashlib.sha256(raw).hexdigest()
+
+
+def read_store(store, namespace="registry.ollama.ai/library"):
+    """Every tag under the store's manifest namespace: {"model:tag": (manifest, digest)}."""
+    root = os.path.join(store, "manifests", *namespace.split("/"))
+    out = {}
+    if not os.path.isdir(root):
+        return out
+    for model in sorted(os.listdir(root)):
+        d = os.path.join(root, model)
+        if not os.path.isdir(d):
+            continue
+        for tag in sorted(os.listdir(d)):
+            path = os.path.join(d, tag)
+            if not os.path.isfile(path):
+                continue
+            with open(path, "rb") as f:
+                raw = f.read()
+            try:
+                out[f"{model}:{tag}"] = (json.loads(raw), manifest_digest(raw))
+            except ValueError:
+                continue
+    return out
+
+
+def fetch(model, tag, registry=DEFAULT_REGISTRY, timeout=30):
+    """The manifest the registry serves for this tag right now, plus its content digest."""
+    req = urllib.request.Request(f"{registry}/v2/library/{model}/manifests/{tag}",
+                                 headers={"Accept": MANIFEST_ACCEPT,
+                                          "User-Agent": "maxusai-store-audit/1"})
+    with urllib.request.urlopen(req, timeout=timeout) as r:
+        raw = r.read()
+    return json.loads(raw), r.headers.get("Docker-Content-Digest") or manifest_digest(raw)
+
+
+def compare(local, remote):
+    """What moved between two manifests of the same tag.
+
+    Returns moved-layer count, the tensor groups they fall in, whether the config blob
+    is unchanged (which is what makes a swap invisible to `ollama show`), and the mean
+    remote/local size ratio over layers matched by tensor name.
+    """
+    rem = {l["digest"] for l in remote["layers"]}
+    rem_by_name = {l.get("name"): l["size"] for l in remote["layers"] if l.get("name")}
+    moved = [l for l in local["layers"] if l["digest"] not in rem]
+    groups = collections.Counter(classify(l.get("name")) for l in moved)
+    ratios = [rem_by_name[l["name"]] / l["size"] for l in moved
+              if l.get("name") in rem_by_name and l.get("size")]
+    return {
+        "moved": len(moved),
+        "total": len(local["layers"]),
+        "groups": dict(groups),
+        "config_same": local.get("config", {}).get("digest") == remote.get("config", {}).get("digest"),
+        "ratio": round(sum(ratios) / len(ratios), 2) if ratios else None,
+        "names": [l.get("name") for l in moved if l.get("name")][:5],
+    }
+
+
+def dtype_reading(ratio):
+    """What a size ratio implies, for the common quantisation moves. None when unclear."""
+    if ratio is None:
+        return ""
+    for lo, hi, text in ((3.3, 3.8, "nvfp4 → bf16"), (1.8, 2.2, "fp8 → bf16"),
+                         (0.26, 0.31, "bf16 → nvfp4"), (0.45, 0.55, "bf16 → fp8")):
+        if lo <= ratio <= hi:
+            return text
+    return ""
+
+
+def render(rows):
+    """Markdown for the tags that moved. `rows` is [(tag, local_digest, remote_digest, compare())]."""
+    out = ["| tag | local manifest | registry manifest | layers changed | where | size ratio | reading |",
+           "|---|---|---|---|---|---|---|"]
+    for tag, ldig, rdig, c in rows:
+        where = ", ".join(f"{k} {v}" for k, v in sorted(c["groups"].items(), key=lambda kv: -kv[1]))
+        reading = dtype_reading(c["ratio"])
+        out.append(f"| `{tag}` | `{ldig[7:19]}` | `{rdig[7:19]}` | {c['moved']} / {c['total']} | "
+                   f"{where} | {c['ratio'] or '—'}× | {reading or '—'} |")
+    return "\n".join(out)
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("match", nargs="*", help="only tags containing one of these substrings")
+    ap.add_argument("--store", default=default_store())
+    ap.add_argument("--registry", default=DEFAULT_REGISTRY)
+    ap.add_argument("--digests", action="store_true", help="print tag -> local manifest digest and stop")
+    a = ap.parse_args(argv)
+
+    store = read_store(a.store)
+    if not store:
+        sys.exit(f"no manifests under {a.store}")
+    picked = {t: v for t, v in store.items() if not a.match or any(m in t for m in a.match)}
+
+    if a.digests:
+        for tag, (_, dig) in sorted(picked.items()):
+            print(f"{tag:<50} {dig}")
+        return 0
+
+    moved, identical, local_only, failed = [], [], [], []
+    for tag, (loc, ldig) in sorted(picked.items()):
+        model, _, name = tag.partition(":")
+        try:
+            rem, rdig = fetch(model, name, a.registry)
+        except urllib.error.HTTPError as e:
+            (local_only if e.code == 404 else failed).append((tag, e.code))
+            continue
+        except Exception as e:                                        # noqa: BLE001
+            failed.append((tag, str(e)))
+            continue
+        c = compare(loc, rem)
+        (moved if c["moved"] else identical).append((tag, ldig, rdig, c))
+
+    if moved:
+        print(f"## Tags whose weights changed under the same name ({len(moved)})\n")
+        print(render(moved))
+        print()
+        for tag, _, _, c in moved:
+            if c["names"]:
+                print(f"- `{tag}` first moved tensors: " + ", ".join(f"`{n}`" for n in c["names"]))
+        print()
+    print(f"{len(identical)} tag(s) identical to the registry, {len(moved)} changed, "
+          f"{len(local_only)} not served under that name (local builds), {len(failed)} unreadable.")
+    if local_only:
+        print("\nLocal only: " + ", ".join(f"`{t}`" for t, _ in local_only))
+    if failed:
+        print("\nUnreadable: " + ", ".join(f"`{t}` ({e})" for t, e in failed))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/docs/maxusai/vision-suite/test_extbench.py b/docs/maxusai/vision-suite/test_extbench.py
new file mode 100644
index 00000000000..dafb2f4978a
--- /dev/null
+++ b/docs/maxusai/vision-suite/test_extbench.py
@@ -0,0 +1,89 @@
+#!/usr/bin/env python3
+"""extbench.py's row cache, retries and scorers — no network.
+
+    python3 test_extbench.py
+"""
+import json
+import os
+import sys
+import tempfile
+import unittest
+from unittest import mock
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import extbench  # noqa: E402
+
+
+class TestRowCache(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.TemporaryDirectory()
+        self.addCleanup(self.tmp.cleanup)
+        patcher = mock.patch.object(extbench, "DIR", self.tmp.name)
+        patcher.start()
+        self.addCleanup(patcher.stop)
+        os.environ.pop("REFRESH_ROWS", None)
+
+    def test_slice_is_fetched_once_then_read_from_disk(self):
+        page = {"rows": [{"row": {"question": f"q{i}", "answer": ["a"], "image": "u"}}
+                         for i in range(3)]}
+        with mock.patch.object(extbench, "http_json", return_value=page) as get:
+            first = extbench.fetch_rows("ocrbench", 0, 3)
+            self.assertEqual(get.call_count, 1)
+        with mock.patch.object(extbench, "http_json", side_effect=AssertionError("refetched")) as get:
+            second = extbench.fetch_rows("ocrbench", 0, 3)
+            self.assertEqual(get.call_count, 0, "a cached slice must not hit the network")
+        self.assertEqual(first, second)
+        self.assertTrue(os.path.exists(extbench.rows_cache_path("ocrbench", 0, 3)))
+
+    def test_refresh_rows_bypasses_the_cache(self):
+        page = {"rows": [{"row": {"question": "q", "answer": ["a"], "image": "u"}}]}
+        with mock.patch.object(extbench, "http_json", return_value=page):
+            extbench.fetch_rows("ocrbench", 0, 1)
+        os.environ["REFRESH_ROWS"] = "1"
+        self.addCleanup(os.environ.pop, "REFRESH_ROWS", None)
+        with mock.patch.object(extbench, "http_json", return_value=page) as get:
+            extbench.fetch_rows("ocrbench", 0, 1)
+            self.assertEqual(get.call_count, 1)
+
+    def test_short_slice_is_not_cached(self):
+        """A truncated fetch must not poison the cache with a partial arm."""
+        page = {"rows": [{"row": {"question": "q", "answer": ["a"], "image": "u"}}]}
+        with mock.patch.object(extbench, "http_json", side_effect=[page, {"rows": []}]):
+            rows = extbench.fetch_rows("ocrbench", 0, 5)
+        self.assertEqual(len(rows), 1)
+        self.assertFalse(os.path.exists(extbench.rows_cache_path("ocrbench", 0, 5)))
+
+    def test_cache_is_keyed_by_offset_and_limit(self):
+        self.assertNotEqual(extbench.rows_cache_path("ocrbench", 0, 200),
+                            extbench.rows_cache_path("ocrbench", 200, 200))
+
+
+class TestRetry(unittest.TestCase):
+    def test_http_json_retries_then_succeeds(self):
+        with mock.patch.object(extbench.urllib.request, "urlopen",
+                               side_effect=[OSError("dns"), mock.MagicMock()]) as op, \
+             mock.patch.object(extbench.json, "load", return_value={"ok": True}), \
+             mock.patch.object(extbench.time, "sleep"):
+            self.assertEqual(extbench.http_json("https://x", attempts=3), {"ok": True})
+            self.assertEqual(op.call_count, 2)
+
+    def test_http_json_raises_the_last_error_when_every_attempt_fails(self):
+        with mock.patch.object(extbench.urllib.request, "urlopen", side_effect=OSError("dns")), \
+             mock.patch.object(extbench.time, "sleep"):
+            with self.assertRaises(OSError):
+                extbench.http_json("https://x", attempts=2)
+
+
+class TestOCRBenchScoring(unittest.TestCase):
+    def test_contains_match_is_case_insensitive(self):
+        self.assertTrue(extbench.score_ocrbench("Friend", {"answer": ["FRIEND"]}))
+        self.assertTrue(extbench.score_ocrbench("the sign reads CHAIN.", {"answer": ["chain"]}))
+        self.assertFalse(extbench.score_ocrbench("Centuries", {"answer": ["CENTRE"]}))
+
+    def test_handwritten_maths_ignores_whitespace_on_both_sides(self):
+        row = {"answer": ["x ^ 2 + 1"], "question_type": "Handwritten Mathematical Expression Recognition"}
+        self.assertTrue(extbench.score_ocrbench("x^2+1", row))
+
+
+if __name__ == "__main__":
+    unittest.main(verbosity=2)
diff --git a/docs/maxusai/vision-suite/test_ocrbench_table.py b/docs/maxusai/vision-suite/test_ocrbench_table.py
new file mode 100644
index 00000000000..8aa32143cf6
--- /dev/null
+++ b/docs/maxusai/vision-suite/test_ocrbench_table.py
@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+"""ocrbench_table.py against synthetic score files.
+
+    python3 test_ocrbench_table.py
+"""
+import contextlib
+import io
+import json
+import os
+import sys
+import tempfile
+import unittest
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import ocrbench_table  # noqa: E402
+
+
+def write(directory, tag, verdicts, model="gemma4:31b-nvfp4"):
+    """A score file whose i-th record is correct iff verdicts[i]."""
+    recs = [{"i": i, "pred": "x", "secs": 2.0, "prompt_eval_count": 1100,
+             "eval_count": 3, "gold": ["x"], "ok": bool(v)} for i, v in enumerate(verdicts)]
+    doc = {"summary": {"tag": tag, "model": model, "benchmark": "ocrbench",
+                       "dataset": "echo840/OCRBench", "split": "test", "offset": 0,
+                       "requested": len(verdicts), "scored": len(verdicts), "errors": 0,
+                       "empty_responses": 0, "think_env": "false", "think_on": False,
+                       "endpoint": "generate", "correct": sum(bool(v) for v in verdicts),
+                       "accuracy": round(sum(bool(v) for v in verdicts) / len(verdicts), 4)},
+           "records": recs}
+    with open(os.path.join(directory, f"ext_{tag}_ocrbench.json"), "w") as f:
+        json.dump(doc, f)
+
+
+class TestStats(unittest.TestCase):
+    def test_stderr_matches_the_binomial_form(self):
+        self.assertAlmostEqual(ocrbench_table.stderr(174, 200), 0.0238, places=3)
+        self.assertEqual(ocrbench_table.stderr(0, 0), 0.0)
+
+    def test_mcnemar_is_symmetric_and_exact(self):
+        self.assertEqual(ocrbench_table.mcnemar_p(0, 0), 1.0)
+        self.assertEqual(ocrbench_table.mcnemar_p(3, 3), ocrbench_table.mcnemar_p(3, 3))
+        # 10 discordant pairs all one way is the strongest 10-pair evidence: 2 * 0.5**10
+        self.assertAlmostEqual(ocrbench_table.mcnemar_p(10, 0), 2 / 1024, places=6)
+        # a 3-item gap on a 200-item slice, spread over many discordant pairs, resolves nothing
+        self.assertGreater(ocrbench_table.mcnemar_p(14, 11), 0.05)
+
+
+class TestTables(unittest.TestCase):
+    def test_arm_table_reports_accuracy_and_engine(self):
+        with tempfile.TemporaryDirectory() as d:
+            write(d, "a", [1] * 172 + [0] * 28)
+            arms = {"nvfp4": [("a", ocrbench_table.load(d, "a"))]}
+            out = ocrbench_table.arm_table(arms, "mlx-cuda")
+            self.assertIn("172 / 200", out)
+            self.assertIn("**0.860**", out)
+            self.assertIn("mlx-cuda", out)
+
+    def test_repeats_count_flipped_items(self):
+        with tempfile.TemporaryDirectory() as d:
+            write(d, "r1", [1, 1, 0, 0])
+            write(d, "r2", [1, 0, 1, 0])          # items 1 and 2 flipped
+            arms = {"nvfp4": [("r1", ocrbench_table.load(d, "r1")),
+                              ("r2", ocrbench_table.load(d, "r2"))]}
+            out = ocrbench_table.repeat_table(arms)
+            self.assertIn("| nvfp4 | 2 |", out)
+            self.assertIn("| 2 |", out)
+
+    def test_paired_table_counts_discordant_pairs_both_ways(self):
+        with tempfile.TemporaryDirectory() as d:
+            write(d, "a", [1, 1, 0, 0, 1])
+            write(d, "b", [1, 0, 1, 0, 1])
+            arms = {"A": [("a", ocrbench_table.load(d, "a"))],
+                    "B": [("b", ocrbench_table.load(d, "b"))]}
+            out = ocrbench_table.paired_table(arms)
+            self.assertIn("| A | B |", out)
+            self.assertIn("| 1 | 1 |", out)        # b = 1, c = 1
+            self.assertIn("| no |", out)           # one pair each way resolves nothing
+
+    def test_main_reports_missing_score_files_instead_of_failing(self):
+        with tempfile.TemporaryDirectory() as d:
+            write(d, "present", [1, 0])
+            buf = io.StringIO()
+            with contextlib.redirect_stdout(buf):
+                ocrbench_table.main(["--dir", d, "here=present", "gone=absent"])
+            self.assertIn("Missing score files", buf.getvalue())
+            self.assertIn("ext_absent_ocrbench.json", buf.getvalue())
+
+    def test_main_exits_when_nothing_is_readable(self):
+        with tempfile.TemporaryDirectory() as d:
+            with self.assertRaises(SystemExit):
+                ocrbench_table.main(["--dir", d, "x=nothing"])
+
+
+class TestCategories(unittest.TestCase):
+    def test_question_types_come_from_the_cached_slice(self):
+        with tempfile.TemporaryDirectory() as d:
+            os.makedirs(os.path.join(d, "extimgs", "ocrbench"))
+            with open(os.path.join(d, "extimgs", "ocrbench", "rows_0_3.json"), "w") as f:
+                json.dump([{"question_type": "Handwriting Recognition"},
+                           {"question_type": "Regular Text Recognition"},
+                           {}], f)
+            types = ocrbench_table.question_types(d, 0, 3)
+            self.assertEqual(types[0], "Handwriting Recognition")
+            self.assertEqual(types[2], "unlabelled", "a row without a type is still an item")
+
+    def test_missing_cache_is_empty_not_an_error(self):
+        with tempfile.TemporaryDirectory() as d:
+            self.assertEqual(ocrbench_table.question_types(d, 0, 200), {})
+
+    def test_category_table_counts_per_arm(self):
+        with tempfile.TemporaryDirectory() as d:
+            write(d, "a", [1, 0, 1, 1])
+            write(d, "b", [1, 1, 1, 0])
+            arms = {"A": [("a", ocrbench_table.load(d, "a"))],
+                    "B": [("b", ocrbench_table.load(d, "b"))]}
+            types = {0: "hand", 1: "hand", 2: "regular", 3: "regular"}
+            out = ocrbench_table.category_table(arms, types)
+            self.assertIn("| hand | 2 | 1/2 | 2/2 |", out)
+            self.assertIn("| regular | 2 | 2/2 | 1/2 |", out)
+
+
+if __name__ == "__main__":
+    unittest.main(verbosity=2)
diff --git a/docs/maxusai/vision-suite/test_store_audit.py b/docs/maxusai/vision-suite/test_store_audit.py
new file mode 100644
index 00000000000..9c85395f1c0
--- /dev/null
+++ b/docs/maxusai/vision-suite/test_store_audit.py
@@ -0,0 +1,106 @@
+#!/usr/bin/env python3
+"""store_audit.py against synthetic manifests — no network, no store.
+
+    python3 test_store_audit.py
+"""
+import json
+import os
+import sys
+import tempfile
+import unittest
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import store_audit  # noqa: E402
+
+
+def layer(name, size, digest):
+    return {"mediaType": "application/vnd.ollama.image.tensor", "digest": f"sha256:{digest}",
+            "size": size, "name": name}
+
+
+def manifest(layers, config="sha256:cfg"):
+    return {"schemaVersion": 2, "config": {"digest": config, "size": 417}, "layers": layers}
+
+
+class TestClassify(unittest.TestCase):
+    def test_vision_and_audio_paths_are_not_language(self):
+        self.assertEqual(store_audit.classify("model.vision_tower.encoder.layers.0.mlp.down_proj.linear.weight"), "vision")
+        self.assertEqual(store_audit.classify("model.embed_vision.embedding_projection.weight"), "vision")
+        self.assertEqual(store_audit.classify("v.blk.3.attn_q.weight"), "vision")
+        self.assertEqual(store_audit.classify("model.embed_audio.embedding_projection.weight"), "audio")
+        self.assertEqual(store_audit.classify("model.language_model.layers.4.mlp.up_proj.weight"), "language")
+        self.assertEqual(store_audit.classify("blk.4.ffn_up.weight"), "language")
+
+    def test_unnamed_layer_is_a_blob(self):
+        # GGUF models carry one unnamed `model` layer; it has no tensor structure to group.
+        self.assertEqual(store_audit.classify(None), "blob")
+
+
+class TestCompare(unittest.TestCase):
+    def test_identical_manifests_report_nothing_moved(self):
+        m = manifest([layer("a", 10, "aa"), layer("b", 20, "bb")])
+        c = store_audit.compare(m, m)
+        self.assertEqual(c["moved"], 0)
+        self.assertTrue(c["config_same"])
+
+    def test_vision_tower_requantised_to_bf16(self):
+        """The gemma4 case: language layers untouched, every tower layer 3.55x bigger."""
+        local = manifest([layer("model.language_model.layers.0.mlp.up_proj.weight", 1000, "l0"),
+                          layer("model.vision_tower.encoder.layers.0.mlp.down_proj.linear.weight", 2790000, "v0"),
+                          layer("model.vision_tower.encoder.layers.0.self_attn.q_proj.linear.weight", 750000, "v1")])
+        remote = manifest([layer("model.language_model.layers.0.mlp.up_proj.weight", 1000, "l0"),
+                           layer("model.vision_tower.encoder.layers.0.mlp.down_proj.linear.weight", 9920000, "V0"),
+                           layer("model.vision_tower.encoder.layers.0.self_attn.q_proj.linear.weight", 2650000, "V1")])
+        c = store_audit.compare(local, remote)
+        self.assertEqual(c["moved"], 2)
+        self.assertEqual(c["groups"], {"vision": 2})
+        self.assertTrue(c["config_same"], "the config blob is unchanged — this is what hides the swap")
+        self.assertAlmostEqual(c["ratio"], 3.54, places=2)
+        self.assertEqual(store_audit.dtype_reading(c["ratio"]), "nvfp4 → bf16")
+
+    def test_ratio_is_none_when_names_do_not_match(self):
+        # A GGUF model is one unnamed layer: it can differ, but there is no per-tensor ratio.
+        local = manifest([{"mediaType": "application/vnd.ollama.image.model", "digest": "sha256:g1", "size": 20e9}])
+        remote = manifest([{"mediaType": "application/vnd.ollama.image.model", "digest": "sha256:g2", "size": 21e9}])
+        c = store_audit.compare(local, remote)
+        self.assertEqual(c["moved"], 1)
+        self.assertIsNone(c["ratio"])
+        self.assertEqual(store_audit.dtype_reading(None), "")
+
+    def test_changed_config_is_reported(self):
+        local = manifest([layer("a", 10, "aa")], config="sha256:one")
+        remote = manifest([layer("a", 11, "bb")], config="sha256:two")
+        self.assertFalse(store_audit.compare(local, remote)["config_same"])
+
+
+class TestRenderAndStore(unittest.TestCase):
+    def test_render_one_row(self):
+        c = {"moved": 194, "total": 1248, "groups": {"vision": 191, "other": 3},
+             "config_same": True, "ratio": 3.52, "names": []}
+        out = store_audit.render([("gemma4:31b-nvfp4", "sha256:" + "a" * 64, "sha256:" + "b" * 64, c)])
+        self.assertIn("`gemma4:31b-nvfp4`", out)
+        self.assertIn("194 / 1248", out)
+        self.assertIn("vision 191", out)
+        self.assertIn("3.52×", out)
+        self.assertIn("nvfp4 → bf16", out)
+
+    def test_read_store_returns_manifests_and_digests(self):
+        with tempfile.TemporaryDirectory() as d:
+            p = os.path.join(d, "manifests", "registry.ollama.ai", "library", "gemma4")
+            os.makedirs(p)
+            body = manifest([layer("a", 10, "aa")])
+            with open(os.path.join(p, "31b-nvfp4"), "w") as f:
+                json.dump(body, f)
+            got = store_audit.read_store(d)
+            self.assertIn("gemma4:31b-nvfp4", got)
+            m, dig = got["gemma4:31b-nvfp4"]
+            self.assertEqual(m["layers"][0]["name"], "a")
+            self.assertTrue(dig.startswith("sha256:"))
+            self.assertEqual(len(dig), 71)
+
+    def test_missing_store_is_empty_not_an_error(self):
+        self.assertEqual(store_audit.read_store("/nonexistent/store"), {})
+
+
+if __name__ == "__main__":
+    unittest.main(verbosity=2)