docs: OCRBench quantisation ladder for gemma4:31b, and a store/registry audit

README.md

@@ -91,6 +91,8 @@ decision and its measurements live (`docs/maxusai/`).
 |---|---|---|---|
 | **MLX admission** | weights against free device memory (and, since v0.34.1, a system-memory bound on integrated GPUs) | weights + KV priced at the requested `num_ctx` + a per-architecture headroom; an explicit rung that does not fit is refused, an automatic one is clamped | ADR 0034 |
 | **MLX memory ceiling** | none | `OLLAMA_MLX_MEMORY_LIMIT` and a cache limit, set per runner from the admitted budget | runner knobs |
+| **nvfp4 global scales** | — | held in MLX's `m × 2688` representation and divided back out by every wrapper that applies the scale itself, which is not the identity in float32 (17 of 31b's 191 vision scales move one ulp) | ADR 0039 (proposed) |
+| **model identity in a record** | a tag | the manifest digest: the library re-published `gemma4:*-nvfp4` with bf16 vision towers under unchanged tags and config blobs | ADR 0038 (proposed) |
 | **gemma4 image chunk vs. generation batch (GGUF)** | the batch follows `num_ctx` (1024 above 4096), so a top-rung gemma4 image (up to 1120 tokens) is decoded in two pieces, bidirectional only within each | a gemma4 vision runner starts from the batch rung that holds its image ceiling (2048 at 1120) and steps down only when it does not fit | ADR 0036 |
 | **gemma4 on MLX** | upstream's own vision and audio tower with a fixed per-checkpoint soft-token set, no per-request budget | vision through upstream's `MediaModel` with a per-request budget seam; audio not shipped | ADR 0021 |
 | **media prompts on MLX** | — | prefill chunks span-aligned around image blocks; a late image is refused | ADR 0014 |

docs/maxusai/adr/0038-a-model-is-identified-by-its-manifest-digest.md

@@ -0,0 +1,81 @@
+# ADR 0038: a measured model is identified by its manifest digest, not its tag
+
+- **Status:** proposed 2026-09-19. Awaiting Glenn.
+- **Date:** 2026-09-19
+- **Deciders:** MaxusAI fork maintainers
+
+## Context
+
+`gemma4:31b-nvfp4` in this store and `gemma4:31b-nvfp4` in the library are not the same
+model. Measured 2026-09-18 by diffing the manifests: **194 of 1248 layers differ, every
+one of them on the vision path**, each about 3.5× larger upstream — `nvfp4` (4 bits plus
+scales) widened to `bf16`. The library re-published the tag with an unquantised vision
+tower. Our copy was pulled 2026-08-17; the re-publish is after that date.
+
+The same move hit `gemma4:26b-nvfp4` (190 of 1072 layers) and `gemma4:12b-nvfp4` (its six
+vision and audio projections), and `qwen3.5:0.8b-mlx` was re-published with its **tensors
+renamed** (473 of 481 layers). 39 of the store's 54 tags are byte-identical to the
+registry; 10 are local builds the registry never served.
+
+Three properties make this invisible:
+
+- the **config blob digest is unchanged** in every case, so `ollama show` reports the
+  same parameters, template and licence for both artifacts;
+- the pull is idempotent by tag, not by content, so `ollama pull` on a host that already
+  has the old copy *replaces* it without a word;
+- nothing in the store records where a blob came from or when, beyond file mtimes.
+
+The consequence is not hypothetical. Every gemma4 vision result in this repo — the #312
+encoder investigation, `x/mlxrunner/testdata`'s vision goldens, the fine-text tiers, the
+OCRBench ladder's first row — was measured against the **quantised** tower. A host that
+pulls that tag today gets the bf16 one and will not reproduce any of them, and the old
+artifact **cannot be fetched back**: the registry no longer serves it under any name.
+
+This also decides what a cross-host comparison means. The Metal host's OCRBench number
+and the CUDA host's are comparable only if both ran the same tower precision, and neither
+number carried anything that could answer that.
+
+## Decision
+
+**A record that names a model names its manifest digest.** The tag is how a human finds
+the model; the digest is what the number describes.
+
+1. Any document, issue comment or score file that reports a measurement states the
+   manifest digest of each model measured. `vision-suite/store_audit.py --digests` prints
+   them.
+2. Before trusting a comparison between two hosts, run `store_audit.py` on both. A tag
+   present on both hosts is not evidence they hold the same weights.
+3. Benchmark and campaign pulls go to a **separate store** (a directory mounted at
+   `/root/.ollama`), never the production store, so a pull cannot overwrite an artifact
+   an existing measurement was taken against.
+4. Do not `ollama pull` a tag this repo has measured into the production store. If a
+   newer artifact is wanted, pull it beside the old one and record both digests.
+5. The store is the archive. A measured artifact that is deleted locally is gone, because
+   the registry serves only the current content of a tag.
+
+## Options considered
+
+- **Cite tags, as before.** Cheapest, and what every record did until now. Rejected: the
+  failure is silent, the artifact is unrecoverable, and the first symptom is an
+  irreproducible result on someone else's host.
+- **Pin by the model's config digest.** Rejected on the evidence: the config digest was
+  identical across the re-publish in all four cases, so it identifies the model's
+  definition, not its weights.
+- **Snapshot every measured model to the array.** Complete, and about 19 GB per gemma4
+  nvfp4 tag. Worth doing for artifacts a published result depends on; not required by
+  this ADR, which only makes the identity recordable.
+- **Ask upstream to publish immutable tags.** Out of our hands, and nothing goes to
+  `ollama/ollama` by standing instruction.
+
+## Consequences
+
+- A number without a digest is now an incomplete record. Existing documents are not
+  retro-fitted wholesale; the digest is added when a document is next touched, and
+  `docs/maxusai/ocrbench-quantisation-ladder.md` carries them from the start.
+- The vision suite and preflight do not yet stamp digests into their score files. They
+  should, and that is the natural follow-up — the harness already has the store path.
+- Re-pulling to "get the latest" becomes a deliberate act with a recorded before and
+  after, not routine hygiene.
+- This says nothing about which artifact is *better*. The bf16 tower may well be the
+  better model; see the OCRBench ladder, where it is 39 % faster per image on CUDA and
+  scores within noise.

docs/maxusai/adr/0039-nvfp4-global-scales-are-stored-as-the-checkpoint-multiplier.md

@@ -0,0 +1,100 @@
+# ADR 0039: nvfp4 global scales are stored as the checkpoint multiplier
+
+- **Status:** proposed 2026-09-19. Awaiting Glenn. Arises from
+  [#312](https://github.com/MaxusAI/ollama/issues/312) and gates part of
+  [#287](https://github.com/MaxusAI/ollama/pull/287).
+- **Date:** 2026-09-19
+- **Deciders:** MaxusAI fork maintainers
+
+## Context
+
+An nvfp4 checkpoint carries a per-tensor multiplier `m` (ModelOpt's `weight_scale_2`).
+MLX wants that scale in its own representation, `m × 448 × 6` — `Nvfp4MaxProduct`, 2688 —
+because its kernels fold the maxima of the E4M3 and E2M1 ranges into the scale.
+
+The v0.34 fold made that representation the **stored** form.
+`ToMLXGlobalScale` (`x/mlxrunner/model/quant.go`) converts once at load:
+
+```go
+return mlx.MulScalar(flat, mlx.Nvfp4MaxProduct)   // holds f32(m × 2688)
+```
+
+and every wrapper that applies the scale itself divides it back out, in `scaleAndCast`
+(`x/mlxrunner/mlx/ops_extra.go`):
+
+```go
+return Mul(out, DivScalar(scale, Nvfp4MaxProduct)).AsType(out.DType())
+```
+
+**`f32(f32(m × 2688) / 2688)` is not `m`.** Checked against the checkpoints in #312: it
+returns `m` exactly for both of 12b's vision global scales and misses by one float32 ulp
+for **17 of 31b's 191**. One ulp in a per-tensor scale flips bf16 roundings of that
+layer's output, and 27 encoder layers grow it into something the golden test sees: on
+MLX-CUDA the 31b vision encoder moved `max sampled Δ 0.0898 → 0.1094` and
+`norm_mean 96.999 → 97.056` across the fold. A two-line control on `8a7ba949` that
+restored the old arithmetic reproduced the pre-fold line **to every digit**, so the round
+trip is the whole of the CUDA-side move.
+
+Three wrappers apply the scale themselves, and all three take the divide:
+
+| wrapper | why it applies the scale itself | reached by |
+|---|---|---|
+| `QuantizedMatmul` | `mlx_quantized_matmul` has no global-scale argument | every dense nvfp4 linear — the whole vision tower and the dense language model |
+| `Dequantize` | MLX's argument accepts only a scalar; callers pass per-expert banks | weight inspection, and #287's prefill dequantisation |
+| `GatherQMM` off Metal, with `rhsIndices` | the CUDA path scales gathered rows in the wrapper | nvfp4 MoE experts |
+
+Two paths hand the scale to MLX and genuinely need the `× 2688` form: `GatherQMM` on
+Metal (or without `rhsIndices`), which passes `gs` into `mlx_gather_qmm`, and the fused
+`QQMM` path, whose identity scale is `Nvfp4MaxProduct` itself.
+
+So the stored representation is chosen for the two consumers that want it, and the three
+that do not want it pay a lossy round trip on every call.
+
+## Decision
+
+**Store `m`. Convert to MLX's representation at the call sites that hand the scale to
+MLX, and nowhere else.**
+
+- `ToMLXGlobalScale` keeps the checkpoint multiplier (cast to float32, flattened) and
+  stops multiplying by `Nvfp4MaxProduct`.
+- `scaleAndCast` multiplies by the scale as given and stops dividing.
+- `GatherQMM`'s native branch and `QQMM` multiply by `Nvfp4MaxProduct` where they build
+  the argument MLX consumes; the identity scale stays as it is.
+- A test asserts the round trip: for every global scale in a served checkpoint, the value
+  a wrapper applies is bit-identical to the checkpoint's `m`.
+
+## Options considered
+
+- **Leave it.** The error is one ulp and every bound the golden test checks still passes.
+  Rejected: it is a silent, permanent divergence from the reference implementation on
+  every dense nvfp4 linear, it made a pin move look like an encoder regression once
+  already (#312), and it costs nothing to remove.
+- **Restore the pre-fold arithmetic everywhere** (drop `× 2688` entirely). This is the
+  control used in #312 and it does reproduce the old numbers, but it breaks the MoE paths
+  that hand the scale to MLX. Rejected.
+- **Store both forms.** Explicit, and two arrays per scale plus a rule about which to
+  reach for. Rejected as more surface than the problem warrants: the conversion is one
+  multiply at two call sites.
+- **Keep the MLX form and make `scaleAndCast` exact** by folding `1/2688` into the
+  multiply as a double. Rejected: it fixes the arithmetic at the point of use while
+  leaving a stored value that is not what any checkpoint says, so the next wrapper added
+  inherits the trap.
+
+## Consequences
+
+- On CUDA the vision encoder returns to bit-parity with the pre-fold build and with
+  `mlx-vlm`, which is what the goldens in `x/mlxrunner/testdata` were generated against.
+  The 31b golden expectations move back (`max Δ 0.1094 → 0.0898`) and should be re-taken
+  in the same change.
+- **#287 inherits the fix.** Its prefill path dequantises nvfp4 weights to bf16 through
+  `Dequantize`, so today it would bake the ulp error into the dequantised weights for the
+  whole prefill. With this decision, dequantised prefill and quantised decode apply the
+  same scale, which removes one objection to that PR. It does not address the others: the
+  PR still changes output deterministically and costs +6.8 GiB peak, and it stays held on
+  those grounds.
+- Metal is unaffected in behaviour: its `GatherQMM` still receives `m × 2688`. Its 26b and
+  31b encoder numbers move for the unrelated reason in #312 (mlx#3912), and this change
+  does not interact with that.
+- One risk, and it is the reason for the test: a call site that hands the scale to MLX and
+  is missed would silently apply a scale 2688× too small. The failure is loud in output
+  quality, not in a crash.