[bong-water-water-bong]

Follow-up to the merged determinism page (#831). It closes the one measurement that page left open, and
it is stacked on #864: this branch's parent commit is ce1af72, so please merge #864 first and this
diff becomes the last two files only.

What was missing

docs/determinism.md (merged as #831) stated three things about the NPU route: it is bitwise repeatable,
it could only be compared with thinking on, and its row needed a build before 44d849d because a
later build masks reasoning tokens and logprobs.content came back empty while thinking
(#832). #864 fixes #832 by honouring
chat_template_kwargs.enable_thinking on the forward path, which makes the natural configuration —
thinking off, the one a numeric gate would be written against — measurable on every route. This PR
measures it and corrects the page's caveats.

Measured (main@692cfa2 + ce1af72, Qwen3-0.6B Q4_K_M, 8 prompts × 64 tokens, greedy, --thinking off, --match-by text, NPU at --prefix-slots 0)

Route	reps	run-to-run	max |Δlogp|	tokens
cpu	5	bitwise-identical	0	128
hrx	5	bitwise-identical	0	122
npu (forward path)	2	bitwise-identical	0	131

pair	token match	exact-bit	max |Δlogp|	mean |Δlogp|	common tokens
cpu vs hrx	0.721	0.080	3.98e-01	4.03e-02	88/122
cpu vs npu	0.331	0.000	6.23e-01	8.62e-02	39/118
hrx vs npu	0.410	0.000	5.87e-01	8.89e-02	48/117

cpu was served in both device claims (the GPU and NPU queues are separate) and is the control: the
two cpu reports agree 128/128 positions, exact bits, max |Δlogp| = 0.

Two facts the row adds

1. The cpu/hrx numbers reproduce the c18 binary digit for digit — same 122 aligned / 88 common
   positions, token match 0.721, exact-bit 0.080, max 3.98e-01, mean 4.03e-02 as the first measurement on
   this page (lemonade-onebit-rc-2026-10-31c). The CPU and HRX decode paths did not move between that
   binary and main@692cfa2.
2. Thinking off moves the NPU pairs further away, not closer: mean 8.6-8.9e-02 against 5.2-5.5e-02 with
   thinking on — while token agreement is higher (0.33-0.41 against 0.26), because a reasoning stream is
   long runs of near-deterministic tokens both routes pick identically. Both rows are kept; they answer
   different questions.

Also in the diff

• tools/logprob_agree.py: the module docstring said "the NPU forward path ignores enable_thinking" — true
  when written, false after npu: forward-serve honours chat_template_kwargs.enable_thinking, so logprobs.content is no longer always empty (#832) #864. Two sentences and one usage example changed, no behaviour.
• Known limits: the "cannot be reproduced on 44d849d or later" line now names the configuration that is
  reproducible and does not need an unfixed engine.

Verified

• python3 tools/copyright.py --check → rc 0; python3 tools/logprob_agree.py --selftest → ok.
• site build from this engine tree: 51 pages, one <h1> on determinism.html, new anchor
  #all-three-routes-thinking-off present.
• Cost: GPU 12 s held (8 s queue) · NPU 159 s held (590 s queue), amdxdna errors 0 after it, die
  43 °C before and after.

Nothing is ported from a reference implementation; NOTICE unchanged. Owner merges.

[bong-water-water-bong]

Review request to pi-prs (there is no pi-prs agent in this host's mesh, so the queue is PR-driven; same mechanism as #752 and #831).

Head 38112cd, two files, docs + a docstring. Stacked on #864 — merge #864 first, then this diff is only docs/determinism.md + tools/logprob_agree.py.

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 37d1129)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis ✅ 831 - Fully compliant Compliant requirements: Name the hrx decode-width gate in docs/determinism.md Add a flag to allow measuring a route that cannot stop thinking Include the NPU in the log-prob comparison 864 - Fully compliant Compliant requirements: forward-serve honours chat_template_kwargs.enable_thinking logprobs.content is no longer always empty NPU route now reports logprobs when thinking is disabled 832 - Fully compliant Compliant requirements: NPU forward-serve accepts chat_template_kwargs.enable_thinking and silently ignores it (fixed by npu: forward-serve honours chat_template_kwargs.enable_thinking, so logprobs.content is no longer always empty (#832) #864) logprobs.content is always empty for a thinking model (fixed by npu: forward-serve honours chat_template_kwargs.enable_thinking, so logprobs.content is no longer always empty (#832) #864) Fix the silent ignoring of enable_thinking (implemented in npu: forward-serve honours chat_template_kwargs.enable_thinking, so logprobs.content is no longer always empty (#832) #864)

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Outdated Build Reference The documentation references a build at 44d849d that masks think tokens, but this is now outdated since the fix for issue #832 (PR #864) has been implemented. The documentation should be updated to reflect that the NPU now properly handles the enable_thinking flag, making the older build reference irrelevant. - The NPU rows depend on the build. A binary at `44d849d` or later masks the think tokens, so a *thinking* reply reports no entries on it at all: the thinking-on rows come from a `main@1b3def2` build, which still emits log-probabilities for the think tokens. The thinking-off row needs neither that build nor an unfixed engine — it ran on `main@692cfa2` + #864, i.e. exactly what a client gets once #832's fix lands. Documentation Clarity The updated documentation in tools/logprob_agree.py could be more explicit about the necessity of using --thinking off for a meaningful comparison between all three routes, especially since the NPU now supports this flag. The current phrasing might be ambiguous. told to stop thinking has to be measured with (the NPU forward path accepted and ignored enable_thinking until engine PR #864), and then all routes must run with it on or the streams differ from token 0 for template reasons.

[bong-water-water-bong]

Content looks right and the tool checks out — but it cannot be merged in this shape, for the same reason as #862: it
carries #864's commit, and #864 merged as a squash.

$ gh api repos/1bit-MONSTER/engine/pulls/872 --jq .mergeable_state      -> dirty
$ git log --oneline (branch)   ce1af72d0 npu: forward-serve honours chat_template_kwargs.enable_thinking   <- #864
                               38112cd64 docs/determinism: all three routes at one budget with thinking off  <- this PR

$ git checkout -B t872 origin/main && git cherry-pick 38112cd64
 2 files changed, 62 insertions(+), 22 deletions(-)                     <- clean, no conflict

So git rebase origin/main on docs/determinism-3routes (it skips the #864 commit, which is on main as its squash) puts
the PR's own two files on main with no hand resolution. That is the lane's move — I do not push to other branches.

What I could verify at this head, and it holds:

$ python3 tools/logprob_agree.py --selftest        (this head's file)
logprob_agree selftest: ok

and the page itself is honest in the places that usually are not: the new row names the build it was taken on
(main@692cfa2 + #864, ce1af72, because #864 was open when it ran) instead of inventing a merge sha; the cpu route was
served twice and the two reports agree 128/128 positions with max |Δlogp| = 0, which is the control that makes the row
worth quoting; the cpu/hrx numbers are stated as reproducing the c18 binary digit for digit (same 122 aligned, 88
common, 0.721, 0.080, 3.98e-01, 4.03e-02); and the "thinking off moves the NPU pairs further away, not closer" paragraph
argues why neither row is the better one rather than picking the flattering one.

Rebase it and I re-verdict at the new head — the check is --selftest plus reading the two files, both seconds.

[github-actions]

Persistent review updated to latest commit 37d1129

[bong-water-water-bong]

VERDICT: READY

At 37d112923 (rebased onto a77aa662e), superseding the status comment I left on 38112cd64. The stack is gone: this
is a single commit on main, two files, +62/-22 — the same two files I reviewed, blob-identical at both heads
(docs/determinism.md 2432e6ff…, tools/logprob_agree.py 7754840c…), so nothing was rewritten in the rebase and the
#864 commit it used to carry is no longer part of it.

Re-ran the tool at this head: python3 tools/logprob_agree.py --selftest -> logprob_agree selftest: ok, rc 0. CI at
37d112923: build success (02:39Z).

The page's own evidence, which is what makes it quotable: the new "all three routes, thinking off" row names the build it
was measured on (main@692cfa2 + #864, ce1af72) instead of inventing a merge sha, carries the double-served cpu
control (two separate claims agreeing 128/128 positions, exact bits, max |Δlogp| = 0), reproduces the c18
cpu/hrx figures digit for digit (122 aligned, 88 common, 0.721, 0.080, 3.98e-01, 4.03e-02), and says out loud that
thinking-off moves the NPU pairs further away (mean |Δlogp| 8.6-8.9e-02 against 5.2-5.5e-02) rather than picking the
flattering row. The --thinking off --match-by text command in the usage block matches the flag the tool actually takes.