From f5f08383375d7d83c6ba4b6bda6003e0e190a000 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 20 Oct 2026 18:08:58 -0300 Subject: [PATCH 1/4] Pin llama.cpp 8792923f: the SET_ROWS add-on hook the private kernels need (census G1) gpu-kernels be68256d (#476) compiles addons/hip-hrx/dispatch-hip-set-rows-q8.cpp against HipSetRowsArgs, which only the fork branch 1bit/hrx-set-rows-q8 had: with the private kernels the release build failed at dispatch-hip-set-rows-q8.cpp:39. Fork #116 squash-merged that branch onto 1bit/hrx-vulkan-patched (8792923f8, the same tree as the head proven here). Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- third_party/llama.cpp | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 6a13c31c..06b8dd39 100644 --- a/README.md +++ b/README.md @@ -61,7 +61,7 @@ The engine has no standalone package of its own. > [1bit-MONSTER/lemonade](https://github.com/1bit-MONSTER/lemonade)) passes Lemonade's own LLM test > suite on HRX. Following a review,[^geramyl] the engine no longer > vendors Lemonade: Lemonade is the host, 1bit is the engine inside it. Ported so far: HRX on AMD's live ggml-hrx, -> on AMD's `e3e6a976` hrx-system line since 2026-10-18, now hrx-system `82804197` with llama.cpp `00f2a68de` (`b14a28362` since 2026-10-20: a stall-aware `/health` and an exit on a lost GPU for [one hop from Lemonade](docs/serve.md#one-hop-from-lemonade---print-backend), plus the census-gap fixes G4-G6) +> on AMD's `e3e6a976` hrx-system line since 2026-10-18, now hrx-system `82804197` with llama.cpp `00f2a68de` (`b14a28362` since 2026-10-20: a stall-aware `/health` and an exit on a lost GPU for [one hop from Lemonade](docs/serve.md#one-hop-from-lemonade---print-backend), plus the census-gap fixes G4-G6; `8792923f8` since 2026-10-20: the SET_ROWS hook the private kernels' q8_0 KV cache needs, census G1) > (2026-10-19, #458, #461; our four fixes replayed plus our long-context live-out fix for ROCm/hrx-system#1535; > measured when the line moved on 2026-10-18 with llama.cpp `8d9ce2c1a`, #441, against the old pins on Strix Halo: > test-backend-ops 1141/1141, long-prompt needles 20/20 + 5/5 + 5/5, KLD 0 and identical output on two diff --git a/third_party/llama.cpp b/third_party/llama.cpp index b14a2836..8792923f 160000 --- a/third_party/llama.cpp +++ b/third_party/llama.cpp @@ -1 +1 @@ -Subproject commit b14a283625d921d41477a866d10b9e356aafcfd4 +Subproject commit 8792923f8ba99637c562a52523a9fb831f57edfe From 2669c70266d2c79a48c151659dac322f5272654f Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 20 Oct 2026 18:08:58 -0300 Subject: [PATCH 2/4] registry: regenerated at llama.cpp 8792923f Co-Authored-By: Claude Opus 5.5 --- registry/architectures.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/registry/architectures.json b/registry/architectures.json index dec40df8..c40264b3 100644 --- a/registry/architectures.json +++ b/registry/architectures.json @@ -1,7 +1,7 @@ { "about": "HF architecture -> GGUF architecture and the backends whose code accepts it. Generated by tools/registry_build.py from the pinned sources; do not edit.", "sources": { - "llama.cpp (hrx)": "b14a283625d921d41477a866d10b9e356aafcfd4" + "llama.cpp (hrx)": "8792923f8ba99637c562a52523a9fb831f57edfe" }, "counts": { "hrx": 292, From f2be44c2e03dc5c233525534475fc9baec81faef Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 20 Oct 2026 18:12:07 -0300 Subject: [PATCH 3/4] hw-gate: build the private HIP kernels, as the release does The gate built llama.cpp without the gpu-kernels add-on, so it passed engine main 1834ceb5, whose pins do not compile together with the private kernels (dispatch-hip-set-rows-q8.cpp:39, HipSetRowsArgs). The build phase now fetches the add-on at the pinned commit (scripts/fetch-private.sh, the runner's gh login) and passes GGML_HRX_HIP_ADDON_DIR like cmake/hrx.cmake; it reconfigures when the gpu-kernels pin moves, fails when libggml-hrx.so lacks ggml_hrx_hip_addon_register, and prints only the compiler's error lines (the job log is public). A box without read access stops with exit 2; HW_GATE_PRIVATE=0 gates the public part. Co-Authored-By: Claude Opus 5.5 --- docs/hw-gate.md | 13 ++++++++---- scripts/hw-gate.sh | 53 ++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 53 insertions(+), 13 deletions(-) diff --git a/docs/hw-gate.md b/docs/hw-gate.md index 9ada3fa5..d8481fe2 100644 --- a/docs/hw-gate.md +++ b/docs/hw-gate.md @@ -36,7 +36,15 @@ PRs from forks never run there: the runner is the owner's box. `scripts/hw-gate.sh` builds this checkout's HRX pair (`third_party/llama.cpp` and `third_party/hrx-system`) with the configuration of `cmake/hrx.cmake`, plus `test-backend-ops` -and `llama-perplexity`. It then runs four checks: +and `llama-perplexity`, with the private HIP kernels compiled in as the release builds them: the +gpu-kernels add-on at the commit `config/gpu-kernels.config` pins, fetched by +`scripts/fetch-private.sh` with the runner box's `gh` login and passed as +`GGML_HRX_HIP_ADDON_DIR`. A llama.cpp pin and a gpu-kernels pin that do not compile together fail +the build step (2026-10-20: gpu-kernels be68256d used `HipSetRowsArgs`, which the pinned +llama.cpp did not have yet), and so does a build whose `libggml-hrx.so` lacks the add-on. A box +that cannot read gpu-kernels stops with exit 2; `HW_GATE_PRIVATE=0` gates the public plumbing +only. On a failed build the job log shows the compiler's error lines without their source +excerpts, since the log is public and the add-on's sources are not. It then runs four checks: | check | what | catches | |---|---|---| @@ -175,9 +183,6 @@ That is the usual GPU-vs-CPU rounding gap of a 4-bit file, not a wrong kernel. - **Lock fairness.** `~/npu.lock` is a plain flock, so a job that re-queues quickly can starve the gate for an hour. A first-come-first-served queue, or a reserved window for the gate, would bound the wall time. The gate's own GPU time is 6 to 13 minutes. -- **Private kernels.** The gate builds the public HRX plumbing only. The private HIP add-on - (gpu-kernels, `ONEBIT_GPU_PRIVATE`) is not built or checked, because the runner serves a public - repository. - **Not exercised:** `1bit serve` itself (the gate talks to llama-server with serve's HRX flags), `--parallel`, `--mtp` / `--dflash`, vision, and contexts past 8K. - **One copy of the KLD bases** on the box. A spare copy of the release-pin set is on ryzen diff --git a/scripts/hw-gate.sh b/scripts/hw-gate.sh index 7fcc46ef..7d729b8b 100755 --- a/scripts/hw-gate.sh +++ b/scripts/hw-gate.sh @@ -15,8 +15,10 @@ # limitations under the License. # # hw-gate.sh: the HRX correctness gate on real hardware (docs/hw-gate.md). It builds this -# checkout's HRX llama.cpp pair (third_party/llama.cpp + third_party/hrx-system) and checks it on -# the Radeon iGPU against goldens recorded at the release pin: +# checkout's HRX llama.cpp pair (third_party/llama.cpp + third_party/hrx-system) with the private +# HIP kernels compiled in, as the release does (the gpu-kernels add-on at the commit +# config/gpu-kernels.config pins, fetched by scripts/fetch-private.sh), and checks it on the Radeon +# iGPU against goldens recorded at the release pin: # # ops test-backend-ops -b HRX0 on the ops the HRX correctness bugs touched # kat tools/kat.py check: known answers, determinism, cached prefix, varied lengths and drift @@ -34,6 +36,9 @@ # tests/golden/hw-gate and the base logits under $HW_GATE_STORE/kld (manifest.json # keeps their sha256). Only at a release pin. # +# HW_GATE_PRIVATE=0 build the public HRX plumbing only, without the private kernels (a box with +# no read access to gpu-kernels). The default builds them: a fetch failure is exit 2. +# # Every GPU step holds ~/npu.lock (flock) only while it runs, inside scripts/mem-guard.sh and # ~/lb/thermal-run.sh. Before each one it waits for the PR-Agent model (coder-llm, :8090) to be # idle, and afterwards it reports any new amdgpu ring timeout (HRX dispatches starve that @@ -50,6 +55,7 @@ MEM_FLOOR=${HW_GATE_MEM_FLOOR:-12} CODER_URL=${HW_GATE_CODER_URL:-http://127.0.0.1:8090} CODER_WAIT=${HW_GATE_CODER_WAIT:-900} TOOLCHAIN=${ONEBIT_HRX_TOOLCHAIN:-/opt/rocm-therock} +PRIVATE=${HW_GATE_PRIVATE:-1} TEXT=${HW_GATE_TEXT:-$MODELS/qwen38/wikitext-2-raw/wiki.test.raw} PORT=${HW_GATE_PORT:-18471} JOBS=${HW_GATE_JOBS:-4} @@ -226,8 +232,9 @@ note "hw-gate $MODE at $(git -C "$ROOT" rev-parse --short HEAD 2>/dev/null): lla # ---- build --------------------------------------------------------------------------------- if [ -z "$BIN" ]; then src=$ROOT/third_party/llama.cpp - # one persistent build tree per source path, so the runner's checkout rebuilds incrementally - key=$(printf '%s' "$src" | sha1sum | cut -c1-10) + # one persistent build tree per source path (and per add-on choice), so the runner's checkout + # rebuilds incrementally + key=$(printf '%s|private=%s' "$src" "$PRIVATE" | sha1sum | cut -c1-10) build=$STORE/build-$key BIN=$build/bin if has build; then @@ -237,28 +244,56 @@ if [ -z "$BIN" ]; then [ -f "$ROOT/third_party/$s/CMakeLists.txt" ] || { note "::error::third_party/$s is not checked out"; exit 2; } done t0=$(date +%s) + # the private HIP kernels, as cmake/hrx.cmake (ONEBIT_GPU_PRIVATE) and scripts/weekly.sh build + # them: the gpu-kernels add-on at the pinned commit, through llama.cpp's GGML_HRX_HIP_ADDON_DIR. + # A llama.cpp pin and a gpu-kernels pin that do not compile together fail the gate here (the + # 2026-10-20 break: the add-on used HipSetRowsArgs, which the pinned llama.cpp did not have). + addon_args=() addon_pin=none + if [ "$PRIVATE" = 1 ]; then + addon_pin=$(sed -n 's/^sha=//p' "$ROOT/config/gpu-kernels.config" | tail -n 1) + if ! addon=$("$ROOT/scripts/fetch-private.sh" "$ROOT/config/gpu-kernels.config" "$STORE/gpu-kernels" 2> "$OUT/private.log"); then + note "::error::cannot fetch the private gpu-kernels add-on at ${addon_pin:0:12} (gh login with read access on this box?): $(tail -n 1 "$OUT/private.log")" + note " (HW_GATE_PRIVATE=0 gates the public plumbing only)" + exit 2 + fi + grep -q GGML_HRX_HIP_ADDON_DIR "$src/ggml/src/ggml-hrx/hip/ggml-hrx-hip.cmake" 2>/dev/null || + { note "::error::third_party/llama.cpp has no GGML_HRX_HIP_ADDON_DIR hook (needs the HIP plumbing)"; exit 2; } + addon_args=(-DGGML_HRX_HIP_ADDON_DIR="$addon") + note "private HIP kernels: gpu-kernels ${addon_pin:0:12} (addons/hip-hrx)" + fi + # configure once per add-on pin (a new pin can add kernel files the old configure never saw); + # otherwise ninja re-runs cmake itself when a CMakeLists.txt changes + [ "$(cat "$build/.gpu-kernels-pin" 2>/dev/null)" = "$addon_pin" ] || rm -f "$build/CMakeCache.txt" # the same configuration as cmake/hrx.cmake (llama_hrx), plus the two test tools : > "$OUT/build.log" built=1 # thermal-run kills a build at 93 C (the box is shared); ninja resumes where it stopped for attempt in 1 2 3 4 5 6; do cool - # configure once; afterwards ninja re-runs cmake itself when a CMakeLists.txt changes - { [ -f "$build/CMakeCache.txt" ] || "$THERMAL" cmake -S "$src" -B "$build" -G Ninja -DCMAKE_BUILD_TYPE=Release \ + { [ -f "$build/CMakeCache.txt" ] || { "$THERMAL" cmake -S "$src" -B "$build" -G Ninja -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_C_COMPILER="$TOOLCHAIN/bin/amdclang" -DCMAKE_CXX_COMPILER="$TOOLCHAIN/bin/amdclang++" \ -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ - -DHRX_SOURCE_DIR="$ROOT/third_party/hrx-system" \ + -DHRX_SOURCE_DIR="$ROOT/third_party/hrx-system" "${addon_args[@]}" \ -DGGML_HRX=ON -DGGML_VULKAN=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_NATIVE=ON -DGGML_CPU=ON \ - -DLLAMA_BUILD_SERVER=ON -DLLAMA_BUILD_TESTS=ON -DLLAMA_CURL=OFF >> "$OUT/build.log" 2>&1; } && + -DLLAMA_BUILD_SERVER=ON -DLLAMA_BUILD_TESTS=ON -DLLAMA_CURL=OFF >> "$OUT/build.log" 2>&1 && + echo "$addon_pin" > "$build/.gpu-kernels-pin"; }; } && "$THERMAL" cmake --build "$build" -j "$JOBS" --target llama-server test-backend-ops llama-perplexity >> "$OUT/build.log" 2>&1 && { built=0; break; } grep -q "error:" <(tail -n 40 "$OUT/build.log") && break # a compile error, not heat note "build stopped (attempt $attempt, Tctl $(tctl) C); resuming" done + # (grep -c, not grep -q: under pipefail grep's early exit would fail nm's side of the pipe) + if [ $built = 0 ] && [ "$PRIVATE" = 1 ] && + [ "$(nm -D --defined-only "$BIN/libggml-hrx.so" 2>/dev/null | grep -c ggml_hrx_hip_addon_register)" = 0 ]; then + note "::error::the build passed but libggml-hrx.so does not export ggml_hrx_hip_addon_register: the private kernels are not in it" + built=1 + fi if [ $built = 0 ]; then result build PASS $(( $(date +%s) - t0 )) "$build" else result build FAIL $(( $(date +%s) - t0 )) "see $OUT/build.log" - tail -n 30 "$OUT/build.log" + # the compiler's error lines only, not its source excerpts: the job log is public and the + # add-on's sources are not + grep -E '(error|Error)[: ]' "$OUT/build.log" | grep -v -E '^ *[0-9]+ \|' | head -n 20 exit 2 fi fi From 9623df0bd15b6df7024740a70a8b9b3727f7895b Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 20 Oct 2026 19:27:47 -0300 Subject: [PATCH 4/4] hw-gate: re-record the goldens and KLD bases with the private HIP kernels The gate now builds llama.cpp with the gpu-kernels add-on (be68256d), as the release does, so the kat goldens and the KLD base logits are recorded from that build (llama.cpp 8792923f, hrx-system 82804197). Recorded with scripts/hw-gate.sh --record --phases kat,kld,needle on strixhalo (performance mode), from the binaries of hw-gate run 39498508232. Why the old goldens no longer fit (ZAYA1-8B Q4_K_M, wiki.test 8 x 512, performance mode): KLD vs the CPU (--device none): public build 0.0381 (p99 0.431, same top 92.45%) add-on build 0.0357 (p99 0.384, same top 92.60%) KLD vs the F16 model on the CPU: public 0.5798, add-on 0.5816 (+-0.019), CPU Q4_K_M 0.5986 add-on vs the old (public) base: 0.0069; hip.mul_mat_id_prefill alone 0.0056, the dense prefill tile (ragged lm_head) alone 0.0003 The kat failure was prefix-reuse token 0, a near-tie: CPU: green -0.942, The -0.964 public: The -0.931, green -1.068 (old golden) add-on: green -0.860, The -1.060 (new golden) The add-on now picks the same token the CPU picks. Kernel checks: hrx_prefill_mmid q4_K/q6_K on the ZAYA expert tensors (T 512 and 135, top-1) match the f16-activation reference to 7e-7; the ragged lm_head (q6_K 2048 x 262147, T 1..512) has the same NMSE vs the CPU in its last 3 columns as in the rest (~5e-5), 2 repeats each; test-backend-ops MUL_MAT_ID 132/132, 3 runs. Base sha256 changes: zaya1-8b, qwen3.8-27b, qwen3-4b-iq4xs. Qwen3-0.6B and Qwen3-Coder-30B-A3B record the same bytes as before. Only one kat text changed (zaya1-8b prefix-reuse). Co-Authored-By: Claude Opus 5.5 --- tests/golden/hw-gate/kat-qwen3-0.6b.json | 734 ++++++++------- tests/golden/hw-gate/kat-qwen3-4b-iq4xs.json | 264 +++--- .../hw-gate/kat-qwen3-coder-30b-a3b.json | 266 +++--- tests/golden/hw-gate/kat-qwen3.8-27b.json | 394 ++++---- tests/golden/hw-gate/kat-zaya1-8b.json | 864 ++++++++---------- tests/golden/hw-gate/manifest.json | 16 +- 6 files changed, 1224 insertions(+), 1314 deletions(-) diff --git a/tests/golden/hw-gate/kat-qwen3-0.6b.json b/tests/golden/hw-gate/kat-qwen3-0.6b.json index 3e307c6b..09e77dca 100644 --- a/tests/golden/hw-gate/kat-qwen3-0.6b.json +++ b/tests/golden/hw-gate/kat-qwen3-0.6b.json @@ -18,28 +18,28 @@ }, { "t": ".", - "lp": -0.41474634408950806, + "lp": -0.36700165271759033, "top": [ - -0.41474634408950806, - -1.0827189683914185, - -8.467513084411621 + -0.36700165271759033, + -1.1829272508621216, + -8.497456550598145 ] }, { "t": "", - "lp": -3.635949542513117e-05, + "lp": -3.922062387573533e-05, "top": [ - -3.635949542513117e-05, - -10.48684310913086, - -12.650276184082031 + -3.922062387573533e-05, + -10.394368171691895, + -12.614109992980957 ] } ], "expect_ok": true, "secs": [ - 0.72, - 0.04, - 0.03 + 0.35, + 0.03, + 0.02 ], "fails": [], "warns": [], @@ -63,29 +63,29 @@ }, { "t": "7", - "lp": -4.303548121242784e-05, + "lp": -4.1604907892178744e-05, "top": [ - -4.303548121242784e-05, - -11.50451374053955, - -12.035765647888184 + -4.1604907892178744e-05, + -11.55091667175293, + -12.038793563842773 ] }, { "t": " \u00d7", - "lp": -0.684101939201355, + "lp": -0.6996909379959106, "top": [ - -0.684101939201355, - -0.8162944912910461, - -2.9468867778778076 + -0.6996909379959106, + -0.8083697557449341, + -2.868718385696411 ] }, { "t": " ", - "lp": -1.1920935776288388e-06, + "lp": -1.4305124977909145e-06, "top": [ - -1.1920935776288388e-06, - -13.725953102111816, - -16.01691436767578 + -1.4305124977909145e-06, + -13.5270357131958, + -15.898425102233887 ] }, { @@ -93,44 +93,44 @@ "lp": -1.1920928955078125e-07, "top": [ -1.1920928955078125e-07, - -16.4316349029541, - -17.14297866821289 + -16.409690856933594, + -17.002517700195312 ] }, { "t": "3", - "lp": -3.576279254957626e-07, + "lp": -2.3841860752327193e-07, "top": [ - -3.576279254957626e-07, - -15.81689453125, - -16.072751998901367 + -2.3841860752327193e-07, + -15.875801086425781, + -16.12158203125 ] }, { "t": " =", - "lp": -0.00042024144204333425, + "lp": -0.0004508915590122342, "top": [ - -0.00042024144204333425, - -8.281874656677246, - -9.56419849395752 + -0.0004508915590122342, + -8.398053169250488, + -9.264424324035645 ] }, { "t": " ", - "lp": -0.5769420266151428, + "lp": -0.6076502799987793, "top": [ - -0.5769420266151428, - -0.8264861702919006, - -7.965495586395264 + -0.6076502799987793, + -0.7885909080505371, + -7.849595546722412 ] }, { "t": "3", - "lp": -0.09963735193014145, + "lp": -0.05225289985537529, "top": [ - -0.09963735193014145, - -2.373662233352661, - -7.078593730926514 + -0.05225289985537529, + -3.008761405944824, + -7.128914833068848 ] }, { @@ -138,43 +138,43 @@ "lp": 0.0, "top": [ 0.0, - -17.484418869018555, - -17.624372482299805 + -17.330638885498047, + -17.870128631591797 ] }, { "t": "1", - "lp": -1.2159421203250531e-05, + "lp": -1.2517053619376384e-05, "top": [ - -1.2159421203250531e-05, - -11.550372123718262, - -14.052669525146484 + -1.2517053619376384e-05, + -11.484469413757324, + -14.135736465454102 ] }, { "t": ".", - "lp": -0.33265331387519836, + "lp": -0.29199305176734924, "top": [ - -0.33265331387519836, - -1.47267746925354, - -3.3495523929595947 + -0.29199305176734924, + -1.594797968864441, + -3.397446393966675 ] }, { "t": "", - "lp": -0.21032550930976868, + "lp": -0.20733511447906494, "top": [ - -0.21032550930976868, - -1.852882742881775, - -4.622714996337891 + -0.20733511447906494, + -1.8719221353530884, + -4.722168445587158 ] } ], "expect_ok": true, "secs": [ - 0.41, - 0.07, - 0.07 + 0.42, + 0.06, + 0.06 ], "fails": [], "warns": [], @@ -189,20 +189,20 @@ "tokens": [ { "t": "```", - "lp": -0.07233511656522751, + "lp": -0.074213407933712, "top": [ - -0.07233511656522751, - -3.1589534282684326, - -3.8045852184295654 + -0.074213407933712, + -3.119396686553955, + -3.807680606842041 ] }, { "t": "python", - "lp": -8.464217535220087e-05, + "lp": -8.356920443475246e-05, "top": [ - -8.464217535220087e-05, - -9.481410026550293, - -12.286858558654785 + -8.356920443475246e-05, + -9.490337371826172, + -12.238492965698242 ] }, { @@ -210,44 +210,44 @@ "lp": -5.602852525044e-06, "top": [ -5.602852525044e-06, - -12.63437271118164, - -13.464797973632812 + -12.71084976196289, + -13.272350311279297 ] }, { "t": "def", - "lp": -0.0021713089663535357, + "lp": -0.002708224579691887, "top": [ - -0.0021713089663535357, - -6.394682884216309, - -7.971531867980957 + -0.002708224579691887, + -6.118144989013672, + -7.963693141937256 ] }, { "t": " add", - "lp": -2.5033982637978625e-06, + "lp": -2.2649790025752736e-06, "top": [ - -2.5033982637978625e-06, - -14.080618858337402, - -14.155436515808105 + -2.2649790025752736e-06, + -14.097787857055664, + -14.165239334106445 ] }, { "t": "(a", - "lp": -9.179157132166438e-06, + "lp": -9.059946933120955e-06, "top": [ - -9.179157132166438e-06, - -12.711084365844727, - -13.064924240112305 + -9.059946933120955e-06, + -12.720630645751953, + -12.963784217834473 ] }, { "t": ",", - "lp": -0.00013304641470313072, + "lp": -0.00014866502897348255, "top": [ - -0.00013304641470313072, - -8.936448097229004, - -13.74870491027832 + -0.00014866502897348255, + -8.821813583374023, + -13.933942794799805 ] }, { @@ -255,35 +255,35 @@ "lp": -1.1920928955078125e-07, "top": [ -1.1920928955078125e-07, - -16.05901336669922, - -16.940807342529297 + -16.159643173217773, + -17.043785095214844 ] }, { "t": "):\n", - "lp": -0.00044981820974498987, + "lp": -0.0004782628093380481, "top": [ - -0.00044981820974498987, - -8.137142181396484, - -9.365777969360352 + -0.0004782628093380481, + -8.039897918701172, + -9.306358337402344 ] }, { "t": " ", - "lp": -1.5139694369281642e-05, + "lp": -1.430521751899505e-05, "top": [ - -1.5139694369281642e-05, - -11.22994613647461, - -14.21961784362793 + -1.430521751899505e-05, + -11.302861213684082, + -14.144652366638184 ] }, { "t": " return", - "lp": -0.00036842457484453917, + "lp": -0.0003531602560542524, "top": [ - -0.00036842457484453917, - -8.48416519165039, - -8.978675842285156 + -0.0003531602560542524, + -8.486595153808594, + -9.074979782104492 ] }, { @@ -291,17 +291,17 @@ "lp": -2.861027041944908e-06, "top": [ -2.861027041944908e-06, - -14.276171684265137, - -14.325879096984863 + -14.188737869262695, + -14.306318283081055 ] }, { "t": " +", - "lp": -1.6689314179529902e-06, + "lp": -1.5497220147153712e-06, "top": [ - -1.6689314179529902e-06, - -13.79083251953125, - -15.460498809814453 + -1.5497220147153712e-06, + -13.83828067779541, + -15.48065185546875 ] }, { @@ -309,43 +309,43 @@ "lp": 0.0, "top": [ 0.0, - -16.645889282226562, - -17.334766387939453 + -16.831436157226562, + -17.518150329589844 ] }, { "t": "\n", - "lp": -3.051804378628731e-05, + "lp": -2.5511113562970422e-05, "top": [ - -3.051804378628731e-05, - -10.844993591308594, - -12.523347854614258 + -2.5511113562970422e-05, + -11.063332557678223, + -12.664172172546387 ] }, { "t": "```", - "lp": -0.0008553347433917224, + "lp": -0.0008911883924156427, "top": [ - -0.0008553347433917224, - -7.064629554748535, - -15.928971290588379 + -0.0008911883924156427, + -7.023548603057861, + -16.194631576538086 ] }, { "t": "", - "lp": -2.0742631022585556e-05, + "lp": -2.002736073336564e-05, "top": [ - -2.0742631022585556e-05, - -12.2215576171875, - -13.172750473022461 + -2.002736073336564e-05, + -12.251181602478027, + -13.154234886169434 ] } ], "expect_ok": true, "secs": [ - 0.39, - 0.09, - 0.08 + 0.13, + 0.07, + 0.07 ], "fails": [], "warns": [], @@ -360,145 +360,145 @@ "tokens": [ { "t": "The", - "lp": -0.990650475025177, + "lp": -1.0217280387878418, "top": [ - -0.990650475025177, - -1.585811972618103, - -2.3913938999176025 + -1.0217280387878418, + -1.5583624839782715, + -2.372527599334717 ] }, { "t": " secret", - "lp": -0.06116794794797897, + "lp": -0.06222878396511078, "top": [ - -0.06116794794797897, - -2.9709084033966064, - -5.321564674377441 + -0.06222878396511078, + -2.962523937225342, + -5.21327543258667 ] }, { "t": " code", - "lp": -3.57628505298635e-06, + "lp": -3.099446303167497e-06, "top": [ - -3.57628505298635e-06, - -12.802373886108398, - -14.935531616210938 + -3.099446303167497e-06, + -12.923256874084473, + -15.072945594787598 ] }, { "t": " mentioned", - "lp": -0.04731937125325203, + "lp": -0.04769258573651314, "top": [ - -0.04731937125325203, - -3.0988922119140625, - -8.43028450012207 + -0.04769258573651314, + -3.0899460315704346, + -8.5035982131958 ] }, { "t": " in", - "lp": -0.02918015606701374, + "lp": -0.02988510951399803, "top": [ - -0.02918015606701374, - -3.554337978363037, - -9.774215698242188 + -0.02988510951399803, + -3.530787229537964, + -9.729779243469238 ] }, { "t": " the", - "lp": -0.00011361290671629831, + "lp": -0.00012017018161714077, "top": [ - -0.00011361290671629831, - -9.665959358215332, - -10.977421760559082 + -0.00012017018161714077, + -9.640790939331055, + -10.894123077392578 ] }, { "t": " notes", - "lp": -0.011218536645174026, + "lp": -0.010292447172105312, "top": [ - -0.011218536645174026, - -4.578547954559326, - -9.134464263916016 + -0.010292447172105312, + -4.667534351348877, + -9.101249694824219 ] }, { "t": " is", - "lp": -0.6254276633262634, + "lp": -0.6345151662826538, "top": [ - -0.6254276633262634, - -0.7679943442344666, - -7.9621758460998535 + -0.6345151662826538, + -0.7574818730354309, + -8.125142097473145 ] }, { "t": " ", - "lp": -0.297891765832901, + "lp": -0.29089370369911194, "top": [ - -0.297891765832901, - -1.5851670503616333, - -4.235895156860352 + -0.29089370369911194, + -1.6279278993606567, + -4.1481781005859375 ] }, { "t": "7", - "lp": -0.0010430290130898356, + "lp": -0.001002516015432775, "top": [ - -0.0010430290130898356, - -8.387449264526367, - -8.815513610839844 + -0.001002516015432775, + -8.429205894470215, + -8.839949607849121 ] }, { "t": "3", - "lp": -0.00015426872414536774, + "lp": -0.00017960491823032498, "top": [ - -0.00015426872414536774, - -9.392223358154297, - -10.797174453735352 + -0.00017960491823032498, + -9.171524047851562, + -10.729631423950195 ] }, { "t": "4", - "lp": -7.868122338550165e-05, + "lp": -7.510467548854649e-05, "top": [ - -7.868122338550165e-05, - -9.659936904907227, - -12.080429077148438 + -7.510467548854649e-05, + -9.723271369934082, + -11.994736671447754 ] }, { "t": "1", - "lp": -1.5258905477821827e-05, + "lp": -1.609338323760312e-05, "top": [ - -1.5258905477821827e-05, - -11.3702392578125, - -13.808389663696289 + -1.609338323760312e-05, + -11.308188438415527, + -13.793705940246582 ] }, { "t": ".", - "lp": -0.003160100197419524, + "lp": -0.003237236524000764, "top": [ - -0.003160100197419524, - -5.92175817489624, - -7.7499823570251465 + -0.003237236524000764, + -5.894894123077393, + -7.747490406036377 ] }, { "t": "", - "lp": -0.4836716949939728, + "lp": -0.5121603012084961, "top": [ - -0.4836716949939728, - -1.4637360572814941, - -2.852102756500244 + -0.5121603012084961, + -1.4178800582885742, + -2.820061683654785 ] } ], "expect_ok": true, "secs": [ - 5.59, - 1.27, - 0.78 + 1.03, + 0.56, + 0.51 ], "fails": [], "warns": [], @@ -513,28 +513,28 @@ "tokens": [ { "t": "green", - "lp": -0.06924847513437271, + "lp": -0.07109595090150833, "top": [ - -0.06924847513437271, - -3.1423285007476807, - -4.112621784210205 + -0.07109595090150833, + -3.1195006370544434, + -4.06843900680542 ] }, { "t": "", - "lp": -7.39124880055897e-05, + "lp": -7.41509284125641e-05, "top": [ - -7.39124880055897e-05, - -10.272279739379883, - -10.900975227355957 + -7.41509284125641e-05, + -10.27230453491211, + -10.932613372802734 ] } ], "expect_ok": true, "secs": [ - 17.85, 0.49, - 0.5 + 0.44, + 0.44 ], "fails": [], "warns": [], @@ -549,154 +549,154 @@ "tokens": [ { "t": "The", - "lp": -0.13308687508106232, + "lp": -0.13275077939033508, "top": [ - -0.13308687508106232, - -2.8159520626068115, - -3.9855496883392334 + -0.13275077939033508, + -2.815966844558716, + -3.993701219558716 ] }, { "t": " secret", - "lp": -0.42798399925231934, + "lp": -0.4370582401752472, "top": [ - -0.42798399925231934, - -1.155975103378296, - -3.9068562984466553 + -0.4370582401752472, + -1.1393554210662842, + -3.8899943828582764 ] }, { "t": " code", - "lp": -8.821526535029989e-06, + "lp": -8.225474630307872e-06, "top": [ - -8.821526535029989e-06, - -11.953139305114746, - -14.380997657775879 + -8.225474630307872e-06, + -12.052096366882324, + -14.57041072845459 ] }, { "t": " mentioned", - "lp": -0.0522216260433197, + "lp": -0.0494268536567688, "top": [ - -0.0522216260433197, - -3.0239510536193848, - -7.8129048347473145 + -0.0494268536567688, + -3.0783536434173584, + -7.82710075378418 ] }, { "t": " in", - "lp": -0.04354742541909218, + "lp": -0.04122262820601463, "top": [ - -0.04354742541909218, - -3.1586577892303467, - -10.30580997467041 + -0.04122262820601463, + -3.212317705154419, + -10.37174129486084 ] }, { "t": " the", - "lp": -0.0006705385749228299, + "lp": -0.0006591465207748115, "top": [ - -0.0006705385749228299, - -7.689007759094238, - -9.620882987976074 + -0.0006591465207748115, + -7.718972206115723, + -9.67682933807373 ] }, { "t": " notes", - "lp": -0.03528797999024391, + "lp": -0.037866707891225815, "top": [ - -0.03528797999024391, - -3.4361610412597656, - -8.291898727416992 + -0.037866707891225815, + -3.3625242710113525, + -8.269620895385742 ] }, { "t": " above", - "lp": -0.364200621843338, + "lp": -0.36531028151512146, "top": [ - -0.364200621843338, - -1.1891307830810547, - -8.741870880126953 + -0.36531028151512146, + -1.186490535736084, + -8.851236343383789 ] }, { "t": " is", - "lp": -0.0013732798397541046, + "lp": -0.001320100505836308, "top": [ - -0.0013732798397541046, - -7.205537796020508, - -8.427202224731445 + -0.001320100505836308, + -7.235553741455078, + -8.491373062133789 ] }, { "t": " **", - "lp": -0.2879897654056549, + "lp": -0.2867545783519745, "top": [ - -0.2879897654056549, - -1.9027225971221924, - -3.378676652908325 + -0.2867545783519745, + -1.9194774627685547, + -3.422880172729492 ] }, { "t": "7", - "lp": -0.04951171204447746, + "lp": -0.04764757677912712, "top": [ - -0.04951171204447746, - -4.210581302642822, - -4.468316078186035 + -0.04764757677912712, + -4.2018890380859375, + -4.562099456787109 ] }, { "t": "3", - "lp": -0.005789916031062603, + "lp": -0.006157481577247381, "top": [ - -0.005789916031062603, - -5.1746110916137695, - -9.907683372497559 + -0.006157481577247381, + -5.111582279205322, + -9.919537544250488 ] }, { "t": "4", - "lp": -0.00012267388228792697, + "lp": -0.0001134936828748323, "top": [ - -0.00012267388228792697, - -9.086270332336426, - -12.343253135681152 + -0.0001134936828748323, + -9.167130470275879, + -12.372809410095215 ] }, { "t": "1", - "lp": -5.555307143367827e-05, + "lp": -5.281110861687921e-05, "top": [ - -5.555307143367827e-05, - -9.832464218139648, - -14.406185150146484 + -5.281110861687921e-05, + -9.88016414642334, + -14.585404396057129 ] }, { "t": "**", - "lp": -0.08701317757368088, + "lp": -0.07551011443138123, "top": [ - -0.08701317757368088, - -2.4916038513183594, - -7.573295593261719 + -0.07551011443138123, + -2.6296348571777344, + -7.449678421020508 ] }, { "t": ".", - "lp": -0.0625743567943573, + "lp": -0.0667109340429306, "top": [ - -0.0625743567943573, - -2.816904067993164, - -7.725593566894531 + -0.0667109340429306, + -2.7538046836853027, + -7.754109859466553 ] } ], "expect_ok": true, "secs": [ - 4.67, - 0.11, - 0.1 + 0.25, + 0.09, + 0.08 ], "fails": [], "warns": [], @@ -711,37 +711,37 @@ "tokens": [ { "t": "2", - "lp": -0.8825514912605286, + "lp": -0.8779115676879883, "top": [ - -0.8825514912605286, - -1.02608323097229, - -2.317089319229126 + -0.8779115676879883, + -1.0352754592895508, + -2.30814266204834 ] }, { "t": "5", - "lp": -1.2597204446792603, + "lp": -1.2893261909484863, "top": [ - -1.2597204446792603, - -1.5046106576919556, - -1.5453535318374634 + -1.2893261909484863, + -1.4838298559188843, + -1.5538429021835327 ] }, { "t": "", - "lp": -0.00021740177180618048, + "lp": -0.00021537477732636034, "top": [ - -0.00021740177180618048, - -8.583866119384766, - -11.237703323364258 + -0.00021537477732636034, + -8.599672317504883, + -11.177116394042969 ] } ], "expect_ok": false, "secs": [ - 0.5, - 0.1, - 0.09 + 0.14, + 0.08, + 0.07 ], "fails": [ "run 0: expected ['7341'] got '25'", @@ -760,64 +760,60 @@ "tokens": [ { "t": "7", - "lp": -0.9714764356613159, + "lp": -0.9711244702339172, "top": [ - -0.9714764356613159, - -1.1655434370040894, - -2.0609328746795654 + -0.9711244702339172, + -1.1565911769866943, + -2.0759217739105225 ] }, { "t": "3", - "lp": -0.001368922763504088, + "lp": -0.0011377844493836164, "top": [ - -0.001368922763504088, - -6.750855922698975, - -9.826823234558105 + -0.0011377844493836164, + -6.95065975189209, + -9.911681175231934 ] }, { "t": "4", - "lp": -0.0003919773444067687, + "lp": -0.0004058111517224461, "top": [ - -0.0003919773444067687, - -7.921795845031738, - -10.936928749084473 + -0.0004058111517224461, + -7.888303279876709, + -10.902517318725586 ] }, { "t": "1", - "lp": -1.5974172129062936e-05, + "lp": -1.6808651707833633e-05, "top": [ - -1.5974172129062936e-05, - -11.91186237335205, - -12.535063743591309 + -1.6808651707833633e-05, + -11.854488372802734, + -12.501630783081055 ] }, { "t": "", - "lp": -7.045517122605816e-05, + "lp": -7.164735143305734e-05, "top": [ - -7.045517122605816e-05, - -9.636577606201172, - -13.408830642700195 + -7.164735143305734e-05, + -9.619608879089355, + -13.42975902557373 ] } ], "expect_ok": true, "secs": [ - 0.64, - 0.2, - 0.19 + 0.24, + 0.17, + 0.17 ], "fails": [], - "warns": [ - "cache path: cached request diverges from the fresh one at token 0 (fresh top-2 margin 0.194): 'The secret code mentioned in the notes is 7341. Therefore,'" - ], + "warns": [], "cache_path": { - "first_divergence": 0, - "fresh_margin": 0.19406700134277344, - "cached_text": "The secret code mentioned in the notes is 7341. Therefore," + "first_divergence": null } }, { @@ -827,20 +823,20 @@ "tokens": [ { "t": "The", - "lp": -1.0918020009994507, + "lp": -1.087453007698059, "top": [ - -1.0918020009994507, - -1.2429097890853882, - -2.3347580432891846 + -1.087453007698059, + -1.2486830949783325, + -2.328179359436035 ] }, { "t": " secret", - "lp": -0.09976392239332199, + "lp": -0.09552831202745438, "top": [ - -0.09976392239332199, - -2.452564239501953, - -5.432344436645508 + -0.09552831202745438, + -2.4972217082977295, + -5.463879108428955 ] }, { @@ -848,26 +844,26 @@ "lp": -4.64917320641689e-06, "top": [ -4.64917320641689e-06, - -12.500500679016113, - -14.947443962097168 + -12.509007453918457, + -15.021594047546387 ] }, { "t": " mentioned", - "lp": -0.07294292747974396, + "lp": -0.07308713346719742, "top": [ - -0.07294292747974396, - -2.6735079288482666, - -8.18033504486084 + -0.07308713346719742, + -2.6710216999053955, + -8.270928382873535 ] }, { "t": " in", - "lp": -0.026648126542568207, + "lp": -0.024735843762755394, "top": [ - -0.026648126542568207, - -3.6446805000305176, - -9.554259300231934 + -0.024735843762755394, + -3.717900514602661, + -9.727934837341309 ] }, { @@ -875,106 +871,106 @@ "lp": -7.438936154358089e-05, "top": [ -7.438936154358089e-05, - -10.357091903686523, - -11.331621170043945 + -10.339757919311523, + -11.397640228271484 ] }, { "t": " notes", - "lp": -0.04926704615354538, + "lp": -0.051175232976675034, "top": [ - -0.04926704615354538, - -3.071382761001587, - -8.125473022460938 + -0.051175232976675034, + -3.037135124206543, + -8.030686378479004 ] }, { "t": " is", - "lp": -0.4907863736152649, + "lp": -0.5024116039276123, "top": [ - -0.4907863736152649, - -0.9496391415596008, - -8.761918067932129 + -0.5024116039276123, + -0.9315535426139832, + -8.77120304107666 ] }, { "t": " ", - "lp": -0.3320504128932953, + "lp": -0.32228773832321167, "top": [ - -0.3320504128932953, - -1.521520733833313, - -4.009090423583984 + -0.32228773832321167, + -1.5426876544952393, + -4.046502113342285 ] }, { "t": "7", - "lp": -0.001176632009446621, + "lp": -0.0011224487097933888, "top": [ - -0.001176632009446621, - -8.382768630981445, - -8.600957870483398 + -0.0011224487097933888, + -8.438843727111816, + -8.660470008850098 ] }, { "t": "3", - "lp": -0.0003612693981267512, + "lp": -0.00037092887214384973, "top": [ - -0.0003612693981267512, - -9.214055061340332, - -9.504433631896973 + -0.00037092887214384973, + -9.127333641052246, + -9.570271492004395 ] }, { "t": "4", - "lp": -0.0009862277656793594, + "lp": -0.000912367133423686, "top": [ - -0.0009862277656793594, - -6.983081817626953, - -11.14902114868164 + -0.000912367133423686, + -7.06411600112915, + -11.16539478302002 ] }, { "t": "1", - "lp": -1.8716033082455397e-05, + "lp": -1.7047073924914002e-05, "top": [ - -1.8716033082455397e-05, - -11.191329956054688, - -13.482221603393555 + -1.7047073924914002e-05, + -11.27971076965332, + -13.568235397338867 ] }, { "t": ".", - "lp": -0.005482415668666363, + "lp": -0.006451511289924383, "top": [ - -0.005482415668666363, - -5.312901496887207, - -7.615555286407471 + -0.006451511289924383, + -5.149899482727051, + -7.4468584060668945 ] }, { "t": " Therefore", - "lp": -0.8971348404884338, + "lp": -0.8462240695953369, "top": [ - -0.8971348404884338, - -1.2468243837356567, - -2.3905715942382812 + -0.8462240695953369, + -1.3503382205963135, + -2.3023016452789307 ] }, { "t": ",", - "lp": -3.576279254957626e-07, + "lp": -4.768372718899627e-07, "top": [ - -3.576279254957626e-07, - -14.960407257080078, - -17.71463394165039 + -4.768372718899627e-07, + -14.570320129394531, + -17.527862548828125 ] } ], "expect_ok": true, "secs": [ - 3.98, - 0.48, - 1.43 + 0.43, + 0.35, + 0.34 ], "fails": [], "warns": [], @@ -984,5 +980,5 @@ } ], "pass": false, - "time": "2026-10-12T06:49:19-0300" + "time": "2026-10-20T19:14:58-0300" } \ No newline at end of file diff --git a/tests/golden/hw-gate/kat-qwen3-4b-iq4xs.json b/tests/golden/hw-gate/kat-qwen3-4b-iq4xs.json index ac07c09e..c8aa7c13 100644 --- a/tests/golden/hw-gate/kat-qwen3-4b-iq4xs.json +++ b/tests/golden/hw-gate/kat-qwen3-4b-iq4xs.json @@ -37,7 +37,7 @@ ], "expect_ok": true, "secs": [ - 0.67, + 0.69, 0.1 ], "fails": [], @@ -106,11 +106,11 @@ "tokens": [ { "t": "def", - "lp": -4.1962550312746316e-05, + "lp": -4.386998261907138e-05, "top": [ - -4.1962550312746316e-05, - -10.142623901367188, - -12.860424041748047 + -4.386998261907138e-05, + -10.096399307250977, + -12.855215072631836 ] }, { @@ -118,8 +118,8 @@ "lp": 0.0, "top": [ 0.0, - -24.621891021728516, - -25.648008346557617 + -24.615257263183594, + -25.64767837524414 ] }, { @@ -127,8 +127,8 @@ "lp": 0.0, "top": [ 0.0, - -27.693904876708984, - -28.667917251586914 + -27.694162368774414, + -28.66384506225586 ] }, { @@ -136,8 +136,8 @@ "lp": 0.0, "top": [ 0.0, - -32.068359375, - -33.56993103027344 + -32.0692253112793, + -33.567108154296875 ] }, { @@ -145,8 +145,8 @@ "lp": 0.0, "top": [ 0.0, - -25.56148910522461, - -26.104116439819336 + -25.55023765563965, + -26.098918914794922 ] }, { @@ -154,8 +154,8 @@ "lp": -3.337865791763761e-06, "top": [ -3.337865791763761e-06, - -12.625632286071777, - -20.016849517822266 + -12.618227005004883, + -20.015594482421875 ] }, { @@ -163,8 +163,8 @@ "lp": 0.0, "top": [ 0.0, - -19.573314666748047, - -23.55028533935547 + -19.573650360107422, + -23.548847198486328 ] }, { @@ -172,8 +172,8 @@ "lp": 0.0, "top": [ 0.0, - -19.746116638183594, - -23.387405395507812 + -19.746173858642578, + -23.38421630859375 ] }, { @@ -181,8 +181,8 @@ "lp": 0.0, "top": [ 0.0, - -33.39646530151367, - -35.11566162109375 + -33.39418411254883, + -35.11564254760742 ] }, { @@ -190,8 +190,8 @@ "lp": 0.0, "top": [ 0.0, - -32.84838104248047, - -35.273109436035156 + -32.845916748046875, + -35.26959991455078 ] }, { @@ -199,8 +199,8 @@ "lp": 0.0, "top": [ 0.0, - -25.60137176513672, - -27.059947967529297 + -25.589168548583984, + -27.054872512817383 ] }, { @@ -208,15 +208,15 @@ "lp": -2.622607780722319e-06, "top": [ -2.622607780722319e-06, - -13.489409446716309, - -13.853133201599121 + -13.486540794372559, + -13.8431978225708 ] } ], "expect_ok": true, "secs": [ - 0.54, - 0.21 + 0.26, + 0.2 ], "fails": [], "warns": [], @@ -234,8 +234,8 @@ "lp": 0.0, "top": [ 0.0, - -17.75491714477539, - -26.589563369750977 + -17.345935821533203, + -26.577651977539062 ] }, { @@ -243,8 +243,8 @@ "lp": 0.0, "top": [ 0.0, - -30.122194290161133, - -30.38653564453125 + -30.081588745117188, + -30.34840965270996 ] }, { @@ -252,8 +252,8 @@ "lp": 0.0, "top": [ 0.0, - -29.414281845092773, - -30.158910751342773 + -29.33576774597168, + -30.120506286621094 ] }, { @@ -261,24 +261,24 @@ "lp": 0.0, "top": [ 0.0, - -29.416168212890625, - -29.582674026489258 + -29.442604064941406, + -29.60540008544922 ] }, { "t": "", - "lp": -2.968355329358019e-05, + "lp": -3.111410842393525e-05, "top": [ - -2.968355329358019e-05, - -10.70214557647705, - -12.20948314666748 + -3.111410842393525e-05, + -10.642685890197754, + -12.16788387298584 ] } ], "expect_ok": true, "secs": [ - 3.2, - 2.37 + 2.32, + 1.92 ], "fails": [], "warns": [], @@ -293,20 +293,20 @@ "tokens": [ { "t": "The", - "lp": -0.04296554625034332, + "lp": -0.026700833812355995, "top": [ - -0.04296554625034332, - -3.1704070568084717, - -10.496527671813965 + -0.026700833812355995, + -3.6378800868988037, + -10.892267227172852 ] }, { "t": " box", - "lp": -1.3113030945532955e-06, + "lp": -1.7881409348774469e-06, "top": [ - -1.3113030945532955e-06, - -13.55001163482666, - -21.039003372192383 + -1.7881409348774469e-06, + -13.260320663452148, + -20.96363639831543 ] }, { @@ -314,8 +314,8 @@ "lp": 0.0, "top": [ 0.0, - -29.117216110229492, - -29.942995071411133 + -28.9838809967041, + -29.78819465637207 ] }, { @@ -323,8 +323,8 @@ "lp": 0.0, "top": [ 0.0, - -27.661611557006836, - -32.35749053955078 + -27.774768829345703, + -32.477821350097656 ] }, { @@ -332,8 +332,8 @@ "lp": 0.0, "top": [ 0.0, - -21.461620330810547, - -22.954038619995117 + -21.365768432617188, + -22.870962142944336 ] }, { @@ -341,26 +341,26 @@ "lp": 0.0, "top": [ 0.0, - -27.19775390625, - -29.295454025268555 + -27.03665542602539, + -29.08212661743164 ] }, { "t": " painted", - "lp": -0.0011156462132930756, + "lp": -0.0011389779392629862, "top": [ - -0.0011156462132930756, - -7.144807815551758, - -8.028390884399414 + -0.0011389779392629862, + -7.019361972808838, + -8.318601608276367 ] }, { "t": " green", - "lp": -0.034898627549409866, + "lp": -0.04391530901193619, "top": [ - -0.034898627549409866, - -3.372915029525757, - -11.85609245300293 + -0.04391530901193619, + -3.1475491523742676, + -11.762188911437988 ] }, { @@ -368,24 +368,24 @@ "lp": 0.0, "top": [ 0.0, - -22.66937828063965, - -26.065753936767578 + -22.36086654663086, + -26.04891586303711 ] }, { "t": "", - "lp": -0.0010726838372647762, + "lp": -0.0012635223101824522, "top": [ - -0.0010726838372647762, - -7.091031074523926, - -8.912800788879395 + -0.0012635223101824522, + -6.920742988586426, + -8.672751426696777 ] } ], "expect_ok": true, "secs": [ - 2.75, - 2.4 + 1.99, + 3.35 ], "fails": [], "warns": [], @@ -400,11 +400,11 @@ "tokens": [ { "t": "7", - "lp": -2.372293056396302e-05, + "lp": -2.157711423933506e-05, "top": [ - -2.372293056396302e-05, - -10.64995288848877, - -22.243431091308594 + -2.157711423933506e-05, + -10.743361473083496, + -22.29591941833496 ] }, { @@ -412,8 +412,8 @@ "lp": 0.0, "top": [ 0.0, - -28.83346176147461, - -29.565458297729492 + -28.85748863220215, + -29.587459564208984 ] }, { @@ -421,8 +421,8 @@ "lp": 0.0, "top": [ 0.0, - -26.088306427001953, - -27.944400787353516 + -26.107213973999023, + -27.962249755859375 ] }, { @@ -430,8 +430,8 @@ "lp": 0.0, "top": [ 0.0, - -27.22777557373047, - -29.653902053833008 + -27.25529670715332, + -29.65184211730957 ] }, { @@ -439,15 +439,15 @@ "lp": -6.437322554120328e-06, "top": [ -6.437322554120328e-06, - -12.364726066589355, - -13.6202974319458 + -12.368338584899902, + -13.618552207946777 ] } ], "expect_ok": true, "secs": [ - 1.56, - 0.28 + 0.28, + 0.15 ], "fails": [], "warns": [], @@ -465,8 +465,8 @@ "lp": 0.0, "top": [ 0.0, - -23.837657928466797, - -27.39961814880371 + -23.439563751220703, + -27.580297470092773 ] }, { @@ -474,8 +474,8 @@ "lp": 0.0, "top": [ 0.0, - -28.3249568939209, - -28.700883865356445 + -28.424129486083984, + -28.751176834106445 ] }, { @@ -483,8 +483,8 @@ "lp": 0.0, "top": [ 0.0, - -27.229145050048828, - -28.33145523071289 + -27.247303009033203, + -28.366674423217773 ] }, { @@ -492,24 +492,24 @@ "lp": 0.0, "top": [ 0.0, - -27.30320167541504, - -28.53386878967285 + -27.178863525390625, + -28.368209838867188 ] }, { "t": "", - "lp": -3.421365181566216e-05, + "lp": -3.612106593209319e-05, "top": [ - -3.421365181566216e-05, - -10.699947357177734, - -11.865020751953125 + -3.612106593209319e-05, + -10.628850936889648, + -11.873640060424805 ] } ], "expect_ok": true, "secs": [ - 0.99, - 0.52 + 0.44, + 0.36 ], "fails": [], "warns": [], @@ -527,8 +527,8 @@ "lp": 0.0, "top": [ 0.0, - -20.02585220336914, - -25.999948501586914 + -19.553802490234375, + -25.82794952392578 ] }, { @@ -536,8 +536,8 @@ "lp": 0.0, "top": [ 0.0, - -27.781736373901367, - -28.470014572143555 + -27.731733322143555, + -28.37761116027832 ] }, { @@ -545,8 +545,8 @@ "lp": 0.0, "top": [ 0.0, - -27.477123260498047, - -27.659160614013672 + -27.345928192138672, + -27.526277542114258 ] }, { @@ -554,24 +554,24 @@ "lp": 0.0, "top": [ 0.0, - -28.078296661376953, - -28.89556121826172 + -28.045589447021484, + -28.92365837097168 ] }, { "t": "", - "lp": -4.541977250482887e-05, + "lp": -4.5300555939320475e-05, "top": [ - -4.541977250482887e-05, - -10.212020874023438, - -12.318214416503906 + -4.5300555939320475e-05, + -10.21352767944336, + -12.273802757263184 ] } ], "expect_ok": true, "secs": [ - 1.36, - 0.95 + 0.84, + 0.76 ], "fails": [], "warns": [], @@ -589,8 +589,8 @@ "lp": 0.0, "top": [ 0.0, - -20.22911834716797, - -26.915477752685547 + -20.12065887451172, + -26.993148803710938 ] }, { @@ -598,8 +598,8 @@ "lp": 0.0, "top": [ 0.0, - -29.362497329711914, - -30.318370819091797 + -29.343807220458984, + -30.303817749023438 ] }, { @@ -607,8 +607,8 @@ "lp": 0.0, "top": [ 0.0, - -29.27713966369629, - -29.305185317993164 + -29.120895385742188, + -29.218040466308594 ] }, { @@ -616,24 +616,24 @@ "lp": 0.0, "top": [ 0.0, - -28.60381317138672, - -29.79412841796875 + -28.606548309326172, + -29.718564987182617 ] }, { "t": "", - "lp": -1.657022767176386e-05, + "lp": -1.5974172129062936e-05, "top": [ - -1.657022767176386e-05, - -11.429160118103027, - -12.603434562683105 + -1.5974172129062936e-05, + -11.485381126403809, + -12.56360149383545 ] } ], "expect_ok": true, "secs": [ - 1.99, - 1.57 + 2.33, + 1.17 ], "fails": [], "warns": [], @@ -643,5 +643,5 @@ } ], "pass": true, - "time": "2026-10-20T14:02:15-0300" + "time": "2026-10-20T19:19:13-0300" } \ No newline at end of file diff --git a/tests/golden/hw-gate/kat-qwen3-coder-30b-a3b.json b/tests/golden/hw-gate/kat-qwen3-coder-30b-a3b.json index 91883e08..3c507bac 100644 --- a/tests/golden/hw-gate/kat-qwen3-coder-30b-a3b.json +++ b/tests/golden/hw-gate/kat-qwen3-coder-30b-a3b.json @@ -12,23 +12,23 @@ "lp": -2.2649790025752736e-06, "top": [ -2.2649790025752736e-06, - -13.031167984008789, - -15.799896240234375 + -13.041383743286133, + -15.812938690185547 ] }, { "t": "", - "lp": -7.152559646783629e-07, + "lp": -8.344653679159819e-07, "top": [ - -7.152559646783629e-07, - -14.449156761169434, - -15.684027671813965 + -8.344653679159819e-07, + -14.357775688171387, + -15.659276008605957 ] } ], "expect_ok": true, "secs": [ - 0.4, + 0.46, 0.12 ], "fails": [], @@ -47,8 +47,8 @@ "lp": 0.0, "top": [ 0.0, - -18.470134735107422, - -20.794422149658203 + -18.56599998474121, + -20.930435180664062 ] }, { @@ -56,8 +56,8 @@ "lp": 0.0, "top": [ 0.0, - -21.349088668823242, - -22.538055419921875 + -21.268781661987305, + -22.509719848632812 ] }, { @@ -65,23 +65,23 @@ "lp": 0.0, "top": [ 0.0, - -24.884639739990234, - -26.48113441467285 + -24.888938903808594, + -26.45635986328125 ] }, { "t": "", - "lp": -0.00039281215867958963, + "lp": -0.0003903077740687877, "top": [ - -0.00039281215867958963, - -7.843700408935547, - -14.529281616210938 + -0.0003903077740687877, + -7.850327968597412, + -14.518157005310059 ] } ], "expect_ok": true, "secs": [ - 0.37, + 0.48, 0.14 ], "fails": [], @@ -97,11 +97,11 @@ "tokens": [ { "t": "```", - "lp": -1.3351529560168274e-05, + "lp": -1.2874686035502236e-05, "top": [ - -1.3351529560168274e-05, - -11.225248336791992, - -17.32335090637207 + -1.2874686035502236e-05, + -11.258050918579102, + -17.342391967773438 ] }, { @@ -109,8 +109,8 @@ "lp": 0.0, "top": [ 0.0, - -21.751779556274414, - -24.201276779174805 + -21.762264251708984, + -24.24568748474121 ] }, { @@ -118,8 +118,8 @@ "lp": 0.0, "top": [ 0.0, - -24.72541046142578, - -25.971561431884766 + -24.716732025146484, + -25.943157196044922 ] }, { @@ -127,8 +127,8 @@ "lp": 0.0, "top": [ 0.0, - -22.144779205322266, - -32.41608428955078 + -22.13996124267578, + -32.31664276123047 ] }, { @@ -136,8 +136,8 @@ "lp": 0.0, "top": [ 0.0, - -22.968482971191406, - -25.424617767333984 + -22.94637107849121, + -25.406265258789062 ] }, { @@ -145,8 +145,8 @@ "lp": 0.0, "top": [ 0.0, - -24.484603881835938, - -24.848657608032227 + -24.53059959411621, + -24.822757720947266 ] }, { @@ -154,8 +154,8 @@ "lp": 0.0, "top": [ 0.0, - -25.36216926574707, - -25.92169189453125 + -25.3516845703125, + -25.91318130493164 ] }, { @@ -163,8 +163,8 @@ "lp": 0.0, "top": [ 0.0, - -28.022029876708984, - -28.077701568603516 + -28.034698486328125, + -28.103511810302734 ] }, { @@ -172,8 +172,8 @@ "lp": 0.0, "top": [ 0.0, - -18.11073875427246, - -18.674861907958984 + -18.099117279052734, + -18.653209686279297 ] }, { @@ -181,8 +181,8 @@ "lp": 0.0, "top": [ 0.0, - -25.811063766479492, - -26.699237823486328 + -25.718994140625, + -26.711442947387695 ] }, { @@ -190,8 +190,8 @@ "lp": 0.0, "top": [ 0.0, - -23.809114456176758, - -24.004743576049805 + -23.81109619140625, + -24.022842407226562 ] }, { @@ -199,8 +199,8 @@ "lp": 0.0, "top": [ 0.0, - -22.988765716552734, - -24.96051597595215 + -22.989730834960938, + -24.953996658325195 ] }, { @@ -208,8 +208,8 @@ "lp": 0.0, "top": [ 0.0, - -27.244075775146484, - -28.443073272705078 + -27.252071380615234, + -28.421966552734375 ] }, { @@ -217,8 +217,8 @@ "lp": 0.0, "top": [ 0.0, - -27.033008575439453, - -27.329627990722656 + -27.034942626953125, + -27.341880798339844 ] }, { @@ -226,8 +226,8 @@ "lp": 0.0, "top": [ 0.0, - -25.889453887939453, - -27.69731903076172 + -25.902507781982422, + -27.68537139892578 ] }, { @@ -235,8 +235,8 @@ "lp": -3.933914285880746e-06, "top": [ -3.933914285880746e-06, - -12.455516815185547, - -20.829395294189453 + -12.440118789672852, + -20.80258560180664 ] }, { @@ -244,14 +244,14 @@ "lp": 0.0, "top": [ 0.0, - -21.102930068969727, - -27.309974670410156 + -21.09564971923828, + -27.302892684936523 ] } ], "expect_ok": true, "secs": [ - 0.64, + 0.63, 0.3 ], "fails": [], @@ -270,8 +270,8 @@ "lp": 0.0, "top": [ 0.0, - -17.58290672302246, - -21.20391845703125 + -17.838695526123047, + -21.124652862548828 ] }, { @@ -279,8 +279,8 @@ "lp": 0.0, "top": [ 0.0, - -28.738040924072266, - -30.73728370666504 + -28.46670913696289, + -30.072309494018555 ] }, { @@ -288,8 +288,8 @@ "lp": 0.0, "top": [ 0.0, - -26.960731506347656, - -27.071170806884766 + -26.389049530029297, + -26.43732261657715 ] }, { @@ -297,24 +297,24 @@ "lp": 0.0, "top": [ 0.0, - -24.520397186279297, - -27.55902862548828 + -23.788440704345703, + -26.702089309692383 ] }, { "t": "", - "lp": -1.1920928955078125e-07, + "lp": 0.0, "top": [ - -1.1920928955078125e-07, - -16.190399169921875, - -16.707834243774414 + 0.0, + -17.282726287841797, + -18.07607650756836 ] } ], "expect_ok": true, "secs": [ - 3.91, - 3.04 + 3.7, + 3.02 ], "fails": [], "warns": [], @@ -332,24 +332,24 @@ "lp": -1.1920928955078125e-07, "top": [ -1.1920928955078125e-07, - -16.117271423339844, - -16.75411605834961 + -16.236488342285156, + -16.81553077697754 ] }, { "t": "", - "lp": -3.671713420771994e-05, + "lp": -2.8849064619862475e-05, "top": [ - -3.671713420771994e-05, - -10.224531173706055, - -15.02718448638916 + -2.8849064619862475e-05, + -10.46827507019043, + -15.108896255493164 ] } ], "expect_ok": true, "secs": [ - 3.23, - 2.87 + 4.41, + 2.88 ], "fails": [], "warns": [], @@ -364,11 +364,11 @@ "tokens": [ { "t": "7", - "lp": -0.0014412651071324944, + "lp": -0.0006138777825981379, "top": [ - -0.0014412651071324944, - -6.546477794647217, - -12.185904502868652 + -0.0006138777825981379, + -7.402865886688232, + -12.377574920654297 ] }, { @@ -376,8 +376,8 @@ "lp": 0.0, "top": [ 0.0, - -26.17568588256836, - -29.69087028503418 + -25.859508514404297, + -29.966625213623047 ] }, { @@ -385,8 +385,8 @@ "lp": 0.0, "top": [ 0.0, - -26.0546875, - -26.59750747680664 + -26.20861053466797, + -26.33755874633789 ] }, { @@ -394,24 +394,24 @@ "lp": 0.0, "top": [ 0.0, - -23.303768157958984, - -26.352880477905273 + -24.849699020385742, + -24.871440887451172 ] }, { "t": "", - "lp": -0.0001866395614342764, + "lp": -0.0001613031781744212, "top": [ - -0.0001866395614342764, - -8.593660354614258, - -13.843860626220703 + -0.0001613031781744212, + -8.738890647888184, + -14.377886772155762 ] } ], "expect_ok": true, "secs": [ - 0.62, - 0.26 + 0.48, + 0.24 ], "fails": [], "warns": [], @@ -429,8 +429,8 @@ "lp": -1.1920928955078125e-07, "top": [ -1.1920928955078125e-07, - -15.793888092041016, - -21.977434158325195 + -15.944900512695312, + -21.95878028869629 ] }, { @@ -438,8 +438,8 @@ "lp": 0.0, "top": [ 0.0, - -27.149118423461914, - -28.233537673950195 + -27.195674896240234, + -28.329837799072266 ] }, { @@ -447,8 +447,8 @@ "lp": 0.0, "top": [ 0.0, - -25.052993774414062, - -25.46981430053711 + -24.516782760620117, + -25.13361358642578 ] }, { @@ -456,8 +456,8 @@ "lp": 0.0, "top": [ 0.0, - -23.117155075073242, - -25.40237045288086 + -22.833393096923828, + -24.74286460876465 ] }, { @@ -465,15 +465,15 @@ "lp": -1.1920928955078125e-07, "top": [ -1.1920928955078125e-07, - -15.978158950805664, - -17.24078369140625 + -16.487308502197266, + -17.23105239868164 ] } ], "expect_ok": true, "secs": [ - 1.0, - 0.61 + 0.73, + 0.57 ], "fails": [], "warns": [], @@ -491,8 +491,8 @@ "lp": -1.1920928955078125e-07, "top": [ -1.1920928955078125e-07, - -16.299541473388672, - -23.16580581665039 + -16.142803192138672, + -24.47840118408203 ] }, { @@ -500,8 +500,8 @@ "lp": 0.0, "top": [ 0.0, - -26.395736694335938, - -27.160341262817383 + -25.587709426879883, + -27.182025909423828 ] }, { @@ -509,8 +509,8 @@ "lp": 0.0, "top": [ 0.0, - -24.13756561279297, - -24.976016998291016 + -24.3331298828125, + -25.160778045654297 ] }, { @@ -518,23 +518,23 @@ "lp": 0.0, "top": [ 0.0, - -22.8209228515625, - -24.976768493652344 + -23.1445369720459, + -25.083118438720703 ] }, { "t": "", - "lp": -3.576279254957626e-07, + "lp": -8.344653679159819e-07, "top": [ - -3.576279254957626e-07, - -14.710966110229492, - -17.085346221923828 + -8.344653679159819e-07, + -13.996563911437988, + -16.84951400756836 ] } ], "expect_ok": true, "secs": [ - 1.66, + 1.43, 1.27 ], "fails": [], @@ -550,11 +550,11 @@ "tokens": [ { "t": "7", - "lp": -1.1920928955078125e-07, + "lp": 0.0, "top": [ - -1.1920928955078125e-07, - -16.400169372558594, - -23.157840728759766 + 0.0, + -16.84979248046875, + -23.161483764648438 ] }, { @@ -562,8 +562,8 @@ "lp": 0.0, "top": [ 0.0, - -25.453289031982422, - -26.563207626342773 + -25.374778747558594, + -26.419090270996094 ] }, { @@ -571,8 +571,8 @@ "lp": 0.0, "top": [ 0.0, - -22.723615646362305, - -23.605812072753906 + -22.801864624023438, + -23.957759857177734 ] }, { @@ -580,8 +580,8 @@ "lp": 0.0, "top": [ 0.0, - -22.719995498657227, - -24.957765579223633 + -25.41216278076172, + -26.586044311523438 ] }, { @@ -589,15 +589,15 @@ "lp": 0.0, "top": [ 0.0, - -16.731304168701172, - -17.859933853149414 + -16.79073143005371, + -17.14387321472168 ] } ], "expect_ok": true, "secs": [ - 2.39, - 1.93 + 3.11, + 1.94 ], "fails": [], "warns": [], @@ -607,5 +607,5 @@ } ], "pass": true, - "time": "2026-10-20T14:01:34-0300" + "time": "2026-10-20T19:18:27-0300" } \ No newline at end of file diff --git a/tests/golden/hw-gate/kat-qwen3.8-27b.json b/tests/golden/hw-gate/kat-qwen3.8-27b.json index 1bce8596..7315e47f 100644 --- a/tests/golden/hw-gate/kat-qwen3.8-27b.json +++ b/tests/golden/hw-gate/kat-qwen3.8-27b.json @@ -18,18 +18,18 @@ }, { "t": "", - "lp": -2.2769234419683926e-05, + "lp": -2.2650021492154337e-05, "top": [ - -2.2769234419683926e-05, - -10.95837116241455, - -12.934026718139648 + -2.2650021492154337e-05, + -10.963041305541992, + -12.937162399291992 ] } ], "expect_ok": true, "secs": [ - 5.21, - 0.72 + 4.41, + 0.52 ], "fails": [], "warns": [], @@ -53,11 +53,11 @@ }, { "t": "9", - "lp": -6.6757424974639434e-06, + "lp": -6.437322554120328e-06, "top": [ - -6.6757424974639434e-06, - -12.571114540100098, - -12.819674491882324 + -6.437322554120328e-06, + -12.60433292388916, + -12.83043384552002 ] }, { @@ -65,8 +65,8 @@ "lp": -5.602852525044e-06, "top": [ -5.602852525044e-06, - -12.782838821411133, - -14.111473083496094 + -12.780040740966797, + -14.123470306396484 ] }, { @@ -74,15 +74,15 @@ "lp": -2.3841860752327193e-07, "top": [ -2.3841860752327193e-07, - -16.455142974853516, - -16.475006103515625 + -16.484329223632812, + -16.491920471191406 ] } ], "expect_ok": true, "secs": [ - 2.35, - 0.88 + 3.63, + 0.67 ], "fails": [], "warns": [], @@ -106,56 +106,56 @@ }, { "t": "python", - "lp": -5.662601688527502e-05, + "lp": -5.8533474657451734e-05, "top": [ - -5.662601688527502e-05, - -10.846617698669434, - -11.535902976989746 + -5.8533474657451734e-05, + -10.7688627243042, + -11.514775276184082 ] }, { "t": "\n", - "lp": -0.00024399156973231584, + "lp": -0.0002642623148858547, "top": [ - -0.00024399156973231584, - -8.826600074768066, - -9.726075172424316 + -0.0002642623148858547, + -8.72321605682373, + -9.66930103302002 ] }, { "t": "def", - "lp": -8.070794865489006e-05, + "lp": -8.309232362080365e-05, "top": [ - -8.070794865489006e-05, - -10.07807445526123, - -11.132798194885254 + -8.309232362080365e-05, + -10.049986839294434, + -11.103116035461426 ] }, { "t": " add", - "lp": -6.85476916260086e-05, + "lp": -7.081282819854096e-05, "top": [ - -6.85476916260086e-05, - -9.730632781982422, - -13.14967155456543 + -7.081282819854096e-05, + -9.695310592651367, + -13.145099639892578 ] }, { "t": "(a", - "lp": -1.5020483260741457e-05, + "lp": -1.5258905477821827e-05, "top": [ - -1.5020483260741457e-05, - -12.29804801940918, - -12.450286865234375 + -1.5258905477821827e-05, + -12.268449783325195, + -12.443462371826172 ] }, { "t": ",", - "lp": -0.00013721930736210197, + "lp": -0.0001400807232130319, "top": [ - -0.00013721930736210197, - -8.924642562866211, - -12.753864288330078 + -0.0001400807232130319, + -8.904770851135254, + -12.73918342590332 ] }, { @@ -163,44 +163,44 @@ "lp": -3.337865791763761e-06, "top": [ -3.337865791763761e-06, - -13.021389961242676, - -14.598483085632324 + -12.980446815490723, + -14.583413124084473 ] }, { "t": "):", - "lp": -8.428451837971807e-05, + "lp": -8.78611026564613e-05, "top": [ - -8.428451837971807e-05, - -9.680156707763672, - -11.473926544189453 + -8.78611026564613e-05, + -9.631641387939453, + -11.476278305053711 ] }, { "t": "\n", - "lp": -0.0001318541617365554, + "lp": -0.00013090037100482732, "top": [ - -0.0001318541617365554, - -9.341947555541992, - -11.186956405639648 + -0.00013090037100482732, + -9.360136985778809, + -11.162250518798828 ] }, { "t": " ", - "lp": -2.3842141672503203e-05, + "lp": -2.467662670824211e-05, "top": [ - -2.3842141672503203e-05, - -11.81259536743164, - -12.029378890991211 + -2.467662670824211e-05, + -11.764810562133789, + -11.994022369384766 ] }, { "t": " return", - "lp": -0.00013435789151117206, + "lp": -0.00013590783055406064, "top": [ - -0.00013435789151117206, - -9.27015209197998, - -10.410176277160645 + -0.00013590783055406064, + -9.265326499938965, + -10.388593673706055 ] }, { @@ -208,60 +208,60 @@ "lp": -9.179157132166438e-06, "top": [ -9.179157132166438e-06, - -12.606521606445312, - -12.969639778137207 + -12.593327522277832, + -12.943744659423828 ] }, { "t": " +", - "lp": -1.1324947081448045e-05, + "lp": -1.1682578588079195e-05, "top": [ - -1.1324947081448045e-05, - -11.995025634765625, - -13.14829158782959 + -1.1682578588079195e-05, + -11.95571231842041, + -13.094393730163574 ] }, { "t": " b", - "lp": -6.55653229841846e-06, + "lp": -6.7949526965094265e-06, "top": [ - -6.55653229841846e-06, - -12.493941307067871, - -13.747005462646484 + -6.7949526965094265e-06, + -12.455221176147461, + -13.741857528686523 ] }, { "t": "\n", - "lp": -3.2067811844171956e-05, + "lp": -3.302151890238747e-05, "top": [ - -3.2067811844171956e-05, - -10.797261238098145, - -12.199297904968262 + -3.302151890238747e-05, + -10.762009620666504, + -12.193074226379395 ] }, { "t": "```", - "lp": -0.00016118395433295518, + "lp": -0.00017012612079270184, "top": [ - -0.00016118395433295518, - -8.818310737609863, - -12.213029861450195 + -0.00017012612079270184, + -8.762348175048828, + -12.143228530883789 ] }, { "t": "", - "lp": -5.746052920585498e-05, + "lp": -5.912955748499371e-05, "top": [ - -5.746052920585498e-05, - -9.887313842773438, - -11.963237762451172 + -5.912955748499371e-05, + -9.860563278198242, + -11.941642761230469 ] } ], "expect_ok": true, "secs": [ - 3.88, - 2.06 + 3.83, + 1.84 ], "fails": [], "warns": [], @@ -276,54 +276,54 @@ "tokens": [ { "t": "7", - "lp": -9.024550672620535e-05, + "lp": -8.941096166381612e-05, "top": [ - -9.024550672620535e-05, - -10.237072944641113, - -11.205108642578125 + -8.941096166381612e-05, + -10.350811004638672, + -11.237091064453125 ] }, { "t": "3", - "lp": -4.49429135187529e-05, + "lp": -3.898219802067615e-05, "top": [ - -4.49429135187529e-05, - -10.0617036819458, - -14.127685546875 + -3.898219802067615e-05, + -10.215682029724121, + -13.881829261779785 ] }, { "t": "4", - "lp": -3.3736796467565e-05, + "lp": -3.325994475744665e-05, "top": [ - -3.3736796467565e-05, - -10.33826732635498, - -15.162035942077637 + -3.325994475744665e-05, + -10.351651191711426, + -15.08224868774414 ] }, { "t": "1", - "lp": -1.9908149624825455e-05, + "lp": -1.8477610865375027e-05, "top": [ - -1.9908149624825455e-05, - -10.892329216003418, - -14.324150085449219 + -1.8477610865375027e-05, + -10.970452308654785, + -14.340970039367676 ] }, { "t": "", - "lp": -1.1920928955078125e-07, + "lp": -2.3841860752327193e-07, "top": [ - -1.1920928955078125e-07, - -16.489173889160156, - -16.742359161376953 + -2.3841860752327193e-07, + -16.267601013183594, + -16.527721405029297 ] } ], "expect_ok": true, "secs": [ - 23.47, - 15.41 + 13.09, + 8.84 ], "fails": [], "warns": [], @@ -338,11 +338,11 @@ "tokens": [ { "t": "green", - "lp": -0.1295344978570938, + "lp": -0.13628681004047394, "top": [ - -0.1295344978570938, - -2.6329562664031982, - -3.0132033824920654 + -0.13628681004047394, + -2.5633153915405273, + -2.9993772506713867 ] }, { @@ -350,15 +350,15 @@ "lp": -1.0728841743912199e-06, "top": [ -1.0728841743912199e-06, - -14.222901344299316, - -15.435184478759766 + -14.121581077575684, + -15.347905158996582 ] } ], "expect_ok": true, "secs": [ - 17.7, - 17.56 + 8.64, + 8.31 ], "fails": [], "warns": [], @@ -373,38 +373,38 @@ "tokens": [ { "t": "7", - "lp": -0.00016523772501386702, + "lp": -0.00016786076594144106, "top": [ - -0.00016523772501386702, - -10.39382266998291, - -10.442679405212402 + -0.00016786076594144106, + -10.339143753051758, + -10.386219024658203 ] }, { "t": "3", - "lp": -7.283953164005652e-05, + "lp": -7.689294579904526e-05, "top": [ - -7.283953164005652e-05, - -9.605546951293945, - -12.738836288452148 + -7.689294579904526e-05, + -9.547911643981934, + -12.70700740814209 ] }, { "t": "4", - "lp": -6.985908112255856e-05, + "lp": -7.140891102608293e-05, "top": [ - -6.985908112255856e-05, - -9.60492992401123, - -14.204554557800293 + -7.140891102608293e-05, + -9.583283424377441, + -14.178752899169922 ] }, { "t": "1", - "lp": -8.356920443475246e-05, + "lp": -8.535749657312408e-05, "top": [ - -8.356920443475246e-05, - -9.429685592651367, - -13.571880340576172 + -8.535749657312408e-05, + -9.406699180603027, + -13.555604934692383 ] }, { @@ -412,15 +412,15 @@ "lp": -9.536747711536009e-07, "top": [ -9.536747711536009e-07, - -15.10450553894043, - -15.10677433013916 + -15.098702430725098, + -15.130549430847168 ] } ], "expect_ok": true, "secs": [ - 6.37, - 3.65 + 1.41, + 0.96 ], "fails": [], "warns": [], @@ -435,54 +435,54 @@ "tokens": [ { "t": "7", - "lp": -0.00028918395400978625, + "lp": -0.00028906468651257455, "top": [ - -0.00028918395400978625, - -8.93349552154541, - -10.222776412963867 + -0.00028906468651257455, + -9.06490421295166, + -10.145061492919922 ] }, { "t": "3", - "lp": -3.49289330188185e-05, + "lp": -3.7313202483346686e-05, "top": [ - -3.49289330188185e-05, - -10.451006889343262, - -12.780409812927246 + -3.7313202483346686e-05, + -10.378912925720215, + -12.696208953857422 ] }, { "t": "4", - "lp": -2.217317342001479e-05, + "lp": -2.3246082491823472e-05, "top": [ - -2.217317342001479e-05, - -10.854178428649902, - -14.056246757507324 + -2.3246082491823472e-05, + -10.804769515991211, + -13.962850570678711 ] }, { "t": "1", - "lp": -2.0861843950115144e-05, + "lp": -2.1696325347875245e-05, "top": [ - -2.0861843950115144e-05, - -10.870553970336914, - -14.345874786376953 + -2.1696325347875245e-05, + -10.840489387512207, + -14.33970832824707 ] }, { "t": "", - "lp": -1.1920935776288388e-06, + "lp": -1.4305124977909145e-06, "top": [ - -1.1920935776288388e-06, - -14.753901481628418, - -14.891024589538574 + -1.4305124977909145e-06, + -14.68392276763916, + -14.809412956237793 ] } ], "expect_ok": true, "secs": [ - 8.48, - 5.85 + 3.85, + 2.2 ], "fails": [], "warns": [], @@ -497,38 +497,38 @@ "tokens": [ { "t": "7", - "lp": -0.00010920166823780164, + "lp": -0.0001242237922269851, "top": [ - -0.00010920166823780164, - -9.890533447265625, - -11.385848045349121 + -0.0001242237922269851, + -9.70173454284668, + -11.374184608459473 ] }, { "t": "3", - "lp": -3.027961793122813e-05, + "lp": -3.0637256713816896e-05, "top": [ - -3.027961793122813e-05, - -10.533531188964844, - -13.436206817626953 + -3.0637256713816896e-05, + -10.527534484863281, + -13.414093017578125 ] }, { "t": "4", - "lp": -2.5868750526569784e-05, + "lp": -2.741851312748622e-05, "top": [ - -2.5868750526569784e-05, - -10.644862174987793, - -14.45397663116455 + -2.741851312748622e-05, + -10.588257789611816, + -14.464113235473633 ] }, { "t": "1", - "lp": -1.5735749911982566e-05, + "lp": -1.633180545468349e-05, "top": [ - -1.5735749911982566e-05, - -11.13611125946045, - -14.660421371459961 + -1.633180545468349e-05, + -11.10401439666748, + -14.594910621643066 ] }, { @@ -536,15 +536,15 @@ "lp": -3.576279254957626e-07, "top": [ -3.576279254957626e-07, - -16.201255798339844, - -16.514781951904297 + -16.06974983215332, + -16.346542358398438 ] } ], "expect_ok": true, "secs": [ - 11.82, - 8.78 + 6.26, + 5.69 ], "fails": [], "warns": [], @@ -559,54 +559,54 @@ "tokens": [ { "t": "7", - "lp": -0.00011897795047843829, + "lp": -0.00012481991143431515, "top": [ - -0.00011897795047843829, - -10.51734447479248, - -11.034518241882324 + -0.00012481991143431515, + -10.45021915435791, + -10.975947380065918 ] }, { "t": "3", - "lp": -3.83861297450494e-05, + "lp": -4.041276406496763e-05, "top": [ - -3.83861297450494e-05, - -10.272382736206055, - -13.424480438232422 + -4.041276406496763e-05, + -10.222339630126953, + -13.365954399108887 ] }, { "t": "4", - "lp": -3.480972009128891e-05, + "lp": -3.671713420771994e-05, "top": [ - -3.480972009128891e-05, - -10.320425033569336, - -14.737032890319824 + -3.671713420771994e-05, + -10.271339416503906, + -14.665650367736816 ] }, { "t": "1", - "lp": -2.2411597456084564e-05, + "lp": -2.2530810383614153e-05, "top": [ - -2.2411597456084564e-05, - -10.771093368530273, - -14.289402961730957 + -2.2530810383614153e-05, + -10.765888214111328, + -14.250859260559082 ] }, { "t": "", - "lp": -4.768372718899627e-07, + "lp": -3.576279254957626e-07, "top": [ - -4.768372718899627e-07, - -15.530964851379395, - -15.553465843200684 + -3.576279254957626e-07, + -15.549956321716309, + -15.572606086730957 ] } ], "expect_ok": true, "secs": [ - 17.07, - 14.6 + 9.14, + 8.78 ], "fails": [], "warns": [], @@ -616,5 +616,5 @@ } ], "pass": true, - "time": "2026-10-12T07:19:33-0300" + "time": "2026-10-20T19:16:08-0300" } \ No newline at end of file diff --git a/tests/golden/hw-gate/kat-zaya1-8b.json b/tests/golden/hw-gate/kat-zaya1-8b.json index f3a703d5..2f4c7710 100644 --- a/tests/golden/hw-gate/kat-zaya1-8b.json +++ b/tests/golden/hw-gate/kat-zaya1-8b.json @@ -9,153 +9,153 @@ "tokens": [ { "t": "Paris", - "lp": -0.035021644085645676, + "lp": -0.039008576422929764, "top": [ - -0.035021644085645676, - -3.502577543258667, - -5.695631980895996 + -0.039008576422929764, + -3.386958599090576, + -5.675050258636475 ] }, { "t": ".", - "lp": -0.0453915148973465, + "lp": -0.04034578800201416, "top": [ - -0.0453915148973465, - -4.121148586273193, - -4.389102458953857 + -0.04034578800201416, + -4.339808940887451, + -4.3988728523254395 ] }, { "t": " ", - "lp": -0.677693247795105, + "lp": -0.7577502131462097, "top": [ - -0.677693247795105, - -1.4301356077194214, - -1.9982603788375854 + -0.7577502131462097, + -1.4420573711395264, + -1.791690468788147 ] }, { "t": "\n\n", - "lp": -0.0009288332075811923, + "lp": -0.0009151114500127733, "top": [ - -0.0009288332075811923, - -7.18426513671875, - -9.39015007019043 + -0.0009151114500127733, + -7.196650981903076, + -9.493497848510742 ] }, { "t": "(", - "lp": -0.8424666523933411, + "lp": -0.8426641821861267, "top": [ - -0.8424666523933411, - -1.5946333408355713, - -2.140697717666626 + -0.8426641821861267, + -1.626640796661377, + -1.9116977453231812 ] }, { "t": "Note", - "lp": -0.004873748868703842, + "lp": -0.004710306413471699, "top": [ - -0.004873748868703842, - -6.867918491363525, - -7.1217122077941895 + -0.004710306413471699, + -6.744778633117676, + -7.240584373474121 ] }, { "t": ":", - "lp": -5.69836629438214e-05, + "lp": -5.746052920585498e-05, "top": [ - -5.69836629438214e-05, - -10.965885162353516, - -12.234661102294922 + -5.746052920585498e-05, + -10.815788269042969, + -12.356147766113281 ] }, { "t": " The", - "lp": -0.22827492654323578, + "lp": -0.2334318906068802, "top": [ - -0.22827492654323578, - -1.8919179439544678, - -3.873290538787842 + -0.2334318906068802, + -1.8515177965164185, + -3.88505482673645 ] }, { "t": " answer", - "lp": -0.06277571618556976, + "lp": -0.05066439509391785, "top": [ - -0.06277571618556976, - -3.463067054748535, - -3.6976709365844727 + -0.05066439509391785, + -3.6721129417419434, + -3.952157497406006 ] }, { "t": " is", - "lp": -0.028207307681441307, + "lp": -0.024640720337629318, "top": [ - -0.028207307681441307, - -4.3850531578063965, - -4.864814281463623 + -0.024640720337629318, + -4.470189571380615, + -5.2206339836120605 ] }, { "t": " one", - "lp": -0.8744703531265259, + "lp": -0.9173271059989929, "top": [ - -0.8744703531265259, - -1.4970537424087524, - -1.5362820625305176 + -0.9173271059989929, + -1.3319084644317627, + -1.6773922443389893 ] }, { "t": " word", - "lp": -1.4305124977909145e-06, + "lp": -1.6689314179529902e-06, "top": [ - -1.4305124977909145e-06, - -13.514165878295898, - -16.619604110717773 + -1.6689314179529902e-06, + -13.367935180664062, + -16.617149353027344 ] }, { "t": " as", - "lp": -0.03541295975446701, + "lp": -0.039704371243715286, "top": [ - -0.03541295975446701, - -3.4066860675811768, - -6.7892351150512695 + -0.039704371243715286, + -3.297440528869629, + -6.576886177062988 ] }, { "t": " requested", - "lp": -0.000147353537613526, + "lp": -0.00012470068759284914, "top": [ - -0.000147353537613526, - -9.165533065795898, - -10.689388275146484 + -0.00012470068759284914, + -9.413558006286621, + -10.749266624450684 ] }, { "t": ",", - "lp": -0.017770737409591675, + "lp": -0.01755518838763237, "top": [ - -0.017770737409591675, - -4.355218887329102, - -6.253973007202148 + -0.01755518838763237, + -4.362953186035156, + -6.219970703125 ] }, { "t": " but", - "lp": -0.889842689037323, + "lp": -0.9154409766197205, "top": [ - -0.889842689037323, - -1.0798089504241943, - -2.092332601547241 + -0.9154409766197205, + -1.13650643825531, + -1.9922689199447632 ] } ], "expect_ok": true, "secs": [ - 2.52, - 0.44 + 0.98, + 0.35 ], "fails": [], "warns": [], @@ -170,128 +170,128 @@ "tokens": [ { "t": "1", - "lp": -0.5278510451316833, + "lp": -0.5574058890342712, "top": [ - -0.5278510451316833, - -1.8445814847946167, - -1.9131735563278198 + -0.5574058890342712, + -1.6552261114120483, + -2.0315957069396973 ] }, { "t": "7", - "lp": -0.008424442261457443, + "lp": -0.007599967531859875, "top": [ - -0.008424442261457443, - -5.448155879974365, - -5.778018474578857 + -0.007599967531859875, + -5.713149070739746, + -5.737021446228027 ] }, { "t": " *", - "lp": -0.11264479905366898, + "lp": -0.11640411615371704, "top": [ - -0.11264479905366898, - -3.725973606109619, - -3.898378849029541 + -0.11640411615371704, + -3.695448637008667, + -3.914097309112549 ] }, { "t": " ", - "lp": -2.3841887468734058e-06, + "lp": -2.5033982637978625e-06, "top": [ - -2.3841887468734058e-06, - -13.184176445007324, - -15.577398300170898 + -2.5033982637978625e-06, + -13.130226135253906, + -15.617280960083008 ] }, { "t": "2", - "lp": -3.218656047465629e-06, + "lp": -3.695494797284482e-06, "top": [ - -3.218656047465629e-06, - -13.296366691589355, - -14.024779319763184 + -3.695494797284482e-06, + -13.158546447753906, + -13.926856994628906 ] }, { "t": "3", - "lp": -1.1682578588079195e-05, + "lp": -1.2874686035502236e-05, "top": [ - -1.1682578588079195e-05, - -12.695161819458008, - -13.287647247314453 + -1.2874686035502236e-05, + -12.65900707244873, + -13.160114288330078 ] }, { "t": " =", - "lp": -0.3213600814342499, + "lp": -0.323148250579834, "top": [ - -0.3213600814342499, - -1.476661205291748, - -4.344151020050049 + -0.323148250579834, + -1.4473013877868652, + -4.391469955444336 ] }, { "t": " ", - "lp": -0.013305939733982086, + "lp": -0.0133696673437953, "top": [ - -0.013305939733982086, - -4.471449375152588, - -7.080763339996338 + -0.0133696673437953, + -4.465102195739746, + -7.050889015197754 ] }, { "t": "3", - "lp": -2.98023678624304e-06, + "lp": -3.57628505298635e-06, "top": [ - -2.98023678624304e-06, - -13.355414390563965, - -13.556750297546387 + -3.57628505298635e-06, + -13.085275650024414, + -13.44157600402832 ] }, { "t": "9", - "lp": -2.7299300199956633e-05, + "lp": -2.5868750526569784e-05, "top": [ - -2.7299300199956633e-05, - -11.190396308898926, - -12.137059211730957 + -2.5868750526569784e-05, + -11.269160270690918, + -12.209689140319824 ] }, { "t": "1", - "lp": -1.6689314179529902e-06, + "lp": -1.7881409348774469e-06, "top": [ - -1.6689314179529902e-06, - -14.628728866577148, - -14.79973030090332 + -1.7881409348774469e-06, + -14.611639022827148, + -14.62666130065918 ] }, { "t": ".", - "lp": -0.3896702229976654, + "lp": -0.36501502990722656, "top": [ - -0.3896702229976654, - -1.1989277601242065, - -5.524967193603516 + -0.36501502990722656, + -1.2577754259109497, + -5.436352252960205 ] }, { "t": " Answer", - "lp": -0.009872190654277802, + "lp": -0.006525542587041855, "top": [ - -0.009872190654277802, - -4.83015251159668, - -6.593011379241943 + -0.006525542587041855, + -5.5228590965271, + -6.1675705909729 ] }, { "t": " with", - "lp": -0.00040628816350363195, + "lp": -0.0004290666547603905, "top": [ - -0.00040628816350363195, - -8.158889770507812, - -9.308809280395508 + -0.0004290666547603905, + -7.9290666580200195, + -9.927958488464355 ] }, { @@ -299,24 +299,24 @@ "lp": -1.1920928955078125e-07, "top": [ -1.1920928955078125e-07, - -15.768244743347168, - -16.661951065063477 + -15.749298095703125, + -16.989566802978516 ] }, { "t": " number", - "lp": -7.987054232216906e-06, + "lp": -7.629423635080457e-06, "top": [ - -7.987054232216906e-06, - -12.480867385864258, - -12.574686050415039 + -7.629423635080457e-06, + -12.476832389831543, + -12.648224830627441 ] } ], "expect_ok": true, "secs": [ - 1.05, - 0.38 + 0.72, + 0.31 ], "fails": [], "warns": [], @@ -331,38 +331,38 @@ "tokens": [ { "t": "```", - "lp": -0.013561598025262356, + "lp": -0.0093879709020257, "top": [ - -0.013561598025262356, - -4.338377475738525, - -8.149600982666016 + -0.0093879709020257, + -4.714056015014648, + -8.288925170898438 ] }, { "t": "python", - "lp": -0.004651919007301331, + "lp": -0.00470731221139431, "top": [ - -0.004651919007301331, - -5.374639511108398, - -12.048423767089844 + -0.00470731221139431, + -5.363189697265625, + -11.776330947875977 ] }, { "t": "\n", - "lp": -3.886298509314656e-05, + "lp": -2.95643403660506e-05, "top": [ - -3.886298509314656e-05, - -11.75958251953125, - -12.211751937866211 + -2.95643403660506e-05, + -12.254313468933105, + -12.376307487487793 ] }, { "t": "def", - "lp": -0.00011742804781533778, + "lp": -0.00011051311594201252, "top": [ - -0.00011742804781533778, - -9.622054100036621, - -9.888689994812012 + -0.00011051311594201252, + -9.64328384399414, + -9.996562957763672 ] }, { @@ -370,8 +370,8 @@ "lp": 0.0, "top": [ 0.0, - -18.427230834960938, - -18.775917053222656 + -18.319839477539062, + -18.62253189086914 ] }, { @@ -379,8 +379,8 @@ "lp": 0.0, "top": [ 0.0, - -17.277053833007812, - -17.847030639648438 + -16.917797088623047, + -17.929920196533203 ] }, { @@ -388,17 +388,17 @@ "lp": 0.0, "top": [ 0.0, - -21.507946014404297, - -21.957822799682617 + -21.49859619140625, + -22.33576774597168 ] }, { "t": ",", - "lp": -1.3589951777248643e-05, + "lp": -1.3113108252582606e-05, "top": [ - -1.3589951777248643e-05, - -11.868204116821289, - -12.950483322143555 + -1.3113108252582606e-05, + -12.120355606079102, + -12.632059097290039 ] }, { @@ -406,17 +406,17 @@ "lp": 0.0, "top": [ 0.0, - -17.58462142944336, - -21.635250091552734 + -17.39578628540039, + -21.996456146240234 ] }, { "t": "):", - "lp": -9.298368240706623e-06, + "lp": -9.655998837843072e-06, "top": [ - -9.298368240706623e-06, - -11.619142532348633, - -16.379865646362305 + -9.655998837843072e-06, + -11.568445205688477, + -16.163415908813477 ] }, { @@ -424,17 +424,17 @@ "lp": -1.1920928955078125e-07, "top": [ -1.1920928955078125e-07, - -16.41338348388672, - -17.852313995361328 + -16.22916603088379, + -17.48102569580078 ] }, { "t": " ", - "lp": -0.0003006910264957696, + "lp": -0.00042012217454612255, "top": [ - -0.0003006910264957696, - -8.252787590026855, - -10.202523231506348 + -0.00042012217454612255, + -7.907568454742432, + -9.95473575592041 ] }, { @@ -442,8 +442,8 @@ "lp": 0.0, "top": [ 0.0, - -18.029903411865234, - -21.223783493041992 + -18.53474998474121, + -21.63280487060547 ] }, { @@ -451,8 +451,8 @@ "lp": 0.0, "top": [ 0.0, - -19.58050537109375, - -19.689638137817383 + -19.295665740966797, + -19.576431274414062 ] }, { @@ -460,8 +460,8 @@ "lp": 0.0, "top": [ 0.0, - -17.32297134399414, - -18.625877380371094 + -16.913801193237305, + -18.739055633544922 ] }, { @@ -469,17 +469,17 @@ "lp": 0.0, "top": [ 0.0, - -22.022560119628906, - -22.30773162841797 + -21.702491760253906, + -22.029876708984375 ] }, { "t": "\n", - "lp": -0.00020011281594634056, + "lp": -0.0002423222322249785, "top": [ - -0.00020011281594634056, - -8.516618728637695, - -17.242643356323242 + -0.0002423222322249785, + -8.325479507446289, + -16.967191696166992 ] }, { @@ -487,24 +487,24 @@ "lp": 0.0, "top": [ 0.0, - -17.925811767578125, - -18.639698028564453 + -17.858491897583008, + -18.715087890625 ] }, { "t": "", - "lp": -0.01615213043987751, + "lp": -0.020124413073062897, "top": [ - -0.01615213043987751, - -4.54276704788208, - -5.251585483551025 + -0.020124413073062897, + -4.312557220458984, + -5.051373481750488 ] } ], "expect_ok": true, "secs": [ - 1.12, - 0.42 + 0.79, + 0.33 ], "fails": [], "warns": [], @@ -519,63 +519,63 @@ "tokens": [ { "t": "7", - "lp": -0.00026187748881056905, + "lp": -0.0003747449954971671, "top": [ - -0.00026187748881056905, - -8.396452903747559, - -10.78415584564209 + -0.0003747449954971671, + -8.056929588317871, + -10.578852653503418 ] }, { "t": "3", - "lp": -0.00021740177180618048, + "lp": -0.00023862585658207536, "top": [ - -0.00021740177180618048, - -8.586224555969238, - -11.256681442260742 + -0.00023862585658207536, + -8.494723320007324, + -11.13725757598877 ] }, { "t": "4", - "lp": -5.006802894058637e-06, + "lp": -6.55653229841846e-06, "top": [ - -5.006802894058637e-06, - -13.306605339050293, - -13.513176918029785 + -6.55653229841846e-06, + -13.044211387634277, + -13.26345157623291 ] }, { "t": "1", - "lp": -8.213857654482126e-05, + "lp": -9.167614916805178e-05, "top": [ - -8.213857654482126e-05, - -10.653112411499023, - -10.662282943725586 + -9.167614916805178e-05, + -10.413032531738281, + -10.643993377685547 ] }, { "t": ".", - "lp": -0.11130199581384659, + "lp": -0.06026395410299301, "top": [ - -0.11130199581384659, - -2.5882842540740967, - -4.198035717010498 + -0.06026395410299301, + -3.5333056449890137, + -4.255828380584717 ] }, { "t": "", - "lp": -0.36143729090690613, + "lp": -0.18907280266284943, "top": [ - -0.36143729090690613, - -1.2396453619003296, - -5.050693988800049 + -0.18907280266284943, + -1.8023368120193481, + -5.4245405197143555 ] } ], "expect_ok": true, "secs": [ - 4.77, - 3.74 + 2.13, + 1.42 ], "fails": [], "warns": [], @@ -586,135 +586,49 @@ { "name": "prefix-reuse", "prompt_tokens": 4743, - "text": "The color of the box in the cellar is **green**.", + "text": "green.", "tokens": [ - { - "t": "The", - "lp": -0.9311938881874084, - "top": [ - -0.9311938881874084, - -1.0682483911514282, - -1.4606376886367798 - ] - }, - { - "t": " color", - "lp": -0.001429625554010272, - "top": [ - -0.001429625554010272, - -6.5779523849487305, - -10.202422142028809 - ] - }, - { - "t": " of", - "lp": -4.529963462118758e-06, - "top": [ - -4.529963462118758e-06, - -12.337414741516113, - -16.32185173034668 - ] - }, - { - "t": " the", - "lp": 0.0, - "top": [ - 0.0, - -19.096017837524414, - -20.847021102905273 - ] - }, - { - "t": " box", - "lp": -1.4305124977909145e-06, - "top": [ - -1.4305124977909145e-06, - -13.481794357299805, - -18.981950759887695 - ] - }, - { - "t": " in", - "lp": 0.0, - "top": [ - 0.0, - -16.75824546813965, - -17.373693466186523 - ] - }, - { - "t": " the", - "lp": 0.0, - "top": [ - 0.0, - -18.2919921875, - -19.998931884765625 - ] - }, - { - "t": " cellar", - "lp": -9.549120295559987e-05, - "top": [ - -9.549120295559987e-05, - -9.351165771484375, - -11.73752212524414 - ] - }, - { - "t": " is", - "lp": -1.657022767176386e-05, - "top": [ - -1.657022767176386e-05, - -11.354931831359863, - -12.530766487121582 - ] - }, - { - "t": " **", - "lp": -0.14311665296554565, - "top": [ - -0.14311665296554565, - -2.0245120525360107, - -6.829367160797119 - ] - }, { "t": "green", - "lp": -1.0728841743912199e-06, + "lp": -0.8594515919685364, "top": [ - -1.0728841743912199e-06, - -13.917038917541504, - -15.92955493927002 + -0.8594515919685364, + -1.059999704360962, + -1.586305856704712 ] }, { - "t": "**.", - "lp": -3.695494797284482e-06, + "t": ".", + "lp": -0.38057199120521545, "top": [ - -3.695494797284482e-06, - -13.258186340332031, - -13.259162902832031 + -0.38057199120521545, + -1.1626030206680298, + -6.350142002105713 ] }, { "t": "", - "lp": -0.02780286781489849, + "lp": -0.6657108664512634, "top": [ - -0.02780286781489849, - -4.0023417472839355, - -5.388497829437256 + -0.6657108664512634, + -0.8204272389411926, + -4.445668697357178 ] } ], "expect_ok": true, "secs": [ - 12.61, - 2.7 + 1.16, + 1.09 ], "fails": [], - "warns": [], + "warns": [ + "cache path: cached request diverges from the fresh one at token 0 (fresh top-2 margin 0.201): 'The color of the box in the cellar is **green**.'" + ], "cache_path": { - "first_divergence": null + "first_divergence": 0, + "fresh_margin": 0.20054811239242554, + "cached_text": "The color of the box in the cellar is **green**." } }, { @@ -724,63 +638,63 @@ "tokens": [ { "t": "7", - "lp": -0.1774652898311615, + "lp": -0.12349457293748856, "top": [ - -0.1774652898311615, - -2.5626046657562256, - -3.1696720123291016 + -0.12349457293748856, + -2.829110622406006, + -3.5754904747009277 ] }, { "t": "3", - "lp": -0.00033002576674334705, + "lp": -0.00041523255640640855, "top": [ - -0.00033002576674334705, - -8.681154251098633, - -9.434825897216797 + -0.00041523255640640855, + -8.483769416809082, + -9.184988975524902 ] }, { "t": "4", - "lp": -2.229238634754438e-05, + "lp": -1.6927862816373818e-05, "top": [ - -2.229238634754438e-05, - -12.03696346282959, - -12.039010047912598 + -1.6927862816373818e-05, + -12.367925643920898, + -12.38786506652832 ] }, { "t": "1", - "lp": -0.0003693785984069109, + "lp": -0.00036747055128216743, "top": [ - -0.0003693785984069109, - -8.5090970993042, - -9.565192222595215 + -0.00036747055128216743, + -8.58629322052002, + -9.594529151916504 ] }, { "t": ".", - "lp": -0.0914091020822525, + "lp": -0.11377266049385071, "top": [ - -0.0914091020822525, - -2.8348286151885986, - -3.6159491539001465 + -0.11377266049385071, + -2.76969838142395, + -3.1501991748809814 ] }, { "t": "", - "lp": -0.1953394114971161, + "lp": -0.2737899720668793, "top": [ - -0.1953394114971161, - -2.1885836124420166, - -3.5391733646392822 + -0.2737899720668793, + -2.0636143684387207, + -2.8958191871643066 ] } ], "expect_ok": true, "secs": [ - 12.43, - 11.36 + 0.59, + 0.25 ], "fails": [], "warns": [], @@ -795,117 +709,117 @@ "tokens": [ { "t": "7", - "lp": -0.051450736820697784, + "lp": -0.06024103984236717, "top": [ - -0.051450736820697784, - -3.1299362182617188, - -5.57319450378418 + -0.06024103984236717, + -3.0424301624298096, + -5.2806220054626465 ] }, { "t": "3", - "lp": -0.0018221040954813361, + "lp": -0.00044111203169450164, "top": [ - -0.0018221040954813361, - -6.758409023284912, - -7.974733829498291 + -0.00044111203169450164, + -8.678476333618164, + -8.712133407592773 ] }, { "t": "4", - "lp": -0.00022276736854109913, + "lp": -0.00012374689686112106, "top": [ - -0.00022276736854109913, - -9.93307876586914, - -10.265695571899414 + -0.00012374689686112106, + -10.267698287963867, + -10.931215286254883 ] }, { "t": "1", - "lp": -0.0005891866167075932, + "lp": -0.0002966963220387697, "top": [ - -0.0005891866167075932, - -8.750654220581055, - -8.973615646362305 + -0.0002966963220387697, + -9.403716087341309, + -9.420147895812988 ] }, { "t": ".", - "lp": -0.0029892255552113056, + "lp": -0.0016922956565394998, "top": [ - -0.0029892255552113056, - -6.0879998207092285, - -8.782262802124023 + -0.0016922956565394998, + -6.713276386260986, + -9.219317436218262 ] }, { "t": " Answer", - "lp": -0.3721570372581482, + "lp": -0.4085305333137512, "top": [ - -0.3721570372581482, - -1.3385523557662964, - -3.8931386470794678 + -0.4085305333137512, + -1.2265390157699585, + -3.9479668140411377 ] }, { "t": " with", - "lp": -0.005495421122759581, + "lp": -0.006505024153739214, "top": [ - -0.005495421122759581, - -5.2145562171936035, - -10.820225715637207 + -0.006505024153739214, + -5.044869422912598, + -10.760242462158203 ] }, { "t": " the", - "lp": -5.006802894058637e-06, + "lp": -5.960482212685747e-06, "top": [ - -5.006802894058637e-06, - -12.943074226379395, - -13.32637882232666 + -5.960482212685747e-06, + -12.727540969848633, + -13.285879135131836 ] }, { "t": " number", - "lp": -1.7881409348774469e-06, + "lp": -1.4305124977909145e-06, "top": [ - -1.7881409348774469e-06, - -13.866220474243164, - -14.171846389770508 + -1.4305124977909145e-06, + -14.079712867736816, + -14.404281616210938 ] }, { "t": " only", - "lp": -6.9141628955549095e-06, + "lp": -6.318112355074845e-06, "top": [ - -6.9141628955549095e-06, - -12.916502952575684, - -13.10547161102295 + -6.318112355074845e-06, + -12.921635627746582, + -13.279397010803223 ] }, { "t": ".", - "lp": -5.960482212685747e-06, + "lp": -5.722062269342132e-06, "top": [ - -5.960482212685747e-06, - -12.199529647827148, - -14.765562057495117 + -5.722062269342132e-06, + -12.239629745483398, + -14.840055465698242 ] }, { "t": "", - "lp": -0.0075531224720180035, + "lp": -0.0035454225726425648, "top": [ - -0.0075531224720180035, - -5.275076866149902, - -6.088224411010742 + -0.0035454225726425648, + -6.3245530128479, + -6.404263019561768 ] } ], "expect_ok": true, "secs": [ - 15.27, - 9.52 + 0.85, + 0.44 ], "fails": [], "warns": [], @@ -920,117 +834,117 @@ "tokens": [ { "t": "7", - "lp": -0.02470596879720688, + "lp": -0.024654466658830643, "top": [ - -0.02470596879720688, - -3.738619089126587, - -7.851190567016602 + -0.024654466658830643, + -3.7376670837402344, + -8.071256637573242 ] }, { "t": "3", - "lp": -8.80995430634357e-05, + "lp": -0.00020702836627606302, "top": [ - -8.80995430634357e-05, - -10.439483642578125, - -10.578102111816406 + -0.00020702836627606302, + -9.506852149963379, + -10.116116523742676 ] }, { "t": "4", - "lp": -3.802848732448183e-05, + "lp": -6.80708180880174e-05, "top": [ - -3.802848732448183e-05, - -11.016250610351562, - -11.456581115722656 + -6.80708180880174e-05, + -10.708338737487793, + -10.808432579040527 ] }, { "t": "1", - "lp": -8.559592970414087e-05, + "lp": -0.00016702616994734854, "top": [ - -8.559592970414087e-05, - -10.144041061401367, - -11.475458145141602 + -0.00016702616994734854, + -9.669734001159668, + -10.999154090881348 ] }, { "t": ".", - "lp": -0.07091658562421799, + "lp": -0.06441079825162888, "top": [ - -0.07091658562421799, - -3.003324031829834, - -4.320361614227295 + -0.06441079825162888, + -3.1412062644958496, + -4.292909145355225 ] }, { "t": " Answer", - "lp": -0.666933000087738, + "lp": -0.723603367805481, "top": [ - -0.666933000087738, - -0.8862131834030151, - -3.0407867431640625 + -0.723603367805481, + -0.819618284702301, + -3.358370780944824 ] }, { "t": " with", - "lp": -0.006940552964806557, + "lp": -0.0022508781403303146, "top": [ - -0.006940552964806557, - -4.989425182342529, - -10.378595352172852 + -0.0022508781403303146, + -6.121297836303711, + -10.256690979003906 ] }, { "t": " the", - "lp": -8.821526535029989e-06, + "lp": -1.7285496141994372e-05, "top": [ - -8.821526535029989e-06, - -12.300503730773926, - -13.333985328674316 + -1.7285496141994372e-05, + -11.499177932739258, + -13.390902519226074 ] }, { "t": " number", - "lp": -2.2649790025752736e-06, + "lp": -2.3841887468734058e-06, "top": [ - -2.2649790025752736e-06, - -13.774499893188477, - -13.777214050292969 + -2.3841887468734058e-06, + -13.55058479309082, + -13.969475746154785 ] }, { "t": " only", - "lp": -1.3351529560168274e-05, + "lp": -2.002736073336564e-05, "top": [ - -1.3351529560168274e-05, - -11.888513565063477, - -12.979015350341797 + -2.002736073336564e-05, + -11.640507698059082, + -12.497420310974121 ] }, { "t": ".", - "lp": -2.3841887468734058e-06, + "lp": -7.987054232216906e-06, "top": [ - -2.3841887468734058e-06, - -13.593515396118164, - -14.47977352142334 + -7.987054232216906e-06, + -12.317981719970703, + -13.608646392822266 ] }, { "t": "", - "lp": -0.0005211399984546006, + "lp": -0.0018748323200270534, "top": [ - -0.0005211399984546006, - -7.661867618560791, - -11.284515380859375 + -0.0018748323200270534, + -6.5036516189575195, + -8.347963333129883 ] } ], "expect_ok": true, "secs": [ - 12.62, - 4.17 + 0.98, + 0.66 ], "fails": [], "warns": [], @@ -1045,63 +959,63 @@ "tokens": [ { "t": "7", - "lp": -0.0073203700594604015, + "lp": -0.0061628189869225025, "top": [ - -0.0073203700594604015, - -4.9429931640625, - -9.596504211425781 + -0.0061628189869225025, + -5.1176605224609375, + -9.789928436279297 ] }, { "t": "3", - "lp": -0.0006165020167827606, + "lp": -0.0006610551499761641, "top": [ - -0.0006165020167827606, - -8.345846176147461, - -8.954522132873535 + -0.0006610551499761641, + -8.283568382263184, + -8.856711387634277 ] }, { "t": "4", - "lp": -0.049714405089616776, + "lp": -0.05081043764948845, "top": [ - -0.049714405089616776, - -3.9123222827911377, - -4.662158489227295 + -0.05081043764948845, + -3.947129964828491, + -4.506924152374268 ] }, { "t": "1", - "lp": -7.629685569554567e-05, + "lp": -7.069360435707495e-05, "top": [ - -7.629685569554567e-05, - -9.710227966308594, - -12.183683395385742 + -7.069360435707495e-05, + -9.806951522827148, + -12.065498352050781 ] }, { "t": ".", - "lp": -0.1390085369348526, + "lp": -0.11820308119058609, "top": [ - -0.1390085369348526, - -2.738231658935547, - -3.041306495666504 + -0.11820308119058609, + -2.939655303955078, + -3.169112205505371 ] }, { "t": "", - "lp": -0.15726305544376373, + "lp": -0.13575804233551025, "top": [ - -0.15726305544376373, - -2.396639108657837, - -3.5623114109039307 + -0.13575804233551025, + -2.5934805870056152, + -3.506420850753784 ] } ], "expect_ok": true, "secs": [ - 15.8, - 12.06 + 1.16, + 0.82 ], "fails": [], "warns": [], @@ -1111,5 +1025,5 @@ } ], "pass": true, - "time": "2026-10-12T07:01:12-0300" + "time": "2026-10-20T19:15:23-0300" } \ No newline at end of file diff --git a/tests/golden/hw-gate/manifest.json b/tests/golden/hw-gate/manifest.json index 5c88d556..b571af57 100644 --- a/tests/golden/hw-gate/manifest.json +++ b/tests/golden/hw-gate/manifest.json @@ -17,7 +17,7 @@ }, "recorded_at": { "hrx-system": "82804197bcbe060b1371d6548337fdc842034cdd", - "llama.cpp": "8d9ce2c1aa60ed6b2334df3576e203e40c533a9c" + "llama.cpp": "8792923f8ba99637c562a52523a9fb831f57edfe" }, "text": { "file": "wiki.test.raw", @@ -27,7 +27,7 @@ "qwen3-4b-iq4xs": { "args": "-c 512 --chunks 8 -b 512 -ub 512 --device HRX0 -ngl 99", "base_bytes": 619931604, - "base_sha256": "f5d43376b320183d4b5ad35ef5ce586f92d0266aec6ddf3d06f503acf40d9d2e", + "base_sha256": "2afc23b39ebc7534cdb06f8b47d5258347f3389b7e8738eb8a85b0109a73ceea", "limits": { "max_kld_p99": 0.2, "max_mean_kld": 0.01, @@ -40,7 +40,7 @@ }, "recorded_at": { "hrx-system": "82804197bcbe060b1371d6548337fdc842034cdd", - "llama.cpp": "00f2a68de2e3275c75f1687222367cda817f2918" + "llama.cpp": "8792923f8ba99637c562a52523a9fb831f57edfe" }, "text": { "file": "wiki.test.raw", @@ -63,7 +63,7 @@ }, "recorded_at": { "hrx-system": "82804197bcbe060b1371d6548337fdc842034cdd", - "llama.cpp": "00f2a68de2e3275c75f1687222367cda817f2918" + "llama.cpp": "8792923f8ba99637c562a52523a9fb831f57edfe" }, "text": { "file": "wiki.test.raw", @@ -73,7 +73,7 @@ "qwen3.8-27b": { "args": "-c 512 --chunks 8 -b 512 -ub 512 --device HRX0 -ngl 99", "base_bytes": 1013178324, - "base_sha256": "7fc1cf24b5eb07bc15e9b6b28af9e15baddf960d85152e3d56eede265dd981ca", + "base_sha256": "d2f9a8af3f9d7d13f94b252dce0579c1b53411799578b4bedd5b1f5c64c427a3", "limits": { "max_kld_p99": 0.2, "max_mean_kld": 0.01, @@ -86,7 +86,7 @@ }, "recorded_at": { "hrx-system": "82804197bcbe060b1371d6548337fdc842034cdd", - "llama.cpp": "8d9ce2c1aa60ed6b2334df3576e203e40c533a9c" + "llama.cpp": "8792923f8ba99637c562a52523a9fb831f57edfe" }, "text": { "file": "wiki.test.raw", @@ -96,7 +96,7 @@ "zaya1-8b": { "args": "-c 512 --chunks 8 -b 512 -ub 512 --device HRX0 -ngl 99", "base_bytes": 1069596564, - "base_sha256": "13f3d4c1ad1b6b0da5994aee165df6edd2228dccc72d9ebeb49a9b6c33e6cd4d", + "base_sha256": "47174ec31a2537ed2d21d8a2c8fbd7e271c5effed9f1d9a918ce59591759adf4", "limits": { "max_kld_p99": 0.2, "max_mean_kld": 0.01, @@ -109,7 +109,7 @@ }, "recorded_at": { "hrx-system": "82804197bcbe060b1371d6548337fdc842034cdd", - "llama.cpp": "8d9ce2c1aa60ed6b2334df3576e203e40c533a9c" + "llama.cpp": "8792923f8ba99637c562a52523a9fb831f57edfe" }, "text": { "file": "wiki.test.raw",