[bong-water-water-bong]

What was measured

A loom agent saw requests 1 and 2 on Flash-Next decoding and prefilling 15-20 % slower than request 3. The same
thing exists on HRX with engine main, and it is a property of the first request after a load, not of the prompt:

Qwen3.8-27B on HRX, three fresh llama-server loads, three identical prompts then three different ones, all
with cache_prompt off, max_tokens 48 (llama-server's own timings):

request	decode tok/s	prefill of the same prompt
1	11.72 / 11.75 / 11.75	945 / 879 / 869 ms
2	12.41 / 12.46 / 12.42	730 / 731 / 733 ms
3	12.35 / 12.36 / 12.36	716 / 720 / 714 ms

Qwen3-4B, same protocol, three loads: the load whose model file was still cold decoded request 1 at 54.67 tok/s
against 71-72 later (−23 %); the two loads with the file in page cache showed −0.9 % and −1.4 %, so the 4B case
is dominated by first touch of the model.

What it is not

• not llama.cpp's warmup: --no-warmup leaves request 1 at 11.66 tok/s (the warmup graph is not the shape
  the client uses);
• not the GPU ramping: 9 s of continuous dispatch before the first request leaves request 1 at 11.74 tok/s
  (and a temperature sweep of the same kernel from 56 °C to 81 °C moves it 291.08 → 294.55 µs, i.e. nothing);
• not prompt caching: every measured request was sent with cache_prompt: false.

What it is

HRX's per-shape command-program build plus the first touch of that shape's buffers, paid by whichever request
uses the shape first. A synthetic request of the measured prompt's length, sent before the six measured ones,
moved the penalty onto itself and left request 1 at 12.47 tok/s, the same as requests 2-3 — and request 5 (a
different prompt length) then paid a fresh ~200 ms prefill build of its own.

The fix

serve_child now sends one synthetic request per backend as soon as the backend answers /health, before the
service reports itself ready: a ~465-token prompt with 8 decode steps, temperature 0, cache_prompt false,
HRX backends only, ONEBIT_NO_WARMUP=1 to turn it off. It runs on the lazily started laya backends too.

A/B through 1bit serve, two loads per arm, same protocol (request 1 against the best of requests 2-3):

arm	request 1 decode	request 1 prefill
ONEBIT_NO_WARMUP=1	−5.1 % / −7.2 %	890 / 930 ms
warm-up on (default)	+1.1 % / +1.1 %	872 / 930 ms

Cost: 2.1-3.3 s of load time on the 27B, logged as 1bit serve: warm-up request on port N took M ms.

What it does not fix

Only the decode shape and the warm-up's own prompt length are covered. A client prompt of another length still
builds its own prefill program once (~130-200 ms on the 27B — visible above as request 1 prefill 872-930 ms
against 720-726 ms for requests 2-3 of the same prompt). Removing that needs static prefill shapes (padding to
a fixed ubatch) in llama.cpp, which this change deliberately does not touch.

Checks

• scripts/hw-gate.sh full on the binaries this engine ships (build-warm/hrx/llama/bin, llama.cpp
  5dcf6623418b, hrx-system 82804197bcbe, gpu-kernels 8ebe8da), commit 5914c13: PASS on all seven
  models — qwen3.8-27b kat/kld/placement (152 graphs), zaya1-8b + qwen3-coder-30b-a3b + qwen3-4b-iq4xs
  kat/kld/placement, qwen3-0.6b + -kq8 + -kvq8 with ops 663/663, kat, kld, needle (7082-token prompt) and
  placement; 0 amdgpu ring timeouts across the three runs.
• the A/B above, and the pre-fix measurements, from ~/wt/mtpgap/{warm,warmreport,serve-warm}.{py,sh}.

🤖 Generated with Claude Code