[bong-water-water-bong]

Gantt row "Lemonade bench: MTP on by default + warm-up on the one-hop path".

Lemonade's standard benchmark (2026-10-21, ~/amd-oss/lemonade-bench-2026-10-21) found two gaps on the product path (Lemonade onebit recipe, one hop):

1. Lemonade's own Qwen3.8-27B-GGUF entry has the mtp label, so its llamacpp recipe runs llama-server with --spec-type draft-mtp (the head is inside the UD-Q4_K_XL file, qwen35.nextn_predict_layers = 1, blk.64.nextn.*) and decodes 24-28 tok/s; onebit ran the same file without a drafter at 11-12.
2. 1bit serve: warm the backend before the first client request (the first request decodes 5-23% slower) #492's warm-up runs inside 1bit serve, but on the one-hop route lemond starts llama-server itself.

Changes

• MTP by default (launch_for): a .gguf with <architecture>.nextn_predict_layers > 0 on hrx gets --spec-type draft-mtp with no -md (Lemonade's exact rule and args). --mtp-max / --mtp-p-min apply; recipes see drafter=mtp. Off for --mtp / --dflash, --parallel > 1, --embedding / --reranking, cpu, and with --no-mtp or ONEBIT_NO_MTP=1 (Lemonade: onebit.args=--no-mtp, allowed by lemonade fork docs: status after steps 1-3, HRX re-verified at the first daily bump #16). Files without the key are unchanged (Qwen3.5-4B UD-Q4_K_XL: no head, plan unchanged apart from the warm-up).
• Warm-up on one hop: --print-backend adds "warmup": {"path", "body"} to an hrx chat plan: serve's own request, now shared code (warmup_body()). Lemonade fork docs: status after steps 1-3, HRX re-verified at the first daily bump #16 sends it after /health and before load() returns. ONEBIT_NO_WARMUP=1 leaves it out. I chose the plan field over a lemond-side copy of the request so the engine stays the single owner of what the warm-up is and when it applies.
• tests/print_backend.sh: 11 new checks (default drafter, its opt-outs, warmup key). docs/serve.md: both, with the numbers below.

Lemonade bench standard

lemonade bench MODEL --backend hrx --scenarios chat-short code-short --runs 5 --warmup 1 --auto-pull --json (the pinned CI's P1, a reload before every run). Private lemond on strixhalo (own cache/config dirs, --no-broadcast, cores 0-3, MemoryMax 110G), lemonade fork 1bit/onebit-warmup (#16) + this branch built with -DONEBIT_HRX=ON -DONEBIT_GPU_PRIVATE=<gpu-kernels 08bb2241> (the pin; 42 register_hip_* symbols in libggml-hrx.so). One binary for every arm; arms differ only by ONEBIT_NO_MTP / ONEBIT_NO_WARMUP in lemond's environment. Model files sha256-checked against HF LFS.

model	scenario	before	MTP only	warm-up only	after	Lemonade vulkan (earlier run)
Qwen3.8-27B UD-Q4_K_XL	chat-short tok/s	11.1	24.9	12.8	26.2	24.2
	chat-short TTFT ms	2298	2424	2325	2446	1882
	code-short tok/s	11.9	29.1	12.5	29.5	28.0
	code-short TTFT ms	724	807	741	824	1882
Qwen3.5-4B UD-Q4_K_XL (no MTP head)	chat-short tok/s	56.6	-	-	61.5	62.6
	chat-short TTFT ms	265	-	-	256	556
	code-short tok/s	59.2	-	-	60.8	63.2
	code-short TTFT ms	265	-	-	262	557

0/5 failed runs everywhere; no dmesg amdgpu faults/resets in any hold; no thermal stop.

Greedy equality (MTP vs --no-mtp, 27B, temperature 0, 128 tokens, 3 prompts)

2 of 3 identical. The third diverges at token 21: without MTP " Earth" logprob -0.684 vs " the" -0.705 (a 0.02 near-tie); with MTP the verify batch (2-4 tokens through the K-quant verify kernels) picks " the", and the rest is the same sentence. Drafting is correct: only tokens the verify pass agrees with are kept; batch-size numerics decide near-ties, as in upstream llama.cpp speculative decoding.

What the warm-up does not fix

The 2.3-2.4 s TTFT on chat-short's 67-token prompt stays. It is a first-time cost per prompt length, and some lengths cost much more than others: one server without MTP, after the warm-up: 67 tokens 2,350 ms, then 531 ms; 74 tokens 559 ms, then 528 ms (with MTP, a first 70-token prompt 1,585 ms, then about 600 ms). The warm-up's own 360-token prompt cannot cover other lengths. That needs its own row: which prefill shapes build expensive programs (67 vs 74 tokens), and whether those programs can be built at load or shared across lengths.

hw-gate

The private-add-on build is gated with scripts/hw-gate.sh --bin <this build>, split into holds of 14 minutes or less; I add the result as a comment. This PR changes no pin and no llama.cpp code.

🤖 Generated with Claude Code

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit e7653e1)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 492 - Partially compliant Compliant requirements: Warm the backend before the first client request to eliminate the 5-23% slower first request The warm-up is done via a synthetic request sent before the service reports itself ready Only HRX backends are warmed up The warm-up request uses a 360+ token prompt with 8 decode steps, temperature 0, cache_prompt false ONEBIT_NO_WARMUP=1 disables the warm-up Non-compliant requirements: The warm-up request should be exactly 465 tokens (the PR uses 360 + 60 * 10 = 960 tokens) Requires further human verification: The exact token count of the warm-up request needs verification against the hardware measurements 16 - Partially compliant Compliant requirements: Documentation updated to reflect status after steps 1-3 HRX re-verified at the first daily bump Docs updated to reflect MTP enabled by default for GGUF files with own MTP head Non-compliant requirements: None Requires further human verification: None

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Warm-up request token count The warm-up request uses 360 + 60 * 10 = 960 tokens, while the ticket specifies 465 tokens. This discrepancy could affect the warm-up effectiveness and performance measurements. static json warmup_body() { std::string text = "warm up the decode and prefill shapes"; for (int i = 0; i < 60; ++i) text += " lorem ipsum dolor sit amet"; return json{ { "messages", json::array({ { { "role", "user" }, { "content", text } } }) }, { "temperature", 0 }, { "max_tokens", 8 }, { "cache_prompt", false } }; } MTP head detection logic The own_mtp_head function checks for .nextn_predict_layers > 0, but does not verify that the actual MTP head exists in the model file. This could lead to incorrect behavior if the key exists but the head is not properly implemented. bool own_mtp_head(const Options& o, const std::string& device) { if (device != "hrx" || o.no_mtp || !o.mtp.empty() || !o.dflash.empty() || !o.role.empty() || o.parallel > 1) return false; if (const char* e = std::getenv("ONEBIT_NO_MTP"); e && *e && std::string(e) != "0") return false; if (fs::path(o.model).extension() != ".gguf") return false; return gguf_int(o.model, gguf_architecture(o.model) + ".nextn_predict_layers") > 0; }

[bong-water-water-bong]

PR-Agent's two notes: (1) gguf_int returns -1 for a missing key, a non-integer value or a non-GGUF file, so > 0 already leaves those unchanged (tests/print_backend.sh checks a file without the key). (2) The warm-up prompt is #492's, unchanged here; what it does not cover (other prompt lengths) is in the PR body as a follow-up row.

🤖 Generated with Claude Code

[github-actions]

Persistent review updated to latest commit e7653e1

[bong-water-water-bong]

hw-gate on this branch's build (a8b7060; llama.cpp 5dcf6623, hrx-system 82804197, private gpu-kernels add-on 08bb2241 compiled in), scripts/hw-gate.sh --bin <build>/hrx/llama/bin, one phase or model per GPU-token hold (each hold under 14 min):

step	result
ops (test-backend-ops HRX0, 9 op types) + fa-decode-q8	PASS: 663/663, 6/6
qwen3-0.6b, -kq8, -kvq8 (kat, kld, needle, placement)	PASS
qwen3.8-27b (kat, kld, placement)	PASS: 9 cases, KLD mean 0.000000 / p99 0.00004, top-1 100%, all on HRX in 152 graphs
qwen3-coder-30b-a3b	PASS
qwen3-4b-iq4xs	PASS
zaya1-8b	not completed: the 93 C Tctl abort fired 5-8 s into the load on 3 tries (box shared with other GPU work), plus 1 earlier rc 137

The 27B KLD ran against the base recorded for this branch's llama.cpp pin (the store's qwen3.8-27b.kld.pre-96f81977, sha256 d2f9a8af... = this branch's manifest). #493 re-recorded the store's file for 96f81977, so the shared store no longer matches older pins. No amdgpu page faults, resets or ring timeouts in dmesg after any hold. This PR changes no llama.cpp code or pin; these are main's binaries at those pins.

🤖 Generated with Claude Code