[Defilan]

What

A new Lab build page: Qwen3.8-Flash-Next with MTP on one Strix Halo. Adds the page, a row in the lab index, and the nav entry.

Why

This is the second-opinion reviewer endpoint the lab stood up on an AMD Ryzen AI Max+ 395, and the page exists because two things about it are not obvious and cost time.

The first is that the model is half upstream and half not, and the split moved while the runtime was being assembled. The qwen4exp architecture merged upstream (ggml-org/llama.cpp#27742) and so did the long-context correctness fixes (#27941); the MTP draft graph (#28243) did not. That is why the runtime is an upstream pin plus one vendored pull request rather than a fork, and the page states the pin rule that follows from it (the tag is the commit the pull request was authored against, so bump both or neither).

The second is that this architecture has a documented history on exactly this chip of degrading at depth, which is the only regime a reviewer operates in, and the page records the retrieval test that says the upstream fixes deal with it.

How

The numbers come from a run on that box, and the two that carry the page are:

• A paired A/B against the same model with speculation removed, same image and same session minutes apart: 32.4 / 33.5 / 35.4 tok/s decode with MTP on against 24.6 / 25.8 / 27.5 off, so about 1.3x, consistent across three runs. Prefill is flat between the arms, which is the control that makes the decode difference mean something. The control arm reports accept=n/a (no speculation) so it is verifiably a control.
• A one-hour soak, 167 requests, zero errors, zero degenerate replies, decode median 34.6 tok/s with the second half faster than the first, and a GTT allocation that was byte-identical in all 54 samples. No amdgpu reset, device loss, ring timeout or OOM kill in dmesg.

Decode figures are the server's own timings.predicted_per_second, not completion tokens over a wall clock that includes the prompt pass. That distinction was worth 20 percent here and the page says so, because a table like this without it is not reproducible.

The page also records two failures honestly, since that is the part other people can reuse:

• A microk8s-hostpath model cache whose provisioner helper pod could never tolerate the GPU device taint on that node. The claim sits Pending, the InferenceService sits in Creating, and no error reaches the user. The operator's own docs describe this case as silent. Fixed with a static PV, and reported as a cluster issue rather than a model one.
• The choice between the self-contained MTP head and the shared- one, decided by reading the loader source for the borrowing path rather than by trying the recommended head first and reading the error.

Checklist

• Tests added/updated. Docs only. No Go, CRD or manifest in this repository is touched, so there is nothing to test here.
• make test passes locally. Not run: docs-only change, no Go compiled or changed.
• make lint passes locally. Same.
• Commit messages follow conventional commits.
• All commits are signed off (git commit -s) per DCO.
• AI assistance disclosed above.
• Documentation updated. This change is documentation.

Assisted-by: opencode (DeepSeek-V4.1-Flash) drafted the page from the run's own artifacts: the soak and probe logs, the node memory series, the runtime repo's Dockerfile and gates, and the applied manifests. I directed the experiment, chose what to measure, and reviewed every number against its source; the failures section is mine to stand behind.

The site sidebar half of publishing this is a separate change in Defilan/llmkube-web, because that sidebar is hand-maintained and does not read nav.yaml.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!