Repository navigation
fix(lemonade): find llama-server under nested backend dirs (EAI-8026) - #242
Conversation
Lemonade's backend installer sometimes extracts llama.cpp one directory deeper than expected (e.g. rocm-stable/llama-b9752/llama-server), so the direct-serve lookup never found an installed backend even after a successful install. find_llama_server_binary now also checks one level of subdirectories, and llama_server_backend_label walks ancestor dirs to recover the correct backend name. EAI-8026 Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Adds a per-PR scenario that serves unsloth/Qwen3-0.6B-GGUF:Q4_0 via the owner/repo:variant syntax, exercising serve_hf_checkpoint's explicit bail on a missing llama-server backend. Unlike the existing short-recipe-name canary, this path isn't masked by the EAI-7423 managed-serve xfail, so a regression in the nested-backend-dir lookup fails this scenario directly. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
|
FYI for reviewer: this fix has been manually validated on a Strix Halo (gfx1151) machine |
michaelroy-amd
left a comment
There was a problem hiding this comment.
Approved. The nested lookup fixes the observed one-level Lemonade layout, preserves GPU-backend priority/no-CPU-fallback behavior, and adds unit plus Gherkin coverage. Local targeted tests, affected clippy/checks, workspace build/clippy, smoke, and all live CI checks passed.\n\nNon-blocking follow-up: engines/lemonade/src/lib.rs:2620-2626 uses fs::read_dir order when multiple build-numbered directories exist under one backend. That order is unspecified, so an upgrade that leaves two versions could select an older binary. Please define a deterministic candidate policy or document/enforce the single-build-directory invariant.\n\nDiagnostic nit: tests/e2e-cucumber/tests/e2e/serving_steps.rs:531-535 discards stderr on launch failure; including it would make this canary's failures more actionable.
#242 landed `Scenario: 14` in model_serving.feature while this branch was also using 14, and the two touch different parts of the file — so git merges them cleanly and the collision is invisible until someone tries to refer to "scenario 14" and finds two of them. Renumbers this branch's five additions to 15-19, after main's highest. No scenario, tag, step or expectation changes. Signed-off-by: fredespi <fredrik.espinoza@gmail.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#242 landed `Scenario: 14` in model_serving.feature while this branch was also using 14, and the two touch different parts of the file — so git merges them cleanly and the collision is invisible until someone tries to refer to "scenario 14" and finds two of them. Renumbers this branch's five additions to 15-19, after main's highest. No scenario, tag, step or expectation changes. Signed-off-by: fredespi <fredrik.espinoza@gmail.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#242 landed `Scenario: 14` in model_serving.feature while this branch was also using 14, and the two touch different parts of the file — so git merges them cleanly and the collision is invisible until someone tries to refer to "scenario 14" and finds two of them. Renumbers this branch's five additions to 15-19, after main's highest. No scenario, tag, step or expectation changes. Signed-off-by: fredespi <fredrik.espinoza@gmail.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`serve-hf-checkpoint-inference` has failed on every Strix Halo Windows run since it was introduced in ROCm#242: the managed lemonade serve of `unsloth/Qwen3-0.6B-GGUF:Q4_0` through the `owner/repo:variant` direct path exits 0, but `/v1/models` never lists the model. It is the lane's only host-independent unexpected failure, so the lane reads red on every open PR and a genuine new Windows regression is easy to miss. Declare it as the known bug it is, scoped to Windows GPU hosts only: the Strix Halo Ubuntu lane serves the same checkpoint fine, and on Windows the short-recipe-name path still passes, so a wider row would mask real regressions on both. Also shorten the serve wait to 240s. The lane leaves E2E_SERVE_TIMEOUT_SECS unset, so today it burns the full 600s default on a serve that never arrives. A healthy serve of this cache-warm checkpoint reaches ready in ~120s on this host, so 240s keeps ~2x headroom -- unlike the 90s the EAI-7423 rows use, which would sit below a healthy serve and hide the fix by keeping the scenario red after the bug is closed. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
#242 landed `Scenario: 14` in model_serving.feature while this branch was also using 14, and the two touch different parts of the file — so git merges them cleanly and the collision is invisible until someone tries to refer to "scenario 14" and finds two of them. Renumbers this branch's five additions to 15-19, after main's highest. No scenario, tag, step or expectation changes. Signed-off-by: fredespi <fredrik.espinoza@gmail.com>
Summary
rocm servefailed with "no GPU llama-server backend is installed" on Strix Halo (gfx1151) even right after a successfulrocm engines install lemonade, because Lemonade's installer sometimes extracts llama.cpp one directory deeper than expected (rocm-stable/llama-b9752/llama-serverinstead ofrocm-stable/llama-server).find_llama_server_binarynow also checks one level of subdirectories under each backend directory.llama_server_backend_labelnow walks ancestor directories to recover the correct backend name (rocm-stable,vulkan, etc.) even when nested.serve-hf-checkpoint-inference) that serves anowner/repo:variantHF checkpoint on lemonade, exercising theserve_hf_checkpointbail path directly instead of through the short-recipe-name router, whose managed-serve fallback is masked by the unrelated EAI-7423 xfail on Linux.Test plan
cargo test -p rocm-engine-lemonade find_llama_server_binary(includes new regression test reproducing the exact nested layout observed on Strix Halo)rocm-engine-lemonadetest suite passes (65/65)cargo clippy --locked --workspace --all-targets -- -D warningspassescargo check -p e2e-cucumber --testspasses with the new scenariorocm serve