Repository navigation
Description
AMD/HIP (gfx1100, RX 7900 XTX): engine loads the Coder pack fully, then verify: launch: unspecified launch failure on the first request — or hangs before ready. No working configuration.
Engine 0.1.29, build from source on Linux, HIP backend, RX 7900 XTX (gfx1100). The build succeeds, the pack prepares, the engine loads the whole model — and it never serves a token.
What happens
Three configurations, all on the same box and pack:
1. --spec 4 (what setup.sh --backend hip wrote). The server comes up and /v1/models reports the model loaded. The first chat request kills the engine:
strata generate: PCIe probe: 7.0 GB/s host->device -> pcie_frac 0.15 (default 0.55)
strata verify: window up to 6 tokens, 74.0 MiB of device buffers
strata verify: captured the 6-token window (upload no error, sync no error)
strata serve: verify: launch: unspecified launch failure
the server then reports the engine stopped unexpectedly (exit code 1), done: 0 tokens in 5 s (0.0 tok/s) (error, cancel=False), and a retry gets the engine stopped unexpectedly (exit code 1) again.
2. --spec 2 (the smallest window the pack accepts). Loads to completion — 644 MiB of VRAM free with everything loaded, session is up (engine 0.1.29) — then never signals ready: the API port stays closed and the log stops.
3. --spec 4 plus the flags docs/AMD_HIP.md prescribes for AMD (--prefill 512 --pool-workers 15 --vram-reserve-mib 1024 --adapt-every 0 --pcie-frac 0). Same hang as (2), with 946 MiB of VRAM free with everything loaded.
--spec 0 cannot be used to test around the verify path: the engine rejects it with
is a native (IQ) pack: it needs --native SHARD1, --spec T (T >= 2) and --prefill CHUNK
so there is no configuration on this hardware that reaches a generated token.
Why it reads as the verify kernel rather than the transfer
The window is allocated and its upload and sync both report success (upload no error, sync no error) on the line immediately before the launch failure, so the fault looks like the kernel launch itself rather than the buffer setup. This release touched exactly that area (#187 QSA block scores read each key block once for all of a verify window's queries, #148 bitwise test of the verify window's multi-token GEMV contract, #154 guards on the verify window's tables), and #228 (QSA select on tensor cores via rocWMMA, opt-in, gfx1100) is in the same neighbourhood — possibly relevant, possibly not.
Environment
- GPU: AMD Radeon RX 7900 XTX (gfx1100), 24 GB VRAM — the card
docs/AMD_HIP.mdnames - ROCm: 7.1.52802 (system install under
/opt/rocm,hipcc+hipBLAS+hipBLASLtpresent), HIP compiler/opt/rocm/llvm/bin/clang++— i.e. the ROCm versiondocs/AMD_HIP.mdrecords as the one this backend was tested with - OS: Ubuntu 24.04, kernel 7.0.0-34-generic; CPU AMD Ryzen 7 7700X (AVX-512), 16 threads; 46 GB RAM; model data on NVMe
- Build:
cmake -S . -B build-hip -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_HIP=ON -DSTRATA_ENABLE_CUDA=OFF -DCMAKE_HIP_ARCHITECTURES=gfx1100→engine/strata16.9 MB;engine/BUILD.jsonreports{"source":"local-hip","backend":"hip","version":"0.1.29","archs":["gfx1100"],"vision":"none","lib_dirs":["/opt/rocm/lib"]} - Pack: family
coder, sizeIQ1_M, prepared by./setup.sh --backend hip --family coder --model IQ1_M --vision no --context 4096 --experimental-speed-projection off --low-ram auto - Load lines from the engine's own log:
experts loaded: 23.42 GiB at 0.19 GiB/s (143 s so far),filling the GPU's expert cache (9367 experts, 17.85 GiB of VRAM)
Reproduce
ROCM_PATH=/opt/rocm ./setup.sh --backend hip --family coder --model IQ1_M --vision no --context 4096
./run-coder-iq1_m.sh
# wait for "ready: http://127.0.0.1:<port>/v1", then send one chat completionWhat I did NOT test
- The other quants (Q2_0, IQ2_XS, IQ3_XXS, IQ3_S need ~48–62 GB system RAM per the installer's own figures; this box has 46 GB, so only the Coder fits)
- The vision path (disabled), non-gfx1100 AMD cards, Windows+HIP, and multi-GPU
- Only one GPU on the box is usable for this; the integrated Radeon (gfx1036) is correctly reported as unsupported
Scope
Nothing here suggests the Coder weights, the pack, or the download are at fault — the engine loads 23.42 GiB of experts and exits inside its own GPU kernels before it reads a prompt. Your release notes say AMD was not retested for this release because the AMD test PC was unreachable; this is data from a gfx1100 machine that does have the card and the ROCm version named in docs/AMD_HIP.md, so hopefully it saves a round trip. I can re-test against a fix or a diagnostic build, and I'm happy to run any test you'd like instrumented.
Activity
Maxritz commentedon Sep 30, 2026
https://github.com/Maxritz/Strata-rocm
rwkeyes commentedon Sep 30, 2026
Confirmed fixed on 0.1.30 — same machine, same pack, same failure point as the original report.
0.1.29: the Coder IQ1_M pack loaded fully (23.42 GiB), then the first request died with
verify: launch: unspecified launch failure. No configuration got past it:--spec 4and--spec 2both faulted, and--spec 0is rejected for a native IQ pack ("needs --native SHARD1, --spec T (T >= 2) and --prefill CHUNK"), so the verify path could not be bypassed.0.1.30: builds and serves. Verified end to end, twice, on the same pack.
Environment: RX 7900 XTX (gfx1100), ROCm 7.1.52802 from
/opt/rocm(note: the shell exportedROCM_PATH=/opt/rocm-6.2.1, which makes detection fail and pull a TheRock nightly — overriding it to/opt/rocmused the system ROCm), source HIP build (ROCM_PATH=/opt/rocm ./setup.sh --backend hip --build), Coder IQ1_M native pack,--context 65536,--kv int8, Ryzen 7 7700X (16 cores, AVX-512) / 46 GB RAM, HDD-backed weights.Measured:
/v1/chat/completions) and Anthropic (/v1/messages) both answer; tool-call smoke passesThe 4K default context was the only thing I had to change for real use; raising it also made it faster here, consistent with your card table.
Expected on this card, already tracked: the "no hipBLASLt tuning table for gfx1100" warning (dense matmuls fall back to plain hipBLAS) — #170.
@Maxritz thanks for the fork pointer — upstream 0.1.30 covers my case, so I haven't needed it, but I'll take a look at it for the gfx1100 tuning table.
Nothing else outstanding from my side — happy to close this if that's useful.
Niko1221 commentedon Oct 1, 2026
Closing as fixed on 0.1.30 (confirmed above). We added your configuration (low-RAM resident, --pcie-frac > 0, the first verify window) to our AMD smoke tests.
Add a default-off deferred-reclaim diagnostic