Repository navigation
--kv q4_0 faults on the first request when the KV streams (--kv-resident < --max-context) and MTP drafts: "checkpoint save: conversation snapshot running-state copy: unspecified launch failure" (0.1.43.1, RX 6900 XT gfx1030, Windows) #2403
Description
What happens
With --kv q4_0, the engine loads normally, but every request fails at once. The engine log says:
strata verify: captured the 6-token window (upload no error, sync no error)
strata verify: captured the 3-token window (upload no error, sync no error)
strata serve: checkpoint save: conversation snapshot running-state copy: unspecified launch failure
The client gets saving a conversation checkpoint failed, and the engine exits with code 1. The server restarts it,
and the next request fails the same way.
The checkpoint copy is only where the fault shows up. With AMD_SERIALIZE_KERNEL=3 AMD_SERIALIZE_COPY=3 HIP_LAUNCH_BLOCKING=1, the error comes from the first verify window instead:
strata verify: captured the 6-token window (upload no error, sync no error)
strata serve: verify: launch: unspecified launch failure
That is the cudaGraphLaunch in src/core/verify.cpp, so a kernel inside the captured verify graph faults.
What it needs
One request ("Name three rivers.", 64 tokens, no thinking) per config. Each config changes one thing from the failing one:
| config | result |
|---|---|
failing config (below): --kv q4_0, --max-context 65536 --kv-resident 32768, --spec 4 --mtp ... |
fault |
same, default KV (int8, no --kv) |
ok |
same, --kv-resident 65536 (no streaming) |
ok |
same, --max-context 8192 --kv-resident 8192 |
ok |
same, --spec 2, no --mtp |
ok |
same, STRATA_HIP_PROMPT_F16 unset |
fault |
same, no control vector (--control-vector-scaled etc.) |
fault |
| same, kernels serialized (env above) | fault (at verify: launch) |
same, STRATA_NO_BATCH_KV_STEP=1 |
fault |
same, STRATA_NO_MULTI_GR=1 |
fault |
So the fault needs all three: q4_0 KV, KV streaming (--kv-resident below --max-context), and MTP drafting.
--kv k8v4 with the same streaming and MTP setup runs without errors.
With STRATA_VERIFY_NODES=1, the 6-token verify graph includes 12 x kv_append_q4_kernel (which takes the
streaming KvHostPools), 12 x the KV-stream resolve_kernel / copy_kernel, and 12 x attn_chunk_kernel_pre75.
The q4_0 append into the host pool, or the stream map, are my first guesses, since the fault needs q4_0 and streaming
together. I have not confirmed which kernel faults. On Windows the driver reports the fault as a "driver timeout"
(the AMD Bug Report Tool pops up).
Failing command
strata.exe --serve --pack <packs>\q2_0 --native Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf
--ple-gguf Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf --expert-profile data\expert-profile.bin
--expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp <mtp>\rt --max-context 65536
--kv-resident 32768 --control-vector-scaled <...>speed-projection.gguf:1.0 --control-vector-layer-range 4 44
--cvec-mode project --cvec-dir per-layer --kv q4_0
Env: STRATA_HIP_PROMPT_F16=1 (it fails without it too).
Log lines that may matter:
strata generate: KV streaming: 32768 of 65536 cells per QSA layer in VRAM, the K/V in 0.42 GiB of pinned RAM
strata generate: expert cache 5585 slots, 7.19 GiB of VRAM
strata verify: window up to 6 tokens, 55.2 MiB of device buffers
strata serve: 2247 MiB of VRAM free with everything loaded
System
- Strata 0.1.43.1, prebuilt Windows HIP zip (ROCm 10.2.0a20260930)
- AMD Radeon RX 6900 XT 16 GB (gfx1030 / RDNA2), AMD Software 26.8.1 (driver 32.0.21045.5002)
- Intel i7-5930K (6 cores, AVX2, no AVX-512), 64 GB RAM, Windows 11 Pro 23H2 (22631)
- Model Q2_0 (GSQ-RCO, native pack)
Possibly related: #1932 (a device fault that also first shows up at checkpoint save, on gfx1100).
I can run more variants or a debug build if that helps.
Activity
ByteJuggler commentedon Oct 14, 2026
Still happens on 0.1.44.1 (prebuilt Windows HIP zip), with the same failing command: the engine log shows the same
checkpoint save: conversation snapshot running-state copy: unspecified launch failureon the first request. The same command with the default int8 KV still works.verify: a Q4_0 KV cache takes the batched verify window by default (S…
verify: a Q4_0 KV cache takes the batched verify window by default (S…
tamnguyenswe commentedon Oct 19, 2026
Hitting this on CUDA/Linux too, so probably not HIP-specific.
v0.1.45 docker (built from the repo Dockerfile,
CUDA_ARCHITECTURES=86), RTX 3080 10GB, Fedora 44, nvidia open driver 615.78.08. Qwen3.8-Flash-Next IQ3_XXS with--max-context 262144 --kv q4_0 --spec 4 --spec-min-p 0.5, plus the default mtp args. Didn't set--kv-resident.One difference from the original report: short requests were fine for me. I was running an agent session (claude code over /v1/messages) and the context grew from ~17k to ~66k without problems. The first request past 64k (65,861 prompt) also finished fine. The one after it died after 32 tokens:
Right after that the whole machine hard-froze and I had to power cycle it. Nothing in the kernel log, no Xid, the journal just stops. No engine log since it was in a
--rmcontainer.int8 KV at 128k is fine for me: same kind of prefix-cached multi-turn session, went to 110k with ~16 requests past 64k and no errors, so I'm sticking with that for now.
Niko1221 commentedon Oct 19, 2026
Thanks for taking the time to report this.
We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to read properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.
To reopen it takes about two minutes:
/reopen. A check runs and either reopens the issue or lists what is still missing.The rules are in CONTRIBUTING.md.