RPC: DeepSeek V4 output degenerates to a repeated token with two RPC workers after a long prompt, and the slot stays corrupted #30794

Name and Version

llama-server built from 42c787e (2026-10-07) on all three hosts, -DGGML_RPC=ON.

Operating systems

Linux (main host, one RPC worker), macOS (one RPC worker)

Which llama.cpp modules do you know to be affected?

llama-server, RPC backend (ggml-rpc-server)

Problem description & steps to reproduce

With DeepSeek-V4-Flash-0731 (unsloth UD-IQ4_XS GGUF, deepseek4 arch) split across the main host
and two RPC workers, a long prompt (~18k tokens) returns the same token over and over
(DDDDDDDD…, token id 38). The slot that processed it then stays broken: every later request
served from that slot, short ones included, returns the same repeated token, and requests with a
grammar (tools, response_format) fail with Unexpected empty grammar stack after accepting piece: D (38). Requests served by the other slot are fine. The same build, model, flags and requests
give correct output with no RPC, or with either RPC worker alone.

Setup (1 GbE LAN, ~0.3–0.6 ms between hosts)

Server command (host A)

llama-server -m DeepSeek-V4-Flash-0731-UD-IQ4_XS-00001-of-00004.gguf \
  --rpc <B>:50052,<C>:50052 --fit on --fit-target 1024 --ctx-size 65536 --parallel 2 \
  --batch-size 2048 --ubatch-size 512 --flash-attn on --jinja --reasoning-format deepseek \
  --reasoning off --temp 0.7 --top-p 0.95

Reproducer: dsv4_rpc_repro.py in this gist (Python standard library only). It sends, at
temperature 0 with thinking disabled: a warm-up and three short prompts; a ~18k-token prompt built
from llama.cpp/src/*.cpp; two concurrent 300-token requests; twelve OpenAI-style tool-calling
requests (--short: one); and three times a short "reply with exactly SHADOWFAX-EXACT-7" probe.

python3 dsv4_rpc_repro.py --url http://127.0.0.1:8080 --llama-src /path/to/llama.cpp [--short]

Results (fresh server for each row; full client logs in the gist)

Earlier runs of the same sequence through our benchmark harness: corrupted with B + C (2 of 2
runs), clean with B only, C only, and none (1 run each). With B + C but without the long prompt
first, the tool-call and probe requests were all correct (2 runs).

Observations

• The first bad reply is the long prompt itself. Afterwards, every request served by that slot
  degenerates, even after its cache is cut back to 1 token (cached_tokens: 1) or reused (18
  tokens). The other slot keeps answering correctly (concurrent_1, most tool calls).
• Throughput looks normal (no crash, no RPC error in the server log at default verbosity).
• Either worker alone is fine, so this does not look like a CUDA-on-sm_120 or Metal kernel
  problem by itself, but like state that goes wrong only with two RPC workers. Possibly related
  to RPC: [create_node] invalid data ptr — split scheduler ships compute-flagged views to the wrong RPC worker (reproduced with deepseek4 and glm-dsa, >=2 workers, master) #28047 (split scheduler shipping views to the wrong RPC worker with >= 2 workers, also
  reproduced with deepseek4), which crashed rather than corrupting output; this build includes
  its fix.
• Not tried yet: other models, a single host with two ggml-rpc-server processes, or which part
  of the prompt length triggers it.

First Bad Commit

Unknown (only 42c787e tested).

Relevant log output

DEGENERATE prefill_16k              prompt=18455 cached=0  276.1s  :: 'DDDDDDDDDDDDDDDDDDDD…'
ok         concurrent_1             prompt=20 cached=0   37.2s  :: '**TCP Congestion Control: Variant 1 (Tahoe)**…'
DEGENERATE concurrent_2             prompt=20 cached=0   37.2s  :: 'DDDDDDDDDDDDDDDDDDDD…'
DEGENERATE tool:error_recovery      prompt=907 cached=1  138.4s  :: 'DDDDDDDDDDDDDDDDDDDD…'
DEGENERATE exact_1                  prompt=22 cached=1    4.0s  :: 'DDDDDDDDDDDDDDDDDDDD…'

server: got exception: Unexpected empty grammar stack after accepting piece: D (38)