Cancelling a streaming request during a long prompt read only takes effect after the full prompt is read #431

With "stream": true, a disconnect is detected (cancel=True, 0 tokens generated), but only after the whole prompt has been read. On a 57K-token prompt, the next request waited about 17 s.

Repro:

1. Stream a 57,396-token request and disconnect after 6 s:

   curl -s -N -m 6 127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d @long-57k-stream.json

2. Immediately send "Say OK." (max_tokens 30).

Observed:

strata serve: prompt 57396 tokens = 0 reused + 57396 read in 22151 ms (2591.2 tok/s), 0 generated ..., 2 checkpoints (cancelled)
[strata] done: 0 tokens in 22 s (0.0 tok/s) (disconnect, cancel=True)

The follow-up request took 17.2 s, even though it only needed about 0.9 s of engine time.

serve/server.py already checks cancel on every PP progress line. So either the engine doesn't act on STOP between prefill chunks, or --prefill auto makes the chunks so large that the first check comes late. I haven't checked which.

Expected: a cancel during the prompt read stops it at the next chunk boundary.

The release notes for 0.1.32 list "a cancelled long prompt followed by a new request" as checked. That claim holds: the new request is answered correctly with no engine restart. This issue is only about how long the new request waits.

Environment

Strata v0.1.32 (tag c499bd1), Linux, Ubuntu 24.04.4, kernel 7.0.0-34
GPU: Radeon AI PRO R9700 32 GB (gfx1201), single card
CPU: i5-12600K (AVX2, no AVX-512), 62 GB DDR5
Setup: ./setup.sh --backend hip --family coder --model IQ1_M (no system ROCm)
ROCm: TheRock 7.10.0a20251120 wheels (gfx120X-all) installed by setup into .venv; hipBLASLt 1.2.0
Server: serve/server.py with setup's generated config (max-context 131072, kv int8, spec 4 + MTP)