[gaetan-puleo]

Stacked on #213 (it changes the reworked qsa3 kernel of that PR). Lifts the 262140-key limit of the QSA prefill attention, so Qwen3.8 Flash Next serves up to 1M tokens with YaRN, and bounds the indexer scratch that grew with depth.

Commits

1. cuda : QSA selected-key block ids in uint32, depth bound 1M. qsa_prefill stored the union's block ids as uint16 with 0xFFFF as padding, so it refused n_kv > 262140, while the model gate sent maskless ubatches there up to 16M keys: a prefill past 262140 keys aborted. Block ids are uint32 (sentinel 0xFFFFFFFF), both selected-key kernels and the model gate are bounded by GGML_QSA_MAX_KEYS = 1048576, deeper contexts take the masked route. Same change as cuda : widen QSA selected-key block ids to uint32 and lift the depth bound to 1M #124, ported to the reworked kernel (VGPR count unchanged at 192, no spill).
2. qwen4exp : an indexer score strip that shrinks with depth. The indexer scored fixed 512-query strips: the raw score F32 [n_blocks, n_idx_h, strip] and its ReLU were 2 x 2 GiB at 1M. The strip halves (rounding up) while the raw score exceeds 2^27 elements (its size at 262144 keys), down to 64. Nothing changes up to 262144 keys. Split KV streams keep one strip per ubatch, as before.
3. server : let an explicit RoPE scale extend the slot context. Slots were capped at n_ctx_train even with --rope-scale. The cap is now base / rope_freq_scale for a scale < 1 (not with --rope-scaling none; the base is --yarn-orig-ctx for YaRN when given). Limitation: an UNSPECIFIED scaling type is not resolved against the GGUF's training type (no public getter); for a GGUF whose training type is none the server would over-report the slot size.

Measurements (gfx1151, UD-Q4_K_XL, -c 1048576 --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 262144)

	before	this PR
prompt > 262140 keys	abort	runs
compute buffer at -c 1048576	5520 MiB	2257 MiB
GTT at load	114220 MiB	110956 MiB
needle at 5% of a 280k-token prompt	rejected / abort	found, 1776 t/s prefill
needle at 5% of an 840k-token prompt	—	found, 1455 t/s prefill, 15.7 t/s decode, GTT 112007 MiB
pp4096 / pp4096 @ d100000	2150 / —	2151 / 1791 t/s
PPL c2048 / c4096 / c4096 ub1024	2.0156 / 3.1703 / 3.1732	unchanged

[gaetan-puleo]

ok time to fix this one, i'll do it this evening

[klucsik]

The 262140 prefill ceiling bites at the model's native 262144 context, and it fails as
an 82 GiB compute-buffer OOM at load rather than as a route change.

We run Qwen3.8 Flash Next in production on a Ryzen AI Max+ 395 (128 GB, gfx1151,
rocm-10.0-strix-llama toolbox) in router mode, parallel = 1, ctx-size = 262144,
ubatch-size = 8192, f16 KV, spec-type = draft-mtp,ngram-mod. Same preset, unchanged for
10 days. After the container restarted and pulled a newer engine build, the model no longer
loads:

E ggml_backend_cuda_buffer_type_alloc_buffer: allocating 84116.75 MiB on device 0: cudaMalloc failed: out of memory
E ggml_gallocr_reserve_n_impl: failed to allocate ROCm0 buffer of size 88202805888
E graph_reserve: failed to allocate compute buffers
E llama_init_from_model: failed to initialize the context: failed to allocate compute pp buffers
E srv  llama_server: exiting due to model loading error

Deterministic - the same 84116.75 MiB on every one of ~25 retries over a 60 minute log.

Bisect

Bisected through the toolbox image tags (the engine is published as kyuz0/amd-strix-halo-toolboxes):

• last good build: rocm-10.0-strix-llama_20261020T200544, source ~67b998bdd - running, serving
• first bad build: rocm-10.0-strix-llama_20261028T060021

The only commit touching this path in that window is 54e7e6e35 "qwen4exp : QSA complete-block
selection with split KV streams", i.e. part of this PR's stack (#213). It tightened the depth
bound in qwen4exp_use_block_selection():

// before
n_kv > hparams.indexer_top_k + ratio - 1 && n_kv <= 16777216 && ...
// after
const bool depth_ok = n_kv <= 262140 || (n_stream == 1 && ubatch.n_tokens <= 512 && n_kv <= 262144);
n_kv > hparams.indexer_top_k + ratio - 1 && depth_ok && ...

Why 262144 is the bad number

262144 is not an exotic setting, it is n_ctx_train - 256K - and it is 4 keys over the
new ceiling. The cache size is GGML_PAD(plan.n_kv, 256u) (llama-kv-cache-dsv4.cpp:645) and
the reserve plan uses the whole cache (plan.n_kv = kv_size, dsv4_build_reserve_comp_plan),
so a context of 262144 asks the gate about 262144 keys and gets depth_ok = false for any
ubatch above 512 tokens.

Falling off the maskless route is not a graceful degradation for this model at this depth:

	gate true	gate false
qsa->bias	I32 [n_blocks*n_seq + 2*n_tps] (KiB)	F32 [n_blocks, n_tps, n_stream]
KQ mask	not read by the selected-key kernel	F32 [n_kv, n_tps], full width
score strips	trimmed by qsa_score_key_limits	untrimmed, ubatch/512 of them

With ratio = 4, n_ctx = 262144, ubatch = 8192: n_blocks = 65536, 16 strips of
F32 [65536, n_idx_h, 512] in raw + ReLU (~4.3 GiB/strip), a F32 [262144, 8192] mask
(~8.6 GiB) and a ~2.1 GiB bias. That is roughly 80 GiB, against the 82 GiB it asked for - so
the route switch, not a leak, is the whole failure. It agrees with what #216 measured from the
neighbouring strided-bias-view problem ("compute buffer 78 GiB", 11216 MiB -> 2043 MiB after).

To be clear about intent: the new gate is correct - #215's own description says a prefill
past 262140 keys used to abort in the kernel, so before 54e7e6e35 we were latently broken and
merely had not yet run a prompt deep enough to find out. The problem is the shape of the
failure. Before: fast route, aborts at depth. Now: silent route change, and at the model's
trained context on a 128 GB unified-memory box the reserve is simply unsurvivable. There is no
log line that says "maskless refused because n_kv > 262140" - the operator sees an allocator
error 1.5 seconds later and has to read the graph builder to find out why.

Asks

1. A warning when the route is refused for depth - one line naming n_kv, the bound and
   the consequence. llama : log the QSA maskless/masked route decision and why it was refused #110 asks for the route decision and its reason generally; this is a
   concrete case where its absence cost a production outage. Even better would be a note at
   context-setup time when n_ctx lands in the dead band between the kernel bound and the
   next padded cache size.
2. qwen4exp : 1M context (uint32 QSA block ids, depth-bounded indexer scratch, RoPE-scaled slots) #215 removes the ceiling, which is what actually unblocks this: with uint32 block ids
   and GGML_QSA_MAX_KEYS = 1048576, ctx-size = 262144 goes back on the maskless route.
   qwen4exp : multi-slot compute memory (split-stream strips, bias upload, compact visibility) #216 on top of it cuts the multi-slot compute memory further. Happy to test either on real
   hardware as soon as they are re-opened for testing - and to re-test on every merge after, if
   that is useful.

Workaround we are landing

ctx-size = 261888 (= 1023 x 256, the largest 256-aligned value under the gate), with
kv-unified-per-slot tracked to the same number. Costs 256 of 262144 context tokens and gets
the compact I32 visibility and the trimmed strips back. It also keeps us inside the kernel's
real limit, so it is strictly safer than the build we came from.

Verified / not verified

• Verified: the failure reproduces deterministically on the newer build with our preset
  (log above); 67b998bdd loads and serves the same preset on the same host today; 54e7e6e35
  is the only change to this predicate in the interval; the size arithmetic above.
• Not verified by us: that ctx-size = 261888 brings the compute buffer down on the new
  build. The GPU is a single 128 GB APU already saturated by the serving instance, so this
  needs a maintenance window. Plan is to run the failing build twice, changing only ctx-size,
  with GGML_ALLOC_DRY_RUN=1, and report the ROCm0 compute buffer for both. Will follow up
  here either way.
• Single Strix Halo machine, HIP backend only; nothing here was checked on Vulkan, CUDA or
  another APU.

Assisted-by: DeepSeek Harness (qwen3.8-FN)