[cedrik-fuoco]

Problem and result:

I found that with STRATA_PREFILL_LAYER_MAJOR=1 and STRATA_HC_REQ8=1 the layer-major prompt path loses the prompt. My probe is a needle copy: a six-character code sits near the start of the prompt and the question at the end, and the code stops coming back at every length I tried, 1,567, 4,045, 12,037 and 100,003 tokens. It happens on the Q2_0 pack and on IQ3_XXS, with the needle at the start, middle or end, and with STRATA_PREFILL_MMQ=0. With HC_REQ8 off the same prompts answer exactly, and after this change they answer exactly with it on.

Related discussion: #3976, where this path was proposed and measured. I do not think anything in that thread contradicts this finding. Its quality evidence was measured without STRATA_HC_REQ8, which is the switch this needs, and STRATA_HC_REQ8 refuses a layer split, so the multi-stage path where the walk is on by default cannot reach the defect today.

Approach

The three hyper-connection bf16_proj calls in run_layer_major were not passing the optional stream argument. Under HC_REQ8 those projections are stored as int8 and bf16_proj expands them into a static scratch buffer on whichever stream it is given. With the argument missing that is the default stream, and the GEMM that reads the buffer runs on the prompt path's own stream. run_impl passes m.cs at the equivalent call sites. My change passes the same stream at the down, up and inject calls and touches nothing else.

The second commit adds a regression check for the property that broke, because bf16_proj is file-static and no test can reach it. I did not carry two other arguments that this path also misses, ldx for STRATA_PF_HCDOWN and STRATA_HCD_EXACT, and xf16 for the m.xn_f16 image. Neither is set in my configuration and I am not claiming either one fixed here.

Validation

Everything below was run at bd14107 and d5a0d82 on an RTX 3070 Ti with 8 GiB and compute capability 8.6, one GPU and no layer split. The machine is a Ryzen 9 9900X with 61.6 GiB of DDR5-6000 on Windows 11. The engine is 0.1.53 from the 0.2.7.1 tree plus this change, built with CUDA 12.6 and MSVC 18.4 at -DCMAKE_CUDA_ARCHITECTURES=86. The model is Qwen3.8-Flash-Next GSQ-RCO on the Q2_0 pack, with --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp --max-context 131072 --kv int8 --kv-resident 16384 --vram-reserve-mib 1400, STRATA_HC_REQ8=1, and the config's own STRATA_* set.

	Check	Result
Passed	CUDA build, sm_86	rc=0
Passed	ctest -R hc_q8_stream	1 of 1 passed, 0.16 s
Passed	Needle, walk on and off, 4K / 16K / 32K at depths 10 / 50 / 90 %	9/9 and 9/9 exact. Every walk-on arm at these lengths failed before the change.
Passed	Needle, walk on, 1,567 / 4,045 / 12,037 / 100,003 tokens, two codes each	exact in all eight runs
Passed	KL against the walk-off run, same 20 prompts, prompt chunk pinned to 512 so the walk engages, 10 engagement lines against none	top-1 100.00 %, KL mean 0.00967, median 0.00000, max 0.1357

Prefill with the walk on and off

Same build and same prompts, only STRATA_PREFILL_LAYER_MAJOR flipped, with --expert-cache 600 pinned on every arm so the expert cache is not part of the comparison.

Two runs per arm, needle at 50 % depth, and all twelve runs recalled the needle exactly.

prompt	walk on, tok/s	walk off, tok/s	delta
12,036	1,182 / 1,210	787 / 765	+54 %
32,043	1,193 / 1,147	810 / 809	+45 %
100,002	1,238 / 1,237	774 / 764	+61 %

The two runs of each arm agree within 4 % and every arm ran at the same 600 slots, so these numbers show the switch rather than the state of the machine. They are about the switch and not about this fix anyway. The fix only changes which stream a matrix is expanded on, so it adds no work. With three lengths I am not going to claim a trend, but all three came out between +45 % and +61 %, which is a good deal more than the earlier runs with an auto-sized cache suggested.

Limits: verified only on the single-GPU configuration above. The timings above measure the switch's effect on this configuration, not the fix.