[Benchmark] RTX 5080 + RTX 4060 Ti 16 GB / 32 GB RAM: Flash-Next Coder IQ1_M - stock 0.1.52 vs the Strata-DualGPU fork (after its own tuner) #4470

Hardware: RTX 5080 16 GB (headless, HDMI dummy) + RTX 4060 Ti 16 GB (drives the desktop), AMD Ryzen 7 9800X3D, 32 GB RAM, Windows 11, driver 617.14. The 4060 Ti is in a chipset slot, so the inter-GPU link is slow - this is the worst case for a layer split.

Model: Qwen3.8-Flash-Next Coder IQ1_M (ISTA-DASLab GSQ-RCO), 64K context, MTP draft layer (--spec 4), resident experts.

Software compared:

• stock app 0.2.7 / engine 0.1.52 (build a42cc3d+cuda13.0), and the same engine binary spawned directly by serve/server.py with the app's argument set
• Strata-DualGPU fork (Hardin22, commit 03a5ec9, engine 0.1.38-dual, CUDA 13.3, sm_120) after running its own 5-minute tuner on this PC

Method: three fixed prompts, greedy (temperature 0), rates read engine-side from the engine log (generated tokens / generation time): a Python coding prompt capped at 4096 tokens (first generation after a fresh load = "cold"), a short snippet prompt ("warm"), and a 49,361-token prompt probe. Desktop in use during all runs; the fork was loaded 4 times and the spread is reported as a range.

Results (decode tok/s):

*the 49K stock row was measured with --pipeline-windows 2 on; without the flag the warm decode was 59.3, so the stock 55.9 at depth is probably slightly understated. Stock rows ran KV q4_0, the fork int8 (q4_0 KV is the slightly faster of the two here, so this favors the stock column).

Notes that may be more useful than the raw numbers:

1. The fork's tuner found more than the version gap. On this Zen 5 8-core CPU its worker sweep went 7 workers = 41.9, 5 = 54.4, 4 = 66.2 tok/s - fewer workers is much faster, presumably because the host loop needs whole cores; its defaults were tuned on a hybrid Intel. And its draft-floor sweep (spec-min-p) went 0.30 = 28.6, 0.50 = 29.2, 0.70 = 50.6 tok/s in its own protocol. The first fork column above was measured at floor 0.50, so it is a lower band, not the fork's ceiling.

2. On stock 0.1.52, the serving config dominates everything. A single conversation through the app's pipelined batch path (Concurrent requests = 4) decodes at 14.9 tok/s and pays ~26 s of window captures at the start of every new conversation; Concurrent requests = 2 takes the serve path at 39.6; the same engine behind the light host does 59.3. Details in my comment on [Engine] 0.2.7 single-stream decode −11.7% on 2× RTX 2080 Ti with IQ3_S and --native (105.9 → 93.5 tok/s); prefill and concurrency unchanged #4410.

3. --pipeline-windows 2 alone on 0.1.52 cost 18% warm decode here (59.3 to 49.4/48.7, reproduced on two loads), prefill unchanged.

4. The prefill gap is the most stable number of all: 2,310-2,506 vs 228 tok/s on the same 49.4K prompt, x10 across every load. On stock the engine reads ~449 GB from the model file during these benches (resident experts spilling), which matches what the fork's docs describe fixing with per-card weight trimming.

Context: same hardware class as the fork author's (5080 + 4060 Ti, 32 GB); his numbers and chemical12's (#1352) are in the same band. Sharing as a data point for anyone on a two-unlike-cards / 32 GB rig deciding between configs - and because #4410 suggests single-stream regressions on 0.2.7 that our config sweeps touch from another angle.