Repository navigation
Description
[Engine] DS4 0.1.52: ~13 tok/s decode observations on 4x RTX 4090; TP2 falls back to TP1 and sampled GPU utilization is asymmetric
Summary
On the official Control 0.1.53 + DS4 engine 0.1.52 (build a42cc3d7d), a
DeepSeek-V4.1-Flash-1017 GSQ-RCO IQ2_XXS mixed pack (stored-weight estimate
about 2.0038 bpw) produced several native decode-log lines around 13 tok/s on
a 4x RTX 4090 system. This is an observation report, not a confirmed
regression or root-cause claim.
The engine startup says that requested TP2 cannot be used for the 2304 expert
dimension with 256-sized quantization blocks, and it falls back to TP1 with a
four-GPU layer split. All four GPUs have engine memory contexts, but that
does not establish simultaneous useful compute or tensor-parallel execution.
Environment
- Strata Control 0.1.53; official DS4 engine 0.1.52,
build a42cc3d7d. - DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ2_XXS, mixed quantization, about
2.0038 bpw stored-weight estimate (not a 1.75 bpw pack). - 4x RTX 4090 (24 GB, 265 W limit each); two Xeon 8380 CPUs (80 physical
cores / 160 threads total); about 256 GiB RAM. - Native flags/settings relevant to this observation:
--max-context 262144,
--prefill-chunk 2048,--experts 6,--dspark auto; text-only pack. - One native generation slot; 15 waiting slots behind an HTTP admission limit
of 16. This is not a claim of 16 native generations in parallel. - Default output cap 32768; engine and queue timeouts 600 s.
- The model data is on a PERC H755-backed pair of 10,000 RPM HDDs, not SSD.
Observed behavior
Native logs include these decode results, from different prompts, output
lengths, and cache states:
50 generated in 3847 ms (13.00 tok/s)
103 generated in 7922 ms (13.00 tok/s)
143 generated in 10786 ms (13.26 tok/s)
The same retained log window also contains short-request results around
15--26 tok/s. They are not an A/B comparison and should not be read as a
stable performance range.
The startup report records a TP2-to-TP1 fallback because of the expert/quant
block geometry. The run reported 8,531 GPU expert slots and 6,829 CPU/RAM
expert slots. Per-request "CPU lane" figures varied from 0.5% to 17.5%; slot
counts are not a time breakdown and do not show a CPU-time root cause.
After the requests, a separate 12-second utilization sample showed GPU 0 at
0% SM utilization and GPUs 1--3 at 100%, with roughly 39 W / 100 W / 116 W /
124 W respectively. This sample was not taken during the user-observed low
utilization period. It therefore demonstrates neither full-run utilization
nor effective-compute saturation.
What has been checked
- Three small correctness probes passed (arithmetic, literal copy, JSON).
These are not a throughput, TTFT, long-context, or soak benchmark. - Actual probe timing fields contained
draft_nvalues 5, 10, and 10. A
generic Control field showingdrafting=offshould therefore not be used to
conclude that DSpark was disabled. - The engine had no cgroup OOM or OOM-kill event in the recorded sample.
- A 4-second disk-wide reading during loading measured 239.54 MiB/s. A
different 4-second disk-wide reading after ingress was open measured 2.59
MiB/s and 643.84 read IOPS. These are different-time whole-disk samples,
not a cold-load average, process-specific I/O rate, or proof that disk I/O
caused the ~13 tok/s observations.
Not claimed
- This report does not establish that HDD-backed Engram reads/page faults,
the TP fallback/layer handoff, CPU or NUMA scheduling, prefill chunking, or
DSpark acceptance is the cause. - It does not establish that
--threadswas omitted incorrectly or that
prefill chunking was ineffective. - It does not claim that PR [Engine] Parallelize staged conversation cache copies #4002 applies to DS4; that PR is Qwen-specific.
- It does not claim a benefit from PR [Engine] DeepSeek prompt: the busy experts take FP16 GEMMs, not the tiles (DS4_FAT_ROWS, opt-in) #4366; it remains unmerged and was
not enabled in this official engine.
Useful next evidence (not collected for this report)
- A controlled, same-input/same-output-length single-request decode run with
per-token timing split for CPU lane, Engram/store waits, layer/synchrony,
and GPU work. - A synchronized utilization/power trace for the same run, rather than a
later idle/other-work sample. - A one-variable-at-a-time comparison of CPU-thread/NUMA placement,
--prefill-chunk, and DSpark settings, with correctness retained. - If storage is suspected, correlate per-request faults/reads with token
timing before making any SSD or cache recommendation.
Related but not treated as the same root cause: #2275 and #3321.
Activity