Skip to content

[Engine] DS4 0.1.52: ~13 tok/s decode observations on 4x RTX 4090; TP2 falls back to TP1 and sampled GPU utilization is asymmetric #4464

Open

Description

@oscarlius

[Engine] DS4 0.1.52: ~13 tok/s decode observations on 4x RTX 4090; TP2 falls back to TP1 and sampled GPU utilization is asymmetric

Summary

On the official Control 0.1.53 + DS4 engine 0.1.52 (build a42cc3d7d), a
DeepSeek-V4.1-Flash-1017 GSQ-RCO IQ2_XXS mixed pack (stored-weight estimate
about 2.0038 bpw) produced several native decode-log lines around 13 tok/s on
a 4x RTX 4090 system. This is an observation report, not a confirmed
regression or root-cause claim.

The engine startup says that requested TP2 cannot be used for the 2304 expert
dimension with 256-sized quantization blocks, and it falls back to TP1 with a
four-GPU layer split. All four GPUs have engine memory contexts, but that
does not establish simultaneous useful compute or tensor-parallel execution.

Environment

  • Strata Control 0.1.53; official DS4 engine 0.1.52, build a42cc3d7d.
  • DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ2_XXS, mixed quantization, about
    2.0038 bpw stored-weight estimate (not a 1.75 bpw pack).
  • 4x RTX 4090 (24 GB, 265 W limit each); two Xeon 8380 CPUs (80 physical
    cores / 160 threads total); about 256 GiB RAM.
  • Native flags/settings relevant to this observation: --max-context 262144,
    --prefill-chunk 2048, --experts 6, --dspark auto; text-only pack.
  • One native generation slot; 15 waiting slots behind an HTTP admission limit
    of 16. This is not a claim of 16 native generations in parallel.
  • Default output cap 32768; engine and queue timeouts 600 s.
  • The model data is on a PERC H755-backed pair of 10,000 RPM HDDs, not SSD.

Observed behavior

Native logs include these decode results, from different prompts, output
lengths, and cache states:

50 generated in 3847 ms  (13.00 tok/s)
103 generated in 7922 ms (13.00 tok/s)
143 generated in 10786 ms (13.26 tok/s)

The same retained log window also contains short-request results around
15--26 tok/s. They are not an A/B comparison and should not be read as a
stable performance range.

The startup report records a TP2-to-TP1 fallback because of the expert/quant
block geometry. The run reported 8,531 GPU expert slots and 6,829 CPU/RAM
expert slots. Per-request "CPU lane" figures varied from 0.5% to 17.5%; slot
counts are not a time breakdown and do not show a CPU-time root cause.

After the requests, a separate 12-second utilization sample showed GPU 0 at
0% SM utilization and GPUs 1--3 at 100%, with roughly 39 W / 100 W / 116 W /
124 W respectively. This sample was not taken during the user-observed low
utilization period. It therefore demonstrates neither full-run utilization
nor effective-compute saturation.

What has been checked

  • Three small correctness probes passed (arithmetic, literal copy, JSON).
    These are not a throughput, TTFT, long-context, or soak benchmark.
  • Actual probe timing fields contained draft_n values 5, 10, and 10. A
    generic Control field showing drafting=off should therefore not be used to
    conclude that DSpark was disabled.
  • The engine had no cgroup OOM or OOM-kill event in the recorded sample.
  • A 4-second disk-wide reading during loading measured 239.54 MiB/s. A
    different 4-second disk-wide reading after ingress was open measured 2.59
    MiB/s and 643.84 read IOPS. These are different-time whole-disk samples,
    not a cold-load average, process-specific I/O rate, or proof that disk I/O
    caused the ~13 tok/s observations.

Not claimed

Useful next evidence (not collected for this report)

  1. A controlled, same-input/same-output-length single-request decode run with
    per-token timing split for CPU lane, Engram/store waits, layer/synchrony,
    and GPU work.
  2. A synchronized utilization/power trace for the same run, rather than a
    later idle/other-work sample.
  3. A one-variable-at-a-time comparison of CPU-thread/NUMA placement,
    --prefill-chunk, and DSpark settings, with correctness retained.
  4. If storage is suspected, correlate per-request faults/reads with token
    timing before making any SSD or cache recommendation.

Related but not treated as the same root cause: #2275 and #3321.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions