Repository navigation
[Benchmark] Radeon AI PRO R9700 + RX 7900 XTX (PCIe 3.0 x1), DeepSeek-V4.1-Flash IQ2_XS, Linux source build 8183c6b #3933
Description
What did you measure, or what went wrong
DeepSeek-V4.1-Flash IST-DAS IQ2_XS (2.25 bpw) on a mixed AMD pair, Radeon AI PRO R9700 (gfx1201) + RX 7900 XTX (gfx1100), Linux, source-built strata-dsv41. As far as I can tell this pairing, and RX 7000 under the DeepSeek engine, has no upstream numbers yet. Every run is listed below, including the failed one and the slow ones.
Hardware / software
| GPU 0 (HIP 0) | Radeon AI PRO R9700 32 GB, gfx1201, PCIe 5.0 x16 (CPU lanes) |
| GPU 1 (HIP 1) | RX 7900 XTX 24 GB, gfx1100. Upstream link is PCIe 3.0 x1 behind the B650 chipset (measured earlier at 0.4–0.8 GB/s H2D) |
| iGPU | Raphael gfx1036, hidden with HIP_VISIBLE_DEVICES=0,1 (see failure 1) |
| CPU | Ryzen 5 7600X (6C/12T, Zen 4, AVX-512 VNNI rows picked) |
| RAM | 62 GiB DDR5, 100 GiB swapfile on another NVMe |
| Model disk | Kingston NV3 2 TB NVMe (btrfs); the engine measured 5.3 GB/s |
| OS | Ubuntu 24.04, kernel 7.0.0-38 |
| ROCm | TheRock multi-arch wheels rocm[libraries,devel,device-gfx1100,device-gfx1201]==10.1.0a20260822 |
| Strata | 8183c6b6 (0.1.49.1), -DSTRATA_BUILD_DSV41=ON -DSTRATA_NATIVE_EXPERTS=ON -DCMAKE_HIP_ARCHITECTURES="gfx1100;gfx1201" |
| Pack | pack_gguf.py with the setup.py IST-DAS recipe (--shards ×4, --engram, --engram-meta UD shard 1, --dspark-gguf, noise 128799, markov rank 256) plus data/dsv41/profile-2.25.bin. --pack-info: every tensor is a device type, all files present. tools/dsv41/smoke.py --gpus 0,1: "The capital of France is Paris." |
All files were checked against the setup.py SHA-256 pins.
Method. Each run is one CLI process: strata-dsv41 --pack P --ids @F --n N --max-context 16384 --eos 1 --verbose --gpus 0,1 --cpu-threads 5 <arm>, launched as taskset -c 0-4,6-10 nice -n 5, so one physical core stays free for the desktop. Every run loads the model fresh (about 50–60 s with both cards, about 20 s on the R9700 alone). Thinking is off (<｜Assistant｜></think>) and decoding is greedy. Speeds are taken from the engine's own prompt: / decode: lines.
- Short prompt: 54 tokens (write an ISO-8601 duration parser).
- Long prompt: 12,593 tokens (review of
tune.py+smoke.py+dsv41_fit.py).
All arms with exit code 0 gave sensible text. I spot-checked several by decoding them; for example, the baseline wrote a correct parse_iso8601_duration.
Results. Prompt and decode are in tok/s, "store hit" is the share of CPU-lane experts found in the RAM store, and swap-out is in MB during the run.
| Run | Prompt tokens | New tokens | Settings beyond the defaults above | Prompt | Decode | RAM store GB | Store hit % | Disk MB/token | Peak RSS GB | Swap-out |
|---|---|---|---|---|---|---|---|---|---|---|
| base-short | 54 | 128 | (release defaults: --dspark auto did not load the drafter) |
2.07 | 6.27 | 11.3 (auto) | 42.5 | 353.5 | 15.0 | – |
| store24 | 54 | 128 | --ssd-cache 24 |
2.12 | 7.55 | 24 | 56.4 | 280.8 | 26.5 | 1 |
| store32 | 54 | 128 | --ssd-cache 32 |
1.93 | 6.82 | 32 | 60.4 | 257.4 | 32.9 | 1235 |
| store40 | 54 | 128 | --ssd-cache 40 |
1.87 | 7.25 | 40 | 65.9 | 242.6 | 39.1 | 0 |
| s2-base | 54 | 256 | --ssd-cache 32 |
2.04 | 8.85 | 32 | 60.5 | 216.8 | 33.3 | 0 |
| s2-base-rep | 54 | 256 | --ssd-cache 32 |
2.08 | 7.72 | 32 | 60.2 | 224.8 | 33.3 | 0 |
| s2-nocache | 54 | 256 | --ssd-cache 32 --no-expert-cache |
2.74 | 8.89 | 32 | 82.9 | 155.4 | 33.2 | 0 |
| s3-nc-a | 54 | 256 | same | 2.18 | 5.65 | 32 | 81.6 | 170.6 | 33.3 | 0 |
| s3-nc-b | 54 | 256 | same | 2.32 | 7.47 | 32 | 81.3 | 171.0 | 33.2 | 0 |
| s2-lfu | 54 | 256 | --ssd-cache 32 --ssd-policy lfu |
2.12 | 8.00 | 32 | 57.6 | 236.8 | 33.3 | 0 |
| s2-thr4 | 54 | 256 | --ssd-cache 32 --cpu-threads 4 |
2.06 | 7.82 | 32 | 60.3 | 224.1 | 33.3 | 0 |
| s2-r9700 | 54 | 256 | --ssd-cache 32 --gpus 0 |
6.43 | 7.76 | 32 | 64.8 | 295.7 | 31.4 | 0 |
| s2-dspark | 54 | 256 | --ssd-cache 32 --dspark on --dspark-conf 0.65 |
1.91 | 9.86 | 32 | 67.1 | 199.6 | 33.5 | 0 |
| s3-dspark-a | 54 | 256 | same | 1.98 | 9.82 | 32 | 67.4 | 201.2 | 33.5 | 0 |
| s3-dspark-b | 54 | 256 | same | 1.91 | 9.94 | 32 | 67.7 | 199.2 | 33.4 | 0 |
| s3-dspark-nc-a | 54 | 256 | DSpark on + --no-expert-cache |
2.25 | 7.56 | 32 | 75.2 | 196.6 | 33.4 | 0 |
| s3-dspark-nc-b | 54 | 256 | same | 2.52 | 7.97 | 32 | 75.4 | 194.9 | 33.4 | 0 |
| s3-dspark-nc-s40-a | 54 | 256 | DSpark on + --no-expert-cache --ssd-cache 40 |
1.68 | 9.84 | 40 | 80.8 | 151.5 | 39.5 | 0 |
| s3-dspark-nc-s40-b | 54 | 256 | same | 1.71 | 2.41 | 40 | 80.3 | 155.3 | 39.6 | 250 (swapping) |
| s4-r9700 | 54 | 256 | DSpark on, --ssd-cache 32 --gpus 0 |
4.33 | 3.76 | 32 | 65.3 | 316.3 | 31.9 | 2 |
| s4-split35-5 | 54 | 256 | DSpark on, --ssd-cache 32, DS4_SPLIT=35,5 |
5.38 | 6.16 | 32 | 68.0 | 226.6 | 33.7 | 0 |
| s4-split34-6 | 54 | 256 | DSpark on, --ssd-cache 32, DS4_SPLIT=34,6 |
4.78 | 4.69 | 32 | 68.4 | 213.6 | 33.7 | 0 |
| s3-long-dual-c | 12593 | 64 | DSpark on, --ssd-cache 32 (engine's own split) |
31.49 | 7.07 | 32 | 61.2 | 300.9 | 33.3 | 0 |
| s3-long-dual | 12593 | 64 | same + --no-expert-cache |
37.99 | 3.88 | 32 | 63.3 | 285.9 | 32.5 | 519 |
| s3-long-r9700 | 12593 | 64 | DSpark on, --ssd-cache 32 --no-expert-cache --gpus 0 |
209.46 | 5.02 | 32 | 61.2 | 442.6 | 31.8 | 58 |
| s4-long-r9700 | 12593 | 64 | DSpark on, --ssd-cache 32 --gpus 0 |
185.23 | 4.88 | 32 | 60.1 | 491.3 | 31.9 | 0 |
| s4-long-split35-5 | 12593 | 64 | DSpark on, --ssd-cache 32, DS4_SPLIT=35,5 |
166.45 | 6.20 | 32 | 64.5 | 336.9 | 33.4 | 0 |
| s4-long-split34-6 | 12593 | 64 | DSpark on, --ssd-cache 32, DS4_SPLIT=34,6 |
139.51 | 0.71 | 32 | 64.6 | 335.4 | 33.3 | 4557 (swapping) |
On short-prompt decode, run-to-run spread on this box is about ±15%: s2-base 8.85 vs s2-base-rep 7.72, and s3-nc-a 5.65 vs s3-nc-b 7.47.
Through serve.server with the chosen settings (DSpark on, --ssd-cache 32, --cpu-threads 5, DS4_SPLIT=35,5, HIP_VISIBLE_DEVICES=0,1), four requests on one warm engine, all finished cleanly:
| Request | Prompt (new + reused) | Generated | Prompt tok/s | Decode tok/s | DSpark drafts kept |
|---|---|---|---|---|---|
| 1 | 21 + 0 | 42 | 2.4 | 2.50 | 14 / 17 |
| 2 | 20 + 0 | 218 | 5.0 | 4.59 | 166 / 186 |
| 3 | 1 + 19 | 192 | 2.7 | 4.46 | 145 / 169 |
| 4 | 25 + 0 | 600 | 2.8 | 9.89 | 344 / 411 |
Observations
- The XTX's x1 link decides the prompt path. With the engine's own split (R9700 layers 0–20 + the drafter, XTX 21–39 + head), the 12.6K prompt read at 31.5 tok/s. Each card streamed about 78 GB of experts from disk at only 0.20 GB/s (
ds4 prompt streamlines), and the XTX's share went over x1. WithDS4_SPLIT=35,5, the XTX gets layers 35–39 + head, and all 1,920 of their experts stay resident on it. The XTX then streamed 0 GB, the R9700 streamed 161.5 GB at 2.51 GB/s, and the prompt reached 166 tok/s. One more layer (34,6) no longer fits: 2,190 of 2,304 experts resident. In that long-prompt run the host also swapped (4.5 GB out) and decode fell to 0.71 tok/s. As far as I can read it, the cost-based split inds4_model.cuweighs each group's free VRAM but not its link speed, so on this box it picks a split that streams over the x1 card. - The R9700 alone reads long prompts fastest (185–209 tok/s). It reuses experts across the two 8192-token chunks from the layer-major layer cache, which the two-GPU run did not report. Its decode is the slowest (3.8–5.0 tok/s).
- The RAM store's auto size is far too small on this box. It picked 11.3 GB when
MemAvailablewas about 23 GB, after subtracting 12 GB./proc/meminfoshowedGPUReclaim: 32022604 kB(amdgpu's reclaimable pool, not counted in MemAvailable) while the process RSS of the whole system summed to 11.6 GB. Setting--ssd-cache 32cut disk reads from 354 to about 200–225 MB/token and raised decode from 6.3 to 7.7–8.9 tok/s. 40 GB pushed one of three runs into swap. - DSpark helps here despite
autoleaving it unloaded (149 GB of experts vs 77–82 GB of VRAM room): 9.8–9.9 tok/s against 7.7–8.9 tok/s, in three runs each, with 87–95% of drafts kept per position. - Short prompts are disk-bound, about 2 tok/s for 54 tokens: one prompt pulled about 18 GB of experts from the files.
Failure 1. The first run had neither --gpus nor HIP_VISIBLE_DEVICES set. It aborted with exit 134 after a 665 s load, having split the model over three devices, including the Raphael iGPU (gfx1036): rocBLAS error: Cannot read .../rocblas/library/TensileLibrary.dat: Illegal seek for GPU arch : gfx1036. The iGPU has no rocBLAS library in these wheels. Every later run used both --gpus 0,1 and HIP_VISIBLE_DEVICES=0,1, so I can't say whether --gpus alone would have been enough. With no --gpus, the engine used every visible device, including the integrated GPU.
All per-run logs (full stderr, command and environment) and results.jsonl are available if useful.
Reproduction status
Measured locally
Activity
Niko1221 commentedon Oct 29, 2026
Thank you for measuring this and writing it up. We are building a Strata website with a page for community benchmarks, and results like these will be added there when it launches, so we are closing this here. If your numbers change with a new version, you are welcome to post them again.