Skip to content

[Benchmark] RX 6900 XT (gfx1030) on Windows 11: the released rdna2 package on a real card - #4780

Closed
Jupiterxia553 wants to merge 1 commit into
Niko1221:mainfrom
Jupiterxia553:bench/rx6900xt-windows-rdna2-20261031
Closed

Jupiterxia553 wants to merge 1 commit into
Niko1221:mainfrom
Jupiterxia553:bench/rx6900xt-windows-rdna2-20261031

Conversation

@Jupiterxia553

Copy link
Copy Markdown

No report existed for the Windows rdna2 package on any Windows machine. docs/AMD_HIP.md
has the RX 6000 section marked as built but untested on a real card, and #4723 covers only
the Linux rx6000 package. This adds the Windows side: engine 0.1.54, 262,144-token context,
Qwen3.8-Flash-Next IQ3_S with a native pack and an MTP draft layer, eleven environment
switches and the arguments as the server ran them.

Results: 869.0 tok/s prompt (9,699 tokens) and 68.5 tok/s decode (400 tokens) with 78.6 %
draft acceptance, one run each on 2026-10-31. This is not the standard protocol:
strata bench run was not executed and there is no results.json; README.md says so in its
first paragraph and lists what was not measured.

Two conditions a Windows reader should know about, both from the engine's own startup lines:
15,570 of the card's 16,368 MiB reach the process (WDDM), and large pages are refused
(VirtualAlloc error 1314), so the expert arena runs on 4 KB pages.

One open question, recorded rather than diagnosed: --expert-cache auto prints an estimate
that the slot count it finishes with contradicts (2,601 -> 3,369 slots here, 2,669 -> 3,302 on
another startup, with the "shrinking" line not followed by a shrunken figure). I did not change
anything to find out which number is wrong.

Results only. No engine change, no default touched.

…ckage on a real card

No report existed for the Windows `rdna2` package on any Windows machine. `docs/AMD_HIP.md`
has the RX 6000 section marked as built but untested on a real card, and Niko1221#4723 covers only
the Linux `rx6000` package. This adds the Windows side: engine 0.1.54, 262,144-token context,
Qwen3.8-Flash-Next IQ3_S with a native pack and an MTP draft layer, eleven environment
switches and the arguments as the server ran them.

Results: 869.0 tok/s prompt (9,699 tokens) and 68.5 tok/s decode (400 tokens) with 78.6 %
draft acceptance, one run each on 2026-10-31. This is not the standard protocol:
`strata bench run` was not executed and there is no `results.json`; README.md says so in its
first paragraph and lists what was not measured.

Two conditions a Windows reader should know about, both from the engine's own startup lines:
15,570 of the card's 16,368 MiB reach the process (WDDM), and large pages are refused
(VirtualAlloc error 1314), so the expert arena runs on 4 KB pages.

One open question, recorded rather than diagnosed: `--expert-cache auto` prints an estimate
that the slot count it finishes with contradicts (2,601 -> 3,369 slots here, 2,669 -> 3,302 on
another startup, with the "shrinking" line not followed by a shrunken figure). I did not change
anything to find out which number is wrong.

Results only. No engine change, no default touched.
@Jupiterxia553

Copy link
Copy Markdown
Author

跟进:PR 正文里「槽位口径不一致」的问题已定位到源码

先更正我自己在正文里留的那个问号。三行看起来自相矛盾的日志,其实是两个不同的量:

日志行 源码位置 是什么
expert cache auto: ... -> N slots src/program/generate.cpp 约 6902 预估。在分配之前打印,按 max_blob(本机 2.54 MiB/槽)计价,描述「一份理想缓存想要多少槽」
only X MiB free once the slots are written (reserve R MiB); shrinking the expert cache 约 7175 在 xcache.close() 之前读空闲显存,然后才调 shrink_to()。它从不报告收缩后的槽数
expert cache M slots, ... GiB of VRAM 约 7186,来自 xcache.slots() 最终实际槽数,按每层真实大小落盘(本机均值约 1.96 MiB/槽)

所以 M > N 是必然的:预估用 max_blob 定价,实际分配用各层真实大小;本机两者相差约 30%。而「既说 shrinking、槽数又比预估多」也不矛盾——X 是收缩前的读数,M 是收缩后的结果。

一个很小的建议:把 shrink_to() 之后的槽数也打一行出来,或者把 6902 那行标注成 estimate。现在这三行放在一起,任何人读都会以为算错了。

真正卡住的不是 auto_cache_slots,是 Windows 的 2560 MiB 地板

kWddmAutoReserveMib = 2560(#1376、#1709 给 ≥15 GiB 卡的 Windows HIP 兜底值)会在没有显式 --vram-reserve-mib 时覆盖其他考虑。本机 RX 6900 XT / gfx1030 / Windows / 16 GiB,WDDM 预算 15570 of 16368 MiB,显式给 reserve 后实测:

--vram-reserve-mib 预估槽数 最终槽数 prefill tok/s prompt 借槽
默认(2560 地板) 2601 3272–3369 866.6 2315
1900 2861 3603 936.3 —
1500 3019 3783–3824 1214.4 2617
  • 按 2560 → 1900 → 2560 → 1500 → 2560 → 1500 换序 ABBA 跑,区间不重叠(2560 给 859.6–871.5,1500 给 1209–1220),排除顺序偏置。prefill +39.6%,decode 64.2–68.4(+3%)。
  • 提速机制就是借槽:prompt 路径借走的槽从 2315 涨到 2617(4.98 GiB),prefill 期间少取专家。
  • 质量:14 题客观电池 13/14,与 reserve 2560 时逐题相同(唯一挂的仍是 47 人容斥,IQ3 两档历史同挂)。
  • 这一轮日志里零分页、零逐出告警。

结论:--expert-cache auto 在 Windows 上偏保守,默认值按最坏情况的 WDDM 开销定价;显存有余量时显式给 --vram-reserve-mib 收益可观。代价要讲清楚——1500 这一档跑起来只剩 1.1–1.3 GiB 空闲,桌面端显存吃紧(浏览器硬解、另一个 GPU 程序)时有被 WDDM 换页的风险,所以我它是当「愿意盯着看」的档位,不是无脑推荐值。保守取 1900 也能拿到 +8% prefill。

@Niko1221

Niko1221 commented Nov 5, 2026

Copy link
Copy Markdown
Owner

Thank you for the benchmark. We have collected these results for the benchmarks page on the Strata website, so we are closing this pull request instead of adding the files to the repository. If you measure again, for example on 0.2.11, you are welcome to open a new report.

The Strata project

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants