[Benchmark] Strix Halo (gfx1151) suite on 0.1.45: --mmap-experts + STRATA_SH_STREAM=0 decode +12-15%, UD-IQ4_XS prefill fixed, shared-GPU-memory placement idea

Measured on 2026-10-19 / 2026-10-20 by mistral13.

Follow-up to #3032 (my first suite report, engine 0.44.1). Same Strix Halo machine, same test suite
(re-run with the engine's own timings), now on Strata 0.1.45 with the bug fix from
#3098 used locally. Coder IQ1_M was replaced by
qwen3.6-35b-a3b-ud-q6_k_xl, and the setup takes the port-handling findings from
#3029 into account (the suite pins one port for config, server and
bench call and kills any stale engine before every step, so no run can accidentally measure a leftover server).

Speed only — no correctness or recall check was run. Main new results:

• --mmap-experts combined with STRATA_SH_STREAM=0 is the fastest decode on this APU for both Flash-Next
  quants: unsloth-ud-iq4_xs chat decode 34.3 → 38.5 tok/s (+12.2 %), and with --pool-workers 6 39.6 tok/s
  (+15.5 %); iq3_xxs 34.7 → 39.4 tok/s (+13.5 %), best agent decode 44.7 tok/s. Note docs/STRIX_HALO.md lists
  STRATA_SH_STREAM=1 as the gfx1151 default (it paid +1.8 % / +6.7 % on 0.1.40) — on 0.1.45 turning it off
  measured clearly faster, so the per-PC calibration or the doc may want a look.
• The big prefill gap of the first suite is gone: unsloth-ud-iq4_xs prompt was 322–355 tok/s on 0.44.1 (much
  worse than iq3_xxs and coder-iq1_m then) and now measures 643–665 tok/s — as fast as the other models. Prompt
  speed is still strongly model-dependent: qwen3.6-35B falls from 596 tok/s (4K) to 373 tok/s (32K) while both
  Flash-Next quants stay flat at 620–665.
• qwen3.6-35b-a3b-ud-q6_k_xl gives the best chat decode (50.5 tok/s, TTFT 0.25 s) of the whole suite, but its
  agent conversation-cache hit is 0 % (30k tokens re-read per session, per-turn TTFT grows to ~27 s) while both
  Flash-Next packs hit ~68.7 % — looks specific to this arch/model path, details below.
• Placement idea measured out (bottom of this report): on this unified-memory APU the mmap variants keep almost
  nothing in "shared GPU memory" (0.9 GiB) while process RAM is nearly full (41.5 GiB) — see
  shared-memory placement idea.

Hardware and software

• GPU and VRAM: AMD Radeon(TM) 8060S Graphics (Strix Halo APU, gfx1151), 64 GiB BIOS carve-out seen as VRAM
  (+ 32 GB shared with system RAM; 108 GB usable by the GPU). No discrete PCIe link — CPU and iGPU share one pool.
• CPU: AMD Ryzen AI MAX+ PRO 395 w/ Radeon 8060S (16 cores / 32 threads, AVX-512)
• Installed RAM: 128 GB unified memory (64 GB seen by the OS + 64 GB VRAM carve-out)
• Storage: not yet verified; OS/SSD details not measured
• OS: Windows 11 (10.0.26200); GPU driver: 32.0.31036.15
• Strata: project 0.1.45 (git checkout 477ac13c), engine 0.1.45 prebuilt Windows HIP binary
  (engine/BUILD.json: prebuilt, windows-x64, archs incl. gfx1151, ROCm 10.2.0, hipBLASLt 100500), with the
  #3098 bug fix used locally (not a source build of the engine)
• Background workloads and power limits: as in community test suite reports, Radeon 8060S (gfx1151) on a Ryzen AI Max+ 395 (Windows 11) #3032, other programs were open during the suite; background load
  was not controlled and no power limit was set.

Model and configuration

• Models (GGUF filenames from this install, quantization in brackets):
  • qwen3.8-flash-next-iq3_xxs [IQ3_XXS + PLE file] — Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf
    (+ -00002 as --ple-gguf)
  • qwen3.8-flash-next-unsloth-ud-iq4_xs [UD-IQ4_XS] — Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf … -00003
  • qwen3.6-35b-a3b-ud-q6_k_xl [UD-Q6_K_XL, arch qwen35moe] — Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf (native path,
    no pack/expert-profile; the file fits resident, no GPU expert cache)
• Context: 262144 tokens; KV type: int8; KV streaming on (setup default); expert cache: auto; vision: off.
• MTP: rt draft layer (--spec 4) for the Flash-Next packs; qwen3.6 native default --mtp 2 (one variant --mtp 4).
  Thinking off, temperature 0 (suite standard). Calibration re-measured per model for this build
  (START-HERE.bat --calibrate); iq3_xxs settled at --pcie-frac 1.00 --spec-min-p 0.70 (43.5 tok/s at calibration).
• Environment: STRATA_HIPBLASLT_TUNING=gfx1151-hipblaslt-100500.txt; variants additionally set
  STRATA_SH_STREAM=0, STRATA_VERIFY_PROFILE=1.

Representative launch commands (full per-variant command in each variant's config-used.json and README.md):

# iq3_xxs variant "--mmap-experts STRATA_SH_STREAM=0"
strata.exe --pack iq3_xxs --native Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf \
  --ple-gguf Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00002-of-00002.gguf \
  --expert-profile expert-profile.bin --expert-cache auto --prefill auto \
  --spec 4 --mtp rt --max-context 262144 --kv int8 --pcie-frac 1.00 --spec-min-p 0.70 --mmap-experts
  env: STRATA_HIPBLASLT_TUNING=gfx1151-hipblaslt-100500.txt, STRATA_SH_STREAM=0

# unsloth-ud-iq4_xs default (START-HERE config; other variants add --pool-workers / --mmap-experts / --expert-cache)
strata.exe --pack unsloth-ud-iq4_xs --native Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf \
  --expert-profile expert-profile.bin --expert-cache auto --prefill auto \
  --spec 4 --mtp rt --max-context 262144 --kv int8 --kv-resident 32768 --resident-budget-gib 36 --spec-min-p 0.5

# qwen3.6-35b-a3b-ud-q6_k_xl default (native path)
strata.exe --native Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf --max-context 262144 --prefill auto --mtp 2

Method

Same self-written updated suite as #3032 (suite/strata_suite.py + suite.toml in the Strata folder): per model it runs
START-HERE.bat setup and --calibrate, edits the generated config per variant, then BENCH.bat --long
(tools/strata_bench.py --long): short chat, fresh 4K/16K/32K prompts (unique tag, no cache help, 24 new tokens),
5-turn agent session with the conversation cache. 3 runs per test, medians with ranges; engine timings for
prompt/decode, client-measured TTFT. 90 s cooldown between runs; stale engines killed first; all bench calls pass
--config and the same port (finding from #3029), so every variant
measures its own freshly started server. Suite window: 2026-10-19 22:09 → 2026-10-20 01:12.

Memory is now sampled every 5 s during each run (memory_samples.csv next to each variant's data):
RAM strata = strata.exe private working set, RAM system = total in-use system RAM, VRAM = dedicated GPU
memory, shared GPU = shared GPU memory (system RAM mapped for the GPU); the peaks include model load.

Results

Ranked by chat decode. pp = prompt tok/s. Full ranking views ranking.md / ranking.csv, per-run folders
(the generated README.md + results.json + engine-log-excerpt.txt) and comparisons/ keep the ranges.

One configuration did not complete: the first calibration attempt for qwen36-ud-q6_k_xl reported
"calibration: no result" (stored, no values); the three qwen3.6 variants themselves ran OK, without calibration
values applied.

What was better than what (vs. each model's default / equivalent)

• STRATA_VERIFY_PROFILE=1 / --pool-workers 6 without the mmap changes is no longer a win on 0.1.45
  (iq3_xxs pool-workers variant is 2.8 tok/s below --mmap-experts STRATA_SH_STREAM=0).
• Docs impact: STRIX_HALO.md marks STRATA_SH_STREAM=1 as the gfx1151 default because it paid on 0.1.40;
  on this 0.1.45 build the opposite measured (both models). A calibration fork "SH_STREAM off vs on" (as
  tools/calibrate.py already does for other engines) would decide per PC.

Prompt speed is strongly model-dependent

• unsloth-ud-iq4_xs: 657 / 663 / 645 at 4K/16K/32K — flat, and ~2× its own 0.44.1 value from community test suite reports, Radeon 8060S (gfx1151) on a Ryzen AI Max+ 395 (Windows 11) #3032
  (where it was the slow prefiler: 322–355 tok/s vs 543–615 for iq3_xxs then).
• iq3_xxs: 604–637 depending on variant — also flat.
• qwen3.6-35b-a3b-ud-q6_k_xl: 579–596 at 4K already the lowest, and it degrades with length:
  483 at 16K, 373 at 32K (TTFT 85 s for one 32K prompt). Its chat decode is the suite leader (50.5 tok/s),
  so this is where this model wants attention.
• No Task Manager re-read this time; the 5 s sampler (memory_samples.csv) replaces it — prefill pushed VRAM
  to ~63 / 50 / 49 GiB (unsloth / iq3_xxs / qwen3.6) with no visible disk activity for the mmap variants.

qwen3.6: conversation cache not hitting

The same 5-turn agent session that the Flash-Next packs answer with a 68.6–68.8 % cache hit (4–73 tokens re-read)
runs on qwen3.6-35b-a3b-ud-q6_k_xl with 0 % reused, ~30,119 tokens re-read per session; per-turn TTFT grows
1.2 → 5.7 → 8.4 → 14.7 → 20.2 → 26.8 s and agent decode sags to 30 tok/s at the last turn. Not measured whether
this is the qwen35moe native path, the missing pack, or KV handling — reporting the observation only.

Possible improvement: shared GPU memory instead of normal RAM

The 0.45 sampler makes the placement visible. Two opposite styles on one memory pool:

On the mmap variant the expert tier lives in ordinary process RAM (page-cache / private allocations) and nearly
compresses the 64 GB OS view (system RAM 57.5 GiB in use), while Windows' "shared GPU memory" — exactly the pool
the GPU can address on this APU — sits almost empty at 0.9 GiB. On the unsloth variant most resident expert data
does go through GPU-visible allocations and process RAM stays at 8.3 GiB.

Suggestion for this platform: when CPU and GPU both touch data (expert pages the CPU assembles/verifies, KV that
could stream, staging buffers), prefer allocating it as shared GPU memory (one cudaMalloc-managed/mapped pool)
instead of process RAM the GPU reaches over a driver mapping — on a unified-memory APU it is the same DRAM, the GPU
path is native, it stops double-counting against the OS memory limit, and it would let the mmap variants use the
shared-GPU pool that is currently idle at ~1 GiB. Worth checking whether the gfx1151 memory tier rules could size
the shared-GPU-backed pool before the plain file-cache tier.

Correctness and limitations

• Speed only; no correctness, recall or needle check; no variant marked as changing numerics was run
  (changes_bits=false throughout). STRATA_SH_STREAM output is stated identical either way in the docs — not
  re-checked here.
• Other programs were open during the suite (as in community test suite reports, Radeon 8060S (gfx1151) on a Ryzen AI Max+ 395 (Windows 11) #3032), no power-limit capture, GPU/HP similar but not idle;
  absolute values may sit a little low, cross-suite notes (0.44.1 vs 0.1.45, driver may also differ) are
  observations, not controlled A/Bs.
• 3 runs per test; chat decode ranges are wide (e.g. 38.1–57.1) because short chats bounce — medians in the table.
• qwen3.6 variants ran without calibration values (calibration produced no result); its --spec-min-p and
  --pcie-frac are engine defaults.
• Memory peaks come from the 5 s sampler and include model load; "expert cache GiB" is the engine's own number;
  - = not applicable (native path has no GPU expert cache).
• The agent conversation feeds the model's own answers back, so token counts drift slightly between runs.

All raw data (per-request results.json, memory_samples.csv, engine log excerpts, comparison diffs,
suite.toml, calibration files) sits in this folder and can be attached. Happy to add more: needle/recall checks,
a controlled same-day A/B SH_STREAM 0 vs 1, a qwen3.6 conversation-cache repro, --parallel/concurrency runs, or
a run with everything closed for a clean baseline — just say what would help.

All results, dmp-files and suite:

benchsuit_0.1.45_gfx1151_windows.zip