[Vop3p]

Problem and result

Pilot data for the draft standard benchmark (strata-qwen38-thinking-v1-draft.1) from an AMD box: strata bench run against a 2x RX 6900 XT layer split, Qwen3.8-Flash-Next IQ3_S, track custom (a source build of v0.2.2 plus five local opt-in patches, this box's production run config). Status complete: 21 of 21 required requests, 0 failed, 0 skipped, sanity 12 of 12, required suite 2 min 48 s after loading; the optional concurrency stage ran (3 rounds of 4, 0 failed). Medians: short decode 66.4 tok/s; prompt-4k 1012.6 tok/s read / 62.9 decode / TTFT 4.08 s; prompt-16k 1510.6 / 62.0 / 10.88 s; 6 of the 9 performance requests reached the 512-token cap while reasoning (as on the two pilot boxes of 2026-10-20-0.2.1-bench). No nonconforming measurement.

Related discussion

docs/BENCHMARK_PROTOCOL.md ("Submit and review": pilot data during the draft phase, labelled with the draft ID); bench/results/2026-10-20-0.2.1-bench (the first two runs). No scope agreement needed for a results-only PR.

Approach

The folder bench/results/2026-10-23-standard-2x-rx-6900xt-dcca09c3/ is the runner's output as written (README.md, results.json, outputs.jsonl, config.json, provenance/; artifact hashes untouched), plus NOTES.md and notes/ from me: how the engine was started (by hand on loopback, --url), what bench_facts could not read on ROCm (GPU names come out as "AMD Radeon; AMD Radeon", card memory, power limit, driver and runtime are null), and a five-shape measurement of the protocol's 4K request on the same engine showing that the decode column (62-66 tok/s) is the speculative decoder's yield under the model card's sampling (1.5-1.9 tokens per window) rather than a slower window (24-27 ms in every shape; greedy decodes 80-90 tok/s on the same engine). The last point may deserve a sentence in the protocol text for readers who compare standard results with benchmark.py or tools/strata_bench.py numbers. No engine or docs change in this PR.

Validation

Passed: strata bench run --url http://127.0.0.1:8089 --out ... --track custom --optional-concurrency from checkout 34cd8534 on 2026-10-23T01:56Z; strata bench report <folder> reads it back as complete; a grep for private paths and keys over the folder is empty.

Failed: none observed.

Not run: --profile mode (no saved profile on this box); the default track (this box runs a patched source build); a second repetition of the whole suite; STRATA_SPEC_COUPLED / STRATA_SPEC_PROB on the sampled requests.

Limits

One box, one quantization, custom track. The hardware fields the runner leaves null on AMD are filled in NOTES.md by hand. The cards run at this box's production 245 W / -81 mV setting (prompt +2-3%, decode unchanged, measured separately).

Developed with an AI coding assistant; every number in the report was measured on the machine described in it.

[Vop3p]

Pushed five more runs of the draft protocol from the same box (commit c856013), all complete, 12 of 12 sanity, optional concurrency included:

run	cards	track	engine and settings	short decode	prompt-4k	prompt-16k
2026-10-23-standard-1x-rx-6900xt-68eeaea6	1	default	release rx6000 package, profile made by strata launch (spec 4/0.5, 64K, kv int8, vision cpu)	43.2	479 / 40.5	507 / 40.7
2026-10-23-standard-2x-rx-6900xt-be7e97da	2	default	same, main + helper, no pipeline windows	61.2	464 / 51.8	645 / 53.3
2026-10-23-standard-1x-rx-6900xt-af8df11c	1	custom	release package + the upstream opt-in switches (PROMPT_F16, SH_STREAM, KERNEL_COPY, rocBLAS table, IO_THREADS, CPU_SHARE auto, HC_PERSIST, PIPED_CONCURRENT, STREAM_MIN), spec 3/0.7, presence_penalty 0	48.5	877 / 43.5	1,107 / 46.0
2026-10-23-standard-2x-rx-6900xt-54277ffd	2	custom	same + --pipeline-windows 2	76.2	849 / 66.4	1,369 / 67.7
2026-10-23-standard-1x-rx-6900xt-b6d9cf1a	1	custom	source build 34cd8534 (v0.2.2 + five local opt-in patches) with the same switches	45.5	996 / 45.0	1,225 / 43.3
2026-10-23-standard-2x-rx-6900xt-dcca09c3 (the first run of this PR)	2	custom	same, this box's production config	66.4	1,013 / 62.9	1,511 / 62.0

MATRIX.md in the first folder puts them side by side. Three things the matrix shows on gfx1030: the release package runs in the default track on one and two cards (the strata launch path ran unattended; one packaging detail: the Terminal points the engine at a rocBLAS table named for rocBLAS 5.4.0 that the package's rocBLAS cannot open, so it logs cannot open rocBLAS tuning file ... default kernels run); the upstream opt-in switches double prompt reading (479 / 507 -> 877 / 1,107 on one card, 464 / 645 -> 849 / 1,369 on the split), the local patches add 10-19% on top; decode under the card's sampling is 40-46 on one card and 52-68 on the split in every variant, inside the run-to-run spread of sampled decoding, so that column does not separate the engine variants here.

Developed with an AI coding assistant; every number above was measured on the machine described in the report.

[Niko1221]

Thank you for measuring this and writing it up. We are building a Strata website with a page for community benchmarks, and results like these will be added there when it launches, so we are closing this pull request here; the numbers are not lost. If your numbers change with a new version, you are welcome to send them again.