Repository navigation
Conversation
…(main e88ee39) - Flash-Next IQ3_S, layer split tools/strata_bench.py --long --clients 2 --runs 3 on a source build with setup.py's HIP options, plus real-text prompt reads (31K / 119K) and single-request decode (4K / 32K) on the same server. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Owner
|
Thank you for measuring this and writing it up. We are building a Strata website with a page for community benchmarks, and results like these will be added there when it launches, so we are closing this pull request here; the numbers are not lost. If your numbers change with a new version, you are welcome to send them again. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and result
A results-only report (no engine change): upstream 0.2.0-beta stock on 2x RX 6900 XT (gfx1030, layer split), Flash-Next IQ3_S, measured with
tools/strata_bench.pyand, on the same server, real-text prompt reads and single-request decode. Folder:bench/results/2026-10-20-community-2x-rx-6900xt-0.2.0/(README in the template's format +results.json).Short chat decode 61.6 tok/s; one request summarizing 4K / 30K tokens of real text: 66.5 / 66.0 tok/s; agent session cache hit 69.5%; two clients 42.8 tok/s together.
Related discussion
None. Earlier reports from this box: #927, #1270, #1657, #1927, #2237 (older versions).
Approach
Main e88ee39 (v0.2.0-beta + 12 docs/packaging commits), source build with setup.py's HIP options (
STRATA_PREFILL_MMQ,STRATA_Q6K_EXPERTS,STRATA_MMQ_KQUANTSon, gfx1030, the pinned ggml), no local patches; the configuration and the non-default environment are listed in the README. A cold server, then the suite, then the real-text runs.Validation
Passed:
python tools/strata_bench.py --url http://127.0.0.1:8089 --clients 2 --long --runs 3(224 s, 0 errors); real-text reads 3x each with 0 tokens reused; the same reads on 20 more cold starts of builds whose default path is byte-identical to this one (two PR branches with their variables unset) agree within 0.7%. The README andresults.jsonwere checked for home paths, host names and addresses.Failed: none observed.
Not run: correctness checks (speed only); a single-card or helper configuration; today's main 9ae93f7 (the 0.2.1 candidate).
Limits
One machine, one model. The real-text corpus is private (mixed English / Chinese / code notes) and not attached; the README describes how the prompts were built. With batch slots on a layer split, 0.2.0 turns
--pipeline-windowsand--batch-mtpoff, so the single-request decode here is lower than this box reaches with them (PR #3095).Developed with an AI coding assistant; every number above was measured on the machine described here.
🤖 Generated with Claude Code