Repository navigation
[Benchmark] complete standard-draft.1 run: Intel Arc Pro B70 / Qwen3.8-Flash-Next GSQ-RCO IQ3_S (engine 0.1.48, SYCL, CPU experts) - #3840
Closed
Maddy55555 wants to merge 1 commit into
Conversation
…8-Flash-Next GSQ-RCO IQ3_S (engine 0.1.48, SYCL, CPU experts)
Owner
|
Thank you for measuring this and writing it up. We are building a Strata website with a page for community benchmarks, and results like these will be added there when it launches, so we are closing this pull request here; the numbers are not lost. If your numbers change with a new version, you are welcome to send them again. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Complete standard-draft.1 run on an Intel Arc Pro B70 (SYCL/XPU), same protocol and model as PR #3799, with two setup changes: engine 0.1.48 (v0.2.3 + max_tokens hotfix) and the CPU-expert path enabled (
STRATA_VERIFY_NO_HOST=0+STRATA_VERIFY_DEVICE_PLAN=0).Protocol conformance
strata-qwen38-thinking-v1-draft.1(draft)20261023T123010Z-a48c5a63, status complete, 0 failed requests, sanity 12/12Machine
12.8 GB/s host->device -> pcie_frac 0.35, autotuned)Model provenance
ed59f92082b1e93c0e96d60a8b11aab089b52f09, source/tokenizer revisionde4b8e4d43b917e7706784d8bb445c9af86a3540(HF LFS OIDs match the SHA-256s below)4c1eb2ceb4915e11…, shard 2316b46f3a2dbd68c…, expert packexpert-profile-sycl.binddddf9b361dd1cb1…(identical to the run in PR [Benchmark] complete standard-draft.1 run: Intel Arc Pro B70 / Qwen3.8-Flash-Next GSQ-RCO IQ3_S #3799 — same model basis, clean version comparison)Headline numbers (median, min–max in README)
vs. the GPU-only-experts run of PR #3799 (engine 0.1.47): decode +26 % at 4k, +46 % at 16k; prompt rate −5…9 %. The host-arena path passed our 16k correctness gate twice before this run; on engine 0.1.44/0.1.45 the same switches produced token-0 output at ≥16k, so we re-gate after every engine update (details in the submitter addendum of the README).
Artifacts
README.mdsha2569000bc2b0ddcf486…,results.json178cdca55080c766…,outputs.jsonlbfc7abf4c29f2188…,engine-log-excerpt.txt8a45489f11ce4e60…(full engine log for the run window, paths sanitized)missingand covered in the submitter addendum.Context
Follow-up to #3799 on the same machine/model; submitted per docs/COMMUNITY_BENCHMARKS.md submit-and-review flow. No community-table edit — reviewers decide.