Repository navigation
Conversation
… at 4K / 32K / 128K on 0.1.41 Stock v0.1.41 (fb58e0d, source build), ISTA-DASLab IQ3_S at a 131,072-token context, three runs each at 4,096 / 32,768 / 128,000 prompt tokens plus tools/needle_bench.py at 32k and 128k (6 of 6). A private build's runs and two larger stand-in files are listed separately, labelled as not v0.1.41. Results only: one new folder, no code or docs touched.
The same prompts, method and flags without the 4060 Ti (CUDA_VISIBLE_DEVICES = the 5080 only, no --expert-cache-device1), measured 2026-10-10: stock v0.1.41 and the private build, three runs each at 4,096 / 32,768 / 128,000 prompt tokens plus tools/needle_bench.py (6 of 6 on both). Prompt reads within 1.3 % of the two-card runs; decode 13-21 % lower on stock v0.1.41 and 7-17 % lower on the private build. New results-5080-* and needles-5080-* files and two more entries in summary.json; "the 5080 without the helper card" leaves the not-tested list. Results only: no code or docs touched. Measured and drafted with Claude Code.
Contributor
Author
|
Added the RTX 5080-alone results (stock v0.1.41 and the private build, measured 2026-10-10) as a new README section, "The RTX 5080 alone". Measured and drafted with Claude Code. |
Niko1221
pushed a commit
that referenced
this pull request
Oct 10, 2026
…che), IQ3_S at 4K / 32K / 128K on 0.1.41 (PR #1681)
Owner
|
Thank you, in v0.1.42 with your commit, with a row in bench/results/COMMUNITY.md (stock v0.1.41 numbers; the private-build set stays listed separately in your README). |
giridhart
pushed a commit
to giridhart/Strata
that referenced
this pull request
Oct 15, 2026
…che), IQ3_S at 4K / 32K / 128K on 0.1.41 (PR Niko1221#1681)
giridhart
pushed a commit
to giridhart/Strata
that referenced
this pull request
Oct 15, 2026
…21#1638 Niko1221#1647 Niko1221#1655 Niko1221#1681 Niko1221#1692 Niko1221#1695 Niko1221#1716 Niko1221#1717 Niko1221#1657 Niko1221#1640; the 6900 XT 0.1.40.1 row now has the numbers Niko1221#1657 added
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title
Issue: none (a results-only report)
Summary
A results-only report per docs/COMMUNITY_BENCHMARKS.md, measured 2026-10-09. Stock v0.1.41 (
fb58e0db, source build), ISTA-DASLab IQ3_S at a 131,072-token context, the RTX 5080 as the main GPU and the RTX 4060 Ti as a helper expert cache (--expert-cache-device1 3300). Ryzen 9 9950X3D, 96 GB DDR5, PCIe 5.0 NVMe, Linux.Three runs per size, each a fresh prompt (0 reused tokens), temperature 0, 256-token cap;
tools/needle_bench.py6 of 6 at 32k and 128k.The README also lists, under their own headings: the same runs with the RTX 5080 alone (stock v0.1.41 and the private build, measured 2026-10-10), and, labelled as not v0.1.41 as released, the same runs on a private build (v0.1.41 + local patches) and two larger files measured on 2026-10-07/08 on an older private build as stand-ins for a bigger model.
What changed
One new folder,
bench/results/2026-10-09-community-rtx-5080-4060ti-0.1.41/:README.md: hardware, software, model, settings, method, results, limitationsmake_prompts.py: rebuilds the prompts from the repository's own text at v0.1.41 (sha256 of its output in the script)benchmark.py: the client (streamed requests, the server'stimings, memory after each request)results-stock.json,results-private.json,needles-stock.json,needles-private.json,summary.json: per-run dataresults-5080-stock.json,results-5080-private.json,needles-5080-stock.json,needles-5080-private.json: the same with the RTX 5080 aloneconfig-stock.json,engine-start-stock.txt: the server config and the engine's start lines, without paths or GPU idsNo code, docs or index files are touched.
Extra Notes
Not tested: vision, other quantizations, Windows, setup's own configuration for this PC, contexts above 131,072,
--batch, and a layer split.Measured and drafted with Claude Code.