Repository navigation
Conversation
Port of c81f998 (CUDA) to the SYCL verifier. A pipelined verifier (set_always_publish) captured its one-token window with the in-graph state advance, and pl_commit_async then ran the commit graph on top of it, so every one-token window advanced the first stage's GDN state twice. Greedy text with --pipeline-windows 2 forked from the serial loop's within the first few dozen tokens on 2x Arc Pro B70; with this change both loops give identical text and state fingerprints (STRATA_STATE_HASH), as STRATA_ONE_TOKEN_COMMIT=0 already did. Co-Authored-By: Claude Opus 5.5
deriknel
force-pushed
the
pr/sycl-pipeline-one-token-commit
branch
from
October 14, 2026 19:42
3c63f75 to
d350ba6
Compare
This was referenced Oct 15, 2026
Owner
|
Thanks for the pull request. We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected. To reopen it takes about two minutes:
The rules are in CONTRIBUTING.md. |
Author
|
/reopen |
|
Reopened, thank you. A maintainer will pick it up from here. If it conflicts with main, rebase it first; that speeds up review. |
sheigl
added a commit
to sheigl/Strata
that referenced
this pull request
Oct 23, 2026
…(cherry-pick of upstream PRs Niko1221#2433 + Niko1221#2436) Verifier::done() still carried dpct's placeholder for CUDA's error check and treated a window that is still running as an error: --pipeline-windows 2 on a SYCL layer split failed the first request with 'verify: error' (HTTP 400). With the upstream fix a window still on the GPU is polled again, as the CUDA verifier does. Measured on this branch (B65 + B70, IQ3_S split, --spec 4, streamed experts, 2.84 GiB mirror, 3 sequential 2,280-token requests per arm, 200 tokens out, temperature 0): serial 52.5 / 60.0 / 61.0 tok/s decode against --pipeline-windows 2's 58.6 / 65.4 / 65.7 tok/s (+8-12%), zero 'verify: error' lines, identical prompts and completions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and result
With
--pipeline-windows 2on SYCL, greedy text forks from the serial loop's within the first few hundred characters. A pipelined verifier (set_always_publish) captures its one-token window with the in-graph state advance, andpl_commit_asyncthen runs the commit graph on top of it, so every one-token window advances the first stage's recurrent (GDN) state twice. c81f998 fixed this in the CUDA verifier on 2026-10-09; the SYCL verifier still has the old condition. With this change the pipelined text matches the serial loop's.Related discussion
Needs #2433, without which
--pipeline-windowsdoes not run on SYCL at all. Ports c81f998. No scope agreement needed: a focused fix.Approach
sycl/src/core/verify.cpp, the two conditions c81f998 changed insrc/core/verify.cpp:Verifier::record_window:self_commitis off whenalways_publish_is set.Verifier::commit: the branch that skips the commit graph for a one-token window is not taken whenalways_publish_is set.Only
--pipeline-windowscallsset_always_publish(true), so the serial loop does not change. The branch still merges cleanly intomain.Validation
Setup for every run:
maine88ee39 built for SYCL (AOTbmg-g31, oneAPI 2026.1, Compute Runtime 26.35.39758.10, IGC 2.41.5) in a container; host CachyOS, kernel 7.2.11,xe. 2x Intel Arc Pro B70 32 GB, layer split 26, Swift 1.5 Flash-Next IQ3_S with every expert in VRAM,--max-context 262144, int8 KV,--spec 4 --spec-min-p 0.6,--prefill 4096. Each build has only this PR's commit on top ofmain. Greedy check: 5 fixed prompts (prose, code, maths, an edit, a story), temperature 0, 400 tokens, thinking off, compared character by character. The serial reference ismainwith pipeline windows off.Passed:
main+ [Engine] sycl: a pipelined window still on the GPU is polled again, not failed #2433 + this PR,--pipeline-windows 2: all 5 greedy texts identical to the serial loop's, in two rounds.main(+7% / +19%).verify: error, no lost device, noxereset.cteston both cards: the same 3 failures as onmain;iq_paritywith fixtures, 0 failures.Failed:
main+ [Engine] sycl: a pipelined window still on the GPU is polled again, not failed #2433 only,--pipeline-windows 2(the bug): pipelined text differs from the serial loop's at character 72 (prose), 146 (code) and 50 (story); maths and the edit are the same.Not run:
sycl/only.Limits
None known. On 0.1.43.1 we also confirmed the cause with
STRATA_ONE_TOKEN_COMMIT=0(identical text and state hashes), not repeated on 0.2.0.We investigated (myself and Claude Opus 5.5 in Claude Code) and I reviewed the findings and ran every test on my two Arc Pro B70s on my CachyOS machine.