[Plugin EP] Preserve streams in single-tensor data transfers - #32666
Merged
Xiaofei Han (xiaofeihan1) merged 1 commit intoSep 17, 2026
Conversation
Forward the caller's stream through CopyTensorImpl, matching the existing batch path while preserving null streams for synchronous copies. Add focused callback regression coverage for async, sync, and batch copies. Fixes microsoft#32643 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot started reviewing on behalf of
Xiaofei Han (xiaofeihan1)
September 17, 2026 05:20
View session
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The focused fix matches existing batched behavior and is adequately covered by regression tests.
Pull request overview
Forwards execution streams through Plugin EP single-tensor transfers, preserving asynchronous operation ordering.
Changes:
- Passes the supplied stream to the Plugin EP callback.
- Adds regression coverage for asynchronous, synchronous, and batched copies.
- No actionable issues identified.
File summaries
| File | Description |
|---|---|
onnxruntime/core/framework/plugin_data_transfer.cc |
Forwards the stream in single-tensor copies. |
onnxruntime/test/framework/data_transfer_manager_test.cc |
Verifies stream forwarding across copy paths. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Xiaofei Han (xiaofeihan1)
requested review from
Edward Chen (edgchen1),
Scott McKay (skottmckay) and
Tianlei Wu (tianleiwu)
September 17, 2026 05:22
Tianlei Wu (tianleiwu)
approved these changes
Sep 17, 2026
33 of 50 tasks
adrastogi
pushed a commit
that referenced
this pull request
Sep 28, 2026
### Description Forward the caller's stream through the Plugin EP single-tensor `CopyTensorImpl` path. This matches the existing batched-copy behavior while preserving a null stream for synchronous copies. Adds focused callback regression coverage for asynchronous, synchronous, and batched copies. ### Motivation and Context The single-tensor Plugin EP data-transfer path discarded the supplied stream, preventing asynchronous copies from receiving the execution stream. Fixes #32643 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 5899c7c)
Tianlei Wu (tianleiwu)
added a commit
that referenced
this pull request
Sep 29, 2026
This cherry-picks the following commits for the release: * #32610 * #32633 * #32641 * #32611 * #32609 * #32607 * #32679 * #32681 * #32666 CI fixes: * #32491 (installs `uuid-dev` for the Linux minimal-build CoreML job; that check also fails on the `rel-1.28.3` base) * Raise the Android minimal baseline binary-size threshold from 1,440,768 to 1,443,840 bytes (release branch only). The checks added by these fixes grow the minimal build from 1,440,538 to 1,443,098 bytes (+2,560). All picks except #32607 applied cleanly with `git cherry-pick -x`, in the order they merged into main (#32666 was added afterward). #32607 required a manual backport because `bert_defs.cc` has diverged on main since 1.28 branched: - `GroupQueryAttention` / `SparseAttention`: backported as-is (`total_sequence_length_index` of 6 / 7, plus the single-element check). The GQA call site in 1.28 has no `sliding_window_cache`, so it only gets the new argument. - `CausalConvWithState`: the 1.28 schema predates `channels_last`, `state_window`, and `dilation`. Only the `ndim` in [1, 3] check and the `weight`/`input` rank == `ndim + 2` checks are backported. These match what the 1.28 CPU kernel already enforces. `ChannelsLastInputRankBelowThreeIsRejected` is dropped. - `GatedDeltaNet`: this op does not exist on 1.28.3, so its hunk and test are dropped. Validation (Windows x64, CPU EP, RelWithDebInfo): - All 14 regression tests added by these PRs pass. - `onnxruntime_test_all`: 1848 tests (including #32666's `PluginCopiesForwardStreams`), 0 failures. - `onnxruntime_provider_test`: 5370 tests, 0 failures. --------- Co-authored-by: Wei Wang <wei4.wang@intel.com> Co-authored-by: shiyi <shiyi.zou@intel.com> Co-authored-by: Bin Miao <bin.miao@intel.com> Co-authored-by: Bryan B <bryan.bernhart@intel.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: tianleiwu <30328909+tianleiwu@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Xiaofei Han <xiaofeihan@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Forward the caller's stream through the Plugin EP single-tensor
CopyTensorImplpath. This matches the existing batched-copy behavior while preserving a null stream for synchronous copies.Adds focused callback regression coverage for asynchronous, synchronous, and batched copies.
Motivation and Context
The single-tensor Plugin EP data-transfer path discarded the supplied stream, preventing asynchronous copies from receiving the execution stream.
Fixes #32643