Skip to content

webgpu: use naive reduction - #28174

Merged
Guenther Schmuelling (guschmue) merged 5 commits into
microsoft:mainfrom
xhcao:use-naive-reduction
Jun 16, 2026
Merged

Guenther Schmuelling (guschmue) merged 5 commits into
microsoft:mainfrom
xhcao:use-naive-reduction

Conversation

@xhcao

@xhcao xhcao commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

Use naive reduction when the output size of ReduceMean is far greater than reduce size. Shared reduction method may need to transpose input, which costs much time.

Description

Motivation and Context

Use naive reduction when the output size of ReduceMean is far greater
than reduce size. Shared reduction method may need to transpose input,
which costs much time.
@xhcao

xhcao commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

When I run florence-2-base-vision-encoder-fp16-0.5-fp32 with input [batch_size:1, height:768, width:768], 10 ReduceMean nodes using shared reduction method cost much time,
ReduceMean|Transpose 6.51 "inputs[0] = {1,128,36864} outputs[0] = {1,36864,128} "
ReduceMean 0.30 "inputs[0] = {1,36864,128} outputs[0] = {1,1,36864} "

It is heavy to transpose the input. If we use the naive method,
ReduceMean 0.11 "inputs[0] = {1,128,36864} outputs[0] = {1,1,36864} "

The model's total time is from ~340ms to ~270ms.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the WebGPU reduction kernel dispatch heuristic to prefer the naive reduction implementation for certain ReduceMean shapes, aiming to avoid transpose overhead that can dominate runtime when the reduced dimension is small but the output is very large.

Changes:

  • Adjusts the use_naive_reduction selection logic in ReduceKernel::ComputeInternal.
  • Adds a ReduceMean-specific threshold to route some cases to ReduceNaiveProgram.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread onnxruntime/core/providers/webgpu/reduction/reduction_ops.cc Outdated
@xhcao

xhcao commented Apr 24, 2026

Copy link
Copy Markdown
Contributor Author

Jie Chen (@jchen10) PTAL

@xhcao

xhcao commented Apr 27, 2026

Copy link
Copy Markdown
Contributor Author

Jiajia Qin (@qjia7) PTAL, thanks

Comment thread onnxruntime/core/providers/webgpu/reduction/reduction_ops.cc Outdated
@xhcao

xhcao commented May 12, 2026

Copy link
Copy Markdown
Contributor Author

@xhcao

xhcao commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

Guenther Schmuelling (@guschmue) It seems that the CI failure is not related with this PR, please re-trigger the bots again, thanks.

@xhcao

xhcao commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Rebase the code.

@xhcao

xhcao commented Jun 5, 2026

Copy link
Copy Markdown
Contributor Author

Guenther Schmuelling (@guschmue) Please take a look it, thanks.

@jchen10

Copy link
Copy Markdown
Contributor

Guenther Schmuelling (@guschmue) Just a gentle ping.

@guschmue
Guenther Schmuelling (guschmue) merged commit a534347 into microsoft:main Jun 16, 2026
84 of 85 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ep:WebGPU ort-web webgpu provider

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants