Add more kernels for Qwen-3.5 ops

[Copilot]

Description

Adds Qwen-3.5-related contrib kernel coverage across CPU, WebGPU, and CUDA. The PR introduces CPU and WebGPU implementations for GatedAdd, LinearAttentionGate, and GatedRMSNorm, and enables CUDA BFloat16 registrations for CausalConvWithState, LinearAttention, and LpNormalization.

Changes

• CPU: Added GatedAdd, LinearAttentionGate, and GatedRMSNorm contrib kernels and registered them in the CPU contrib kernel table.
• WebGPU: Added GatedAdd, LinearAttentionGate, and GatedRMSNorm contrib kernels, registered them in the WebGPU contrib kernel table, and fixed LinearAttentionGate shader binding order.
• CUDA: Added BFloat16 kernel registrations for CausalConvWithState and LinearAttention, plus LpNormalization BF16 support starting at opset 22.
• Tests: Expanded contrib op coverage for GatedAdd, LinearAttentionGate, GatedRMSNorm, CausalConvWithState, and LinearAttention, and added a CUDA capability-gated BF16 LpNormalization test.

Motivation and Context

These changes add kernel and type coverage needed by Qwen-3.5-style model patterns across supported execution providers, including BF16 CUDA paths for the affected ops.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[github-actions]

You can commit the suggested changes from lintrunner.

[Copilot]

Pull request overview

Adds CPU and WebGPU execution support for Qwen-3.5 gated attention operators.

Changes:

• Implements GatedAdd, GatedRMSNorm, and LinearAttentionGate kernels.
• Registers the new CPU and WebGPU kernels.
• Extends shared tests across CPU, CUDA, and WebGPU.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
onnxruntime/test/contrib_ops/linear_attention_gates_op_test.cc	Runs gate and RMSNorm tests across supported EPs.
onnxruntime/test/contrib_ops/gated_add_op_test.cc	Extends GatedAdd tests to CPU and WebGPU.
onnxruntime/contrib_ops/webgpu/webgpu_contrib_kernels.cc	Registers WebGPU kernels.
onnxruntime/contrib_ops/webgpu/bert/linear_attention_gates.h	Declares WebGPU gate programs and kernels.
onnxruntime/contrib_ops/webgpu/bert/linear_attention_gates.cc	Implements WebGPU gate and RMSNorm shaders.
onnxruntime/contrib_ops/webgpu/bert/gated_add.h	Declares the WebGPU GatedAdd kernel.
onnxruntime/contrib_ops/webgpu/bert/gated_add.cc	Implements the WebGPU GatedAdd shader.
onnxruntime/contrib_ops/cpu/cpu_contrib_kernels.cc	Registers CPU kernels.
onnxruntime/contrib_ops/cpu/bert/linear_attention_gates.h	Declares CPU gate kernels.
onnxruntime/contrib_ops/cpu/bert/linear_attention_gates.cc	Implements CPU gate and RMSNorm kernels.
onnxruntime/contrib_ops/cpu/bert/gated_add.h	Declares the CPU GatedAdd kernel.
onnxruntime/contrib_ops/cpu/bert/gated_add.cc	Implements the CPU GatedAdd kernel.

Suppressed comments (1)

onnxruntime/contrib_ops/cpu/bert/linear_attention_gates.cc:157

• Narrowing num_rows to int32_t is unsafe because TryBatchParallelFor supports std::ptrdiff_t. With more than INT32_MAX normalization groups, the cast can produce a non-positive total, so no work runs and the uninitialized output is returned as successful. Preserve the wider count.

      context->GetOperatorThreadPool(), static_cast<int32_t>(num_rows),

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.