[Melike Kaptan (melkap01-Arm)]

Description

Integrates the Arm KleidiAI SVE2.1 FP16 HalfGemm kernel into MLAS.

Key changes:

• Updates KleidiAI from v1.20.0 to v1.30.0.
• Adds SVE2.1 runtime detection on Linux and Windows.
• Selects SME/SME2 first when available, otherwise SVE2.1.
• Adds SVE2.1 runtime and backend-native RHS packing.
• Records the packing thread’s SVE vector length in the native-packed buffer.
• Prevalidates every native-packed batch entry against the caller’s current vector length before
  writing any output.
• Rejects mixed-VL batches without partially modifying their outputs.
• Validates worker-thread vector lengths before executing SVE2.1 tiles. Work is replayed safely on
  the validated caller thread when worker VLs differ.
• Rejects unsupported native-packed combinations such as a separate bias or nonzero ldb.
• Adds overflow protection and rejects K values exceeding the kernel’s supported range.
• Adds Arm copyright and SPDX licence declarations to the modified implementation files.

Motivation and Context

SVE vector length is configured per thread, and the SVE2.1 packed RHS layout depends on that vector
length. A buffer packed by one thread cannot safely be consumed by a thread using a different VL.

This change enables KleidiAI HalfGemm acceleration on SVE2.1 systems without SME while preserving
SME/SME2 priority and preventing mismatched packed layouts from producing incorrect or partially
modified output.

Testing

Added coverage for:

• Exact and tail tile shapes.
• Runtime-packed and backend-native-packed RHS.
• Padded leading dimensions and guard regions.
• Single-threaded and threaded execution.
• Same-VL multi-entry batches.
• Batch entries packed with different vector lengths.
• Worker threads using a different VL from the packing/caller thread.
• Unsupported bias, transpose, leading-dimension, and K configurations.
• Packed-buffer size and overflow handling.

A local Release KleidiAI build and CTest suite completed successfully. SVE2.1-specific tests are
hardware-gated and skip on systems without the required SVE2.1 capabilities.

[Copilot]

🟡 Changes recommended

Native-packed batched execution can reject a later vector-length mismatch after earlier outputs have already been modified.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds KleidiAI SVE2.1 acceleration for MLAS FP16 HalfGemm while retaining SME/SME2 priority.

Changes:

• Adds Linux/Windows SVE2.1 detection and dispatch.
• Integrates SVE2.1 packing, execution, and vector-length handling.
• Updates KleidiAI and adds targeted tests.

File summaries

File	Description
cmake/deps.txt	Updates KleidiAI to v1.30.0.
onnxruntime/core/common/cpuid_info.cc	Detects SVE2.1 capabilities.
onnxruntime/core/common/cpuid_info.h	Exposes SVE2.1 capability state.
onnxruntime/core/mlas/lib/platform.cpp	Enables HalfGemm overrides on SVE2.1.
onnxruntime/core/mlas/lib/kleidiai/halfgemm_kleidiai.cpp	Implements SVE2.1 packing and execution.
onnxruntime/core/mlas/lib/kleidiai/mlasi_kleidiai.h	Declares test hooks.
onnxruntime/test/mlas/unittest/test_halfgemm.cpp	Adds SVE2.1 kernel tests.
onnxruntime/test/mlas/unittest/test_halfgemm.h	Adds stride, packing, and canary helpers.

Review details

• Files reviewed: 8/8 changed files
• Comments generated: 1
• Review effort level: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Scott McKay (skottmckay)]

Melike Kaptan (@melkap01-Arm) We're seeing some build errors from the packaging pipelines on arm64.

N:_work\1\s\onnxruntime\core\mlas\lib\kleidiai\halfgemm_kleidiai.cpp(361): error C4789: in function 'virtual void __cdecl HalfGemmKleidiAISVE2p1_RejectsKAboveKernelLimit_Test::TestBody(void) __ptr64' buffer 'packed_b' of size 1 bytes will be overrun; 64 bytes will be written starting at offset 0 [N:_work\1\s\build\Release\onnxruntime_mlas_test.vcxproj]

Build command:

C:\ToolCache\Python\3.11.9\arm64\python.exe N:_work\1\s\tools\ci_build\build.py --config Release --enable_lto --build_dir N:_work\1\s\build --skip_submodule_sync --cmake_generator "Visual Studio 17 2022" --enable_pybind --enable_onnx_tests --use_vcpkg --use_vcpkg_ms_internal_asset_cache --build --use_azure --parallel --use_binskim_compliant_compile_flags --update

I assume most of those arguments aren't relevant and simply building with VS 2022 might be enough to repro.

Would you be able to take a look?

[Melike Kaptan (melkap01-Arm)]

Melike Kaptan (Melike Kaptan (@melkap01-Arm)) We're seeing some build errors from the packaging pipelines on arm64.

N:_work\1\s\onnxruntime\core\mlas\lib\kleidiai\halfgemm_kleidiai.cpp(361): error C4789: in function 'virtual void __cdecl HalfGemmKleidiAISVE2p1_RejectsKAboveKernelLimit_Test::TestBody(void) __ptr64' buffer 'packed_b' of size 1 bytes will be overrun; 64 bytes will be written starting at offset 0 [N:_work\1\s\build\Release\onnxruntime_mlas_test.vcxproj]

Build command:

C:\ToolCache\Python\3.11.9\arm64\python.exe N:_work\1\s\tools\ci_build\build.py --config Release --enable_lto --build_dir N:_work\1\s\build --skip_submodule_sync --cmake_generator "Visual Studio 17 2022" --enable_pybind --enable_onnx_tests --use_vcpkg --use_vcpkg_ms_internal_asset_cache --build --use_azure --parallel --use_binskim_compliant_compile_flags --update

I assume most of those arguments aren't relevant and simply building with VS 2022 might be enough to repro.

Would you be able to take a look?

Hi. This is a Windows on ARM issue. I was thinking this issue was not visible in your current public pipeline, so the builds there all passed..
The issue is clear, and the solution is also clear. But I am trying to obtain an instance or a pipeline to check if there is no other similar issues anywhere in the branch.
So, I would like to ask you about the urgency/priority of the issue at the moment?

[Scott McKay (skottmckay)]

Blocking our packaging pipelines so reasonably urgent but not blocking a current release.

Can you work with an ORT developer to iterate on the solution? Edward Chen (@edgchen1) would you be able to help out with local testing by manually running the relevant part of the packaging pipeline against a branch with a potential fix?

We're working on adding an arm64 CI that runs pre-checkin but need some capacity to be allocated to enable that: #32791

[Melike Kaptan (melkap01-Arm)]

Blocking our packaging pipelines so reasonably urgent but not blocking a current release.

Can you work with an ORT developer to iterate on the solution? Edward Chen (Edward Chen (@edgchen1)) would you be able to help out with local testing by manually running the relevant part of the packaging pipeline against a branch with a potential fix?

We're working on adding an arm64 CI that runs pre-checkin but need some capacity to be allocated to enable that: #32791

That is very helpful. Thanks
MR : #32798 Fix Branch name : melkap01-Arm:Sve2p1_test_fix