Update CK and fix performance issue on dev machine - #13531
Merged
Merged
Conversation
…red. The flags must be added to all HIP sources, we ensure it by checking hip-lang::device is properly configured.
cloudhan
changed the title
cloudhan
requested review from
PeixuanZuo,
Changming Sun (snnn),
Weixing Zhang (weixingzhang) and
Yao Zhang (zhangyaobit)
November 1, 2022 09:20
| onnxruntime_add_shared_library_module(onnxruntime_providers_rocm ${onnxruntime_providers_rocm_src}) | ||
|
|
||
| if(NOT MSVC) | ||
| target_compile_options(onnxruntime_providers_rocm PRIVATE -Wno-sign-compare -D__HIP_PLATFORM_HCC__=1) |
Contributor
There was a problem hiding this comment.
Can you elaborate a bit why this flag and the above ones are not needed anymore?
Contributor
Author
There was a problem hiding this comment.
They are not used anymore. Just clean up the leftover after #13406
Yao Zhang (zhangyaobit)
previously approved these changes
Nov 2, 2022
…vel. Note this have no implication on performance.
Yao Zhang (zhangyaobit)
approved these changes
Nov 3, 2022
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reland #13493 with problem solved, also updated.
-mllvm -amdgpu-early-inline-all=trueis critical to CK's performance, ensure it is properly configured.hip-lang::device'sINTERFACE_COMPILE_OPTIONS, we must not manually add the flags.TL,DR:
hip-lang::devicesometime will be not be properly configured if ourCMAKE_PREFIX_PATHis not configured carefully. In the CI docker, the configuration is in good state, but on dev machine it is not, which then silently result poor performance for kernels. We fixed it in this PR and add a guard to avoid unsuccessful future editing and to prevent convoluted debugging process._CMAKE_HIP_DEVICE_RUNTIME_TARGETis shared in/opt/rocm/lib/cmake/hip-lang/hip-lang-config.cmakeand it is internal to CMake, the variable name will not be changed in the foreseeable future.