Skip to content

[HIPIFY] Add LLVM lib dir to hipify-clang RPATH - #2705

Open
Samiii777 wants to merge 4 commits into
ROCm:amd-developfrom
Samiii777:fix-hipify-clang-rpath-llvm-libdir
Open

Samiii777 wants to merge 4 commits into
ROCm:amd-developfrom
Samiii777:fix-hipify-clang-rpath-llvm-libdir

Conversation

@Samiii777

Copy link
Copy Markdown

Fixes #2704.

Problem

hipify-clang from the ROCm 7.14 packages cannot start, because its RUNPATH does not include the directory that actually ships libLLVM:

$ hipify-clang --version
hipify-clang: error while loading shared libraries: libLLVM.so.23.0git: cannot open shared object file: No such file or directory

$ readelf -d /opt/rocm/core-7.14/bin/hipify-clang | grep RUNPATH
 0x000000000000001d (RUNPATH)  Library runpath: [$ORIGIN/../lib]

$ find /opt/rocm -name 'libLLVM.so.23.0git'
/opt/rocm/core-7.14/lib/llvm/lib/libLLVM.so.23.0git

$ORIGIN/../lib resolves to <prefix>/lib, but in the versioned ROCm layout LLVM is installed one level deeper under <prefix>/lib/llvm/lib. /opt/rocm/lib is also not registered with the dynamic loader (ROCm/TheRock#6716), so there is no fallback and the failure is unconditional on a stock install.

Change

Append $ORIGIN/../lib/llvm/lib to the RPATH. The original entry is kept first, so a flat layout that installs libLLVM directly into <prefix>/lib continues to resolve exactly as before.

Test

Building HIPIFY needs a full LLVM/Clang dev tree, so I verified the RPATH mechanism directly by reproducing the install layout (bin/, lib/, lib/llvm/lib/) and linking a binary against a stand-in library placed where libLLVM actually ships:

case RPATH result
current upstream $ORIGIN/../lib fails — reproduces the reported error
this PR $ORIGIN/../lib:$ORIGIN/../lib/llvm/lib resolves with no LD_LIBRARY_PATH
flat layout regression guard $ORIGIN/../lib:$ORIGIN/../lib/llvm/lib, lib in ../lib still resolves

3 passed, 0 failed. The first row is a control: it must keep failing, otherwise the test would be vacuous.

Independently, adding only that directory to the search path is sufficient to make the shipped binary work:

$ LD_LIBRARY_PATH=/opt/rocm/core-7.14/lib/llvm/lib hipify-clang --version
AMD LLVM version 23.0.0git
  Optimized build.

Environment

Ubuntu 26.04, ROCm 7.14.0~pre3 (amdrocm-hipify7.14) from repo.radeon.com/rocmradeon/apt/26.13, gfx1100.

Note

Opened as a draft. hipify-perl is unaffected (Perl script, no shared-library dependency). Related earlier fixes in this area: #678 and #1959.

hipify-clang's RPATH is $ORIGIN/../lib, which resolves to <prefix>/lib. In
the versioned ROCm packaging layout libLLVM is installed to
<prefix>/lib/llvm/lib, so the binary cannot start:

  hipify-clang: error while loading shared libraries:
  libLLVM.so.23.0git: cannot open shared object file

Append $ORIGIN/../lib/llvm/lib so both layouts resolve.
@emankov emankov added the clang clang compiler related issue or change label Aug 13, 2026
@emankov
emankov requested a review from ranapratap55 August 13, 2026 11:50
@emankov emankov added the package Packaging issue/fix label Aug 13, 2026
@Samiii777
Samiii777 marked this pull request as ready for review August 13, 2026 15:33
Comment thread CMakeLists.txt Outdated
set_target_properties(hipify-clang PROPERTIES INSTALL_RPATH "")
# Set RPATH for the binary.
set_target_properties(hipify-clang PROPERTIES LINK_FLAGS "-Wl,--enable-new-dtags -Wl,--rpath,\\$ORIGIN/../lib" )
set_target_properties(hipify-clang PROPERTIES LINK_FLAGS "-Wl,--enable-new-dtags -Wl,--rpath,\\$ORIGIN/../lib:\\$ORIGIN/../lib/llvm/lib" )

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hipify-clang should be statically linked against LLVM and clang. Dynamic linking is undocumented and unsupported behavior.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static linking is not available in the configuration where this bug occurs.

TheRock sets LLVM_BUILD_LLVM_DYLIB ON / LLVM_LINK_LLVM_DYLIB ON unconditionally for Linux in compiler/pre_hook_amd-llvm.cmake, which is why the shipped ROCm 7.14 package contains libLLVM.so.23.0git (#2704). In that mode clang's static libraries carry a transitive dependency on libLLVM, so combining them with statically linked LLVM components reintroduces the ODR "duplicate cl::opt registered" failures that #1840 / #1873 fixed — which is exactly why hipify-clang links its LLVM components via LLVM_LINK_COMPONENTS today.

So making hipify-clang statically linked would require changing TheRock's LLVM build configuration first, which is outside this repository. This PR does not add or extend dynamic linking; it only corrects where the binary looks for the libLLVM it is already linked against, which is why it is scoped to the RPATH.

If you do want to pursue static linking against LLVM/clang, that looks like a separate piece of work spanning HIPIFY and TheRock — should I open an issue for it?

@Samiii777
Samiii777 force-pushed the fix-hipify-clang-rpath-llvm-libdir branch from 72f0dd3 to b492315 Compare August 18, 2026 21:17
Replace the hardcoded $ORIGIN/../lib/llvm/lib fallback with a derivation
from LLVM_LIBRARY_DIRS relative to CMAKE_INSTALL_PREFIX, so the RPATH
stays correct for whatever layout the consumed LLVM package reports
instead of just the one layout observed so far. Directories outside the
install prefix are skipped so no build-machine path leaks into the
shipped binary.

hipify-clang cannot be statically linked against LLVM/clang here:
TheRock builds LLVM with LLVM_LINK_LLVM_DYLIB=ON on Linux
unconditionally, and statically linking clang's libraries against a
dylib-mode LLVM reintroduces the ODR "duplicate cl::opt registered"
crash that ROCm#1840/ROCm#1873 fixed. Document that constraint inline so it
isn't relitigated without the history.
@Samiii777
Samiii777 force-pushed the fix-hipify-clang-rpath-llvm-libdir branch from b492315 to 2920d8b Compare August 18, 2026 21:23
Comment thread CMakeLists.txt Resolved
Comment thread CMakeLists.txt Resolved
LINK_FLAGS is a single unquoted string, so building "-Wl,--rpath,a:b" by
hand breaks on any path containing a space, and a path containing ':'
silently corrupts the entry list. Assigning LINK_FLAGS here also
discarded the -Wl,-rpath-link (GNU ld) and -Wl,-z,origin (FreeBSD) flags
that llvm_setup_rpath() appends from add_llvm_executable().

Use the INSTALL_RPATH target property instead: it is a real CMake list
that CMake quotes and joins itself, and setting it replaces the value
llvm_setup_rpath() installed, so the existing "clear it first" line is no
longer needed. --enable-new-dtags moves to target_link_options() with the
LINKER: prefix, which is additive rather than clobbering.

Library dirs that cannot be represented are now reported rather than
silently dropped: a dir outside CMAKE_INSTALL_PREFIX is omitted at STATUS
level (keeping build-machine paths out of a redistributable binary; the
build-tree binary still resolves libLLVM through the BUILD_RPATH
llvm_setup_rpath() derives from LLVM_LIBRARY_DIR for that case), and a
dir containing ':' is omitted with a warning.

Verified the generated RPATH for in-prefix, out-of-prefix, space- and
colon-containing library dirs; the space case now emits a correctly
quoted entry where the previous LINK_FLAGS form split the argument.
Comment thread CMakeLists.txt Resolved
Comment thread CMakeLists.txt Resolved
Add $ORIGIN/../${CMAKE_INSTALL_LIBDIR} variants alongside the lib ones
(ROCm keeps LLVM under lib/llvm/lib even on lib64 distros), and only set
the $ORIGIN INSTALL_RPATH / --enable-new-dtags on non-Apple platforms.
@Samiii777
Samiii777 requested a review from kzhuravl as a code owner October 23, 2026 03:27
@emankov emankov changed the title [HIPIFY] Add LLVM lib dir to hipify-clang RPATH [HIPIFY] Add LLVM lib dir to hipify-clang RPATH Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clang clang compiler related issue or change package Packaging issue/fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[HIPIFY] hipify-clang fails to start: RUNPATH misses LLVM lib dir in versioned ROCm layout (libLLVM.so.23.0git not found)

3 participants