Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[aomp-extras] Build error with LLVM AOMP version 11.6.2 #116

Closed
tpkessler opened this issue Jul 2, 2020 · 7 comments
Closed

[aomp-extras] Build error with LLVM AOMP version 11.6.2 #116

tpkessler opened this issue Jul 2, 2020 · 7 comments
Labels
question Further information is requested

Comments

@tpkessler
Copy link

I'm using Arch Linux and successfully compiled and ran aomp 11.6.1. With the new release 11.6.2 however, I get the following build error for aomp-extras:

-- The C compiler identification is Clang 11.0.0
-- The CXX compiler identification is Clang 11.0.0
-- Check for working C compiler: /opt/rocm/aomp/bin/clang
-- Check for working C compiler: /opt/rocm/aomp/bin/clang - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /opt/rocm/aomp/bin/clang++
-- Check for working CXX compiler: /opt/rocm/aomp/bin/clang++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /usr/lib/libz.so (found version "1.2.11") 

-- AOMP LLVM Found!
--      LLVM_PACKAGE_VERSION:     11.0.0_AOMP_Version_11.6-2 
--      LLVM_DIR:                 /opt/rocm/aomp/lib/cmake/llvm
--      LLVM LLVM_INSTALL_PREFIX: /opt/rocm/aomp
--      CMAKE_INSTALL_PREFIX:     /opt/rocm/aomp

-- Linker detection: GNU ld
------------- NON STANDALONE BUILD ---------------
   -- Building aompextras default set of AMD GPUs: gfx700;gfx701;gfx801;gfx803;gfx900
   -- You may override default with GFXLIST environment variable 
   -- Building libm libdevice with LLVM 11.0.0_AOMP_Version_11.6-2 found at /opt/rocm/aomp
   -- Building default set of AMD GPUs: gfx700;gfx701;gfx801;gfx803;gfx900
   -- You may override default with GFXLIST environment variable 
   -- SKIPPING libm generation for sm_30;sm_35;sm_50;sm_60;sm_61;sm_70 because cuda-clang already defined them.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/torsten/Dokumente/rocm-arch/aomp-extras/src/build
make: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[1]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
Scanning dependencies of target libm-amdgcn-gfx701
Scanning dependencies of target irif_lib
make[2]: Leaving directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Leaving directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
Scanning dependencies of target aomputils
Scanning dependencies of target prepare-builtins
Scanning dependencies of target utils_scripts
Scanning dependencies of target libm-amdgcn-gfx900
Scanning dependencies of target libm-amdgcn-gfx801
Scanning dependencies of target libm-amdgcn-gfx803
make[2]: Leaving directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Leaving directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Leaving directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Leaving directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Leaving directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Leaving directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
Scanning dependencies of target libm-amdgcn-gfx700
make[2]: Leaving directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
make[2]: Entering directory '/home/torsten/Dokumente/rocm-arch/aomp-extras/src/build'
[  1%] Generating libm_c.gfx701.bc
[  1%] Generating libm_cpp.gfx701.bc
[  1%] Generating irif.amdgcn.bc
[  1%] Generating aompcc
[  2%] Generating clang-ocl
[  3%] Building CXX object utils/CMakeFiles/prepare-builtins.dir/src/prepare-builtins.cpp.o
[  4%] Generating libm_c.gfx700.bc
[  4%] Generating raja_build.sh
[  4%] Built target aomputils
[  4%] Generating libm_cpp.gfx803.bc
[  4%] Generating cloc.sh
[  4%] Generating libm_c.gfx801.bc
[  5%] Generating libm_c.gfx803.bc
[  6%] Generating mymcpu
[  6%] Generating libm_c.gfx900.bc
[  6%] Generating aompversion
[  7%] Generating libm_cpp.gfx801.bc
[  7%] Generating modulefile
[  8%] Generating libm_cpp.gfx700.bc
clang-11: error: no such file or directory: 'hip.bc'
clang-11: error: no such file or directory: 'opencl.bc'
clang-11: error: no such file or directory: 'ocml.bc'
clang-11: error: no such file or directory: 'ockl.bc'
[  9%] Generating gputable.txt
clang-11: error: no such file or directory: 'oclc_finite_only_off.bc'
clang-11: error: no such file or directory: 'oclc_daz_opt_on.bc'
clang-11: error: no such file or directory: 'oclc_correctly_rounded_sqrt_on.bc'
clang-11: error: no such file or directory: 'oclc_unsafe_math_off.bc'
clang-11: error: no such file or directory: 'oclc_isa_version_701.bc'
clang-11: error: no such file or directory: 'oclc_wavefrontsize64_on.bc'
make[2]: *** [aomp-device-libs/libm/CMakeFiles/libm-amdgcn-gfx701.dir/build.make:94: aomp-device-libs/libm/libm_c.gfx701.bc] Error 1
make[2]: *** Waiting for unfinished jobs....

It seems to me that the filenames of the *.bc files clang is looking for are missing a .amdgcn.. From rocm-device-libs 3.5.1:

$ ls /opt/rocm/lib/*.bc
hc.amdgcn.bc                               oclc_isa_version_701.amdgcn.bc  oclc_isa_version_908.amdgcn.bc
hip.amdgcn.bc                              oclc_isa_version_702.amdgcn.bc  oclc_isa_version_1010.amdgcn.bc
ockl.amdgcn.bc                             oclc_isa_version_801.amdgcn.bc  oclc_isa_version_1011.amdgcn.bc
oclc_correctly_rounded_sqrt_off.amdgcn.bc  oclc_isa_version_802.amdgcn.bc  oclc_isa_version_1012.amdgcn.bc
oclc_correctly_rounded_sqrt_on.amdgcn.bc   oclc_isa_version_803.amdgcn.bc  oclc_unsafe_math_off.amdgcn.bc
oclc_daz_opt_off.amdgcn.bc                 oclc_isa_version_810.amdgcn.bc  oclc_unsafe_math_on.amdgcn.bc
oclc_daz_opt_on.amdgcn.bc                  oclc_isa_version_900.amdgcn.bc  oclc_wavefrontsize64_off.amdgcn.bc
oclc_finite_only_off.amdgcn.bc             oclc_isa_version_902.amdgcn.bc  oclc_wavefrontsize64_on.amdgcn.bc
oclc_finite_only_on.amdgcn.bc              oclc_isa_version_904.amdgcn.bc  ocml.amdgcn.bc
oclc_isa_version_700.amdgcn.bc             oclc_isa_version_906.amdgcn.bc  opencl.amdgcn.bc

With the previous version, aomp-extras compiled fine. This is an excerpt from the build log with llvm 11.6.1, but aomp-extras at 11.6.2:

-- The CXX compiler identification is Clang 11.0.0
-- Check for working C compiler: /opt/rocm/aomp/bin/clang
-- Check for working C compiler: /opt/rocm/aomp/bin/clang - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /opt/rocm/aomp/bin/clang++
-- Check for working CXX compiler: /opt/rocm/aomp/bin/clang++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /usr/lib/libz.so (found version "1.2.11") 

-- AOMP LLVM Found!
--      LLVM_PACKAGE_VERSION:     11.0.0_AOMP_Version_11.6-1 
--      LLVM_DIR:                 /opt/rocm/aomp/lib/cmake/llvm
--      LLVM LLVM_INSTALL_PREFIX: /opt/rocm/aomp
--      CMAKE_INSTALL_PREFIX:     /opt/rocm/aomp

-- Linker detection: GNU ld
------------- NON STANDALONE BUILD ---------------
   -- Building aompextras default set of AMD GPUs: gfx700;gfx701;gfx801;gfx803;gfx900
   -- You may override default with GFXLIST environment variable 
   -- Building libm libdevice with LLVM 11.0.0_AOMP_Version_11.6-1 found at /opt/rocm/aomp
   -- Building default set of AMD GPUs: gfx700;gfx701;gfx801;gfx803;gfx900
   -- You may override default with GFXLIST environment variable 
   -- SKIPPING libm generation for sm_30;sm_35;sm_50;sm_60;sm_61;sm_70 because cuda-clang already defined them.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/torsten/Dokumente/rocm-arch/aomp-extras/src/build
@estewart08
Copy link
Contributor

estewart08 commented Jul 2, 2020 •
edited
Loading

So I noticed a few things that I want to ask about. The extras build script is reporting a NON STANDALONE BUILD and finds llvm in /opt/rocm/aomp/bin. Could you please detail your cloning and building process for AOMP 11.6-2? If your intent is to build

roct rocr project libdevice extras openmp pgmath flang flang_runtime comgr rocminfo vdi hipvdi ocl

Then this should be a STANDALONE build and should not use anything from the /opt/rocm/aomp install.

For a normal STANDALONE BUILD I would expect something like this

   cd $HOME ; mkdir -p git/aomp11 ; cd git/aomp11
   git clone https://github.com/rocm-developer-tools/aomp
   cd $HOME/git/aomp11/aomp/bin
   git checkout rel_11.6-2
   git pull
   export AOMP_CHECK_GIT_BRANCH=0
   ./clone_aomp.sh
   ./build_aomp.sh

**Note that AOMP env variable is not set here so it will default install to /home/$USER/rocm/aomp
I have a feeling your AOMP env variable is set to AOMP=/opt/rocm/aomp.

@tpkessler
Copy link
Author

tpkessler commented Jul 2, 2020 •
edited
Loading

My intent is to install aomp side-by-side with ROCm so I would like to build as "NON STANDALONE".
I downloaded LLVM from

https://github.com/ROCm-Developer-Tools/amd-llvm-project/archive/aomp-11.6-1.tar.gz

These are the cmake flags for LLVM:

          -DCMAKE_BUILD_TYPE=Release \
          -DLLVM_HOST_TRIPLE=$CHOST \
          -DLLVM_BUILD_UTILS=ON \
          -DLLVM_ENABLE_BINDINGS=OFF \
          -DLLVM_ENABLE_OCAMLDOC=OFF \
          -DLLVM_ENABLE_PROJECTS='clang;lld;compiler-rt' \
          -DLLVM_ENABLE_ASSERTIONS=ON \
          -DLLVM_VERSION_SUFFIX="_AOMP_Version_11.6-2" \
          -DBUG_REPORT_URL='https://github.com/ROCm-Developer-Tools/aomp' \
          -DLLVM_TARGETS_TO_BUILD='AMDGPU;X86' \
          -DCMAKE_SHARED_LINKER_FLAGS='-Wl,--disable-new-dtags' \
          -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON \
          -DCMAKE_INSTALL_RPATH='$ORIGIN:$ORIGIN/../lib:/opt/rocm/aomp/lib' \
          -DLLVM_INCLUDE_BENCHMARKS=OFF \
          -DLLVM_BUILD_TESTS=OFF \
          -DLLVM_INCLUDE_TESTS=OFF \
          -DCLANG_INCLUDE_TESTS=OFF \
          -DOCAMLFIND=NO

For extras I use the file from

https://github.com/ROCm-Developer-Tools/aomp-extras/archive/aomp-11.6-1.tar.gz

and configure it with

    CC=/opt/rocm/aomp/bin/clang \
    CXX=/opt/rocm/aomp/bin/clang++ \
    AOMP=/opt/rocm/aomp \
    cmake -Wno-dev -B build \
          -DCMAKE_INSTALL_PREFIX='/opt/rocm/aomp' \
          -DCMAKE_BUILD_TYPE=Release \
          -DLLVM_DIR='/opt/rocm/aomp' \
          -DROCM_DIR='/opt/rocm' \
          -DAOMP_STANDALONE_BUILD=0 \
          -DDEVICELIBS_ROOT="$_device_dir" \
          -DCMAKE_SHARED_LINKER_FLAGS='-Wl,--disable-new-dtags' \
          -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON \
          -DCMAKE_INSTALL_RPATH='$ORIGIN:$ORIGIN/../lib:/opt/rocm/aomp/lib'

_device_dir points to the extracted tar ball I've downloaded from

https://github.com/RadeonOpenCompute/ROCm-Device-Libs/archive/rocm-3.5.1.tar.gz

@estewart08
Copy link
Contributor

estewart08 commented Jul 2, 2020 •
edited
Loading

Thanks for the clarification. AOMP does have a package inside of ROCm 3.5 called aomp-amdgpu, which is based off of 11.5-0. It looks like you are installing components into /opt/rocm/aomp instead of using the pre-compiled package.

I would strongly suggest not to mix and match ROCm components with components from our Git Hub AOMP releases.
This explains why you are having issues, due to the fact that 11.6-2 (using the amd-stg-open branch for devicelibs,
which is not what is in ROCm 3.5) has a different directory structure and naming scheme for devicelibs' bitcode files.
The llvm/clang and extras from 11.6-1 uses the old naming/structure format. In essence, the STANDALONE AOMP is ahead in
development vs what is released in ROCm.

Here are some options:

1. Use the AOMP 11.6-2 STANDALONE package or build it from source
2. Use the aomp-amdgpu package inside of ROCm 3.5
3. Stay with your 11.6-1 and ROCm 3.5 build (This combo is still untested on our side)
4. Use a combination of 11.6-2 components with ROCm 3.5 components and build AOMP from scratch into 
   /opt/rocm/aomp. This option results in a version of AOMP that we have not tested, requires additional logic to move 
   and rename the bitcode files, and could result in more issues down the road.

Are you able to use precompiled packages? Can you build AOMP 11.6-2 STANDALONE from source? Or are you set on building 11.6-2 with ROCm 3.5 components from source?

Just trying to understand your reasoning here for building AOMP in /opt/rocm/aomp. I assume it is due to the fact you are on Arch Linux.

@tpkessler
Copy link
Author

Thank you very much @estewart08 for your detailed explanation! I'm able to compile AOMP 11.6-2 STANDALONE from source and can successfully compile and run the examples. I've created a package for Arch Linux that will soon be available on the AUR.

@gregrodgers
Copy link
Contributor

This will be fixed in aomp 11.7-1 due any hour.

@gregrodgers
Copy link
Contributor

@tpkessler Can we close this now? I would recommend installing ROCM 4.1 rocm-dkms package and AOMP 13.0-2.

@gregrodgers gregrodgers added the question Further information is requested label Apr 20, 2021
@tpkessler
Copy link
Author

Yes. I totally forgot about this issue. I've been able to build aomp-extras for a long time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gregrodgers @estewart08 @tpkessler