[Issue]: rocMLIR compilation issue

[RandUser123sa]

Problem Description

Hi ppl,

Im trying to compile version 6.1.0 with these configuration options

cmake
-Wno-dev
-G Ninja
-D CMAKE_CXX_FLAGS="${CXXFLAGS} -fcf-protection=none"
-D CMAKE_INSTALL_PREFIX=/usr
-D CMAKE_BUILD_TYPE=Release
-D CMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang
-D CMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++
-D BUILD_FAT_LIBROCKCOMPILER=ON
-D BUILD_SHARED_LIBS=ON
-D ROCM_TEST_CHIPSET=gfx900
..

ninja

and I got error:

[3/299] Building CXX object mlir/lib/Dialect/Rock/Transforms/CMakeFiles/obj.MLIRRockTransforms.dir/ViewToTransform.cpp.o
FAILED: mlir/lib/Dialect/Rock/Transforms/CMakeFiles/obj.MLIRRockTransforms.dir/ViewToTransform.cpp.o
/opt/rocm/llvm/bin/clang++ -DGTEST_HAS_RTTI=0 -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_LIBCPP_ENABLE_ASSERTIONS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/mnt/arhiv/rocm/rocm-build/build/rocmlir/mlir/lib/Dialect/Rock/Transforms -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/mlir/lib/Dialect/Rock/Transforms -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/external/llvm-project/llvm/include -I/mnt/arhiv/rocm/rocm-build/build/rocmlir/external/llvm-project/llvm/include -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/external/llvm-project/mlir/include -I/mnt/arhiv/rocm/rocm-build/build/rocmlir/external/llvm-project/llvm/tools/mlir/include -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/external/mlir-hal/mlir/include -I/mnt/arhiv/rocm/rocm-build/build/rocmlir/external/mlir-hal/include -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/external/mlir-hal/include -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/mlir/include -I/mnt/arhiv/rocm/rocm-build/build/rocmlir/mlir/include -fcf-protection=none -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Werror=global-constructors -O3 -DNDEBUG -std=gnu++17 -fPIC -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_LIBCPP_ENABLE_ASSERTIONS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_LIBCPP_ENABLE_ASSERTIONS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-exceptions -funwind-tables -fno-rtti -UNDEBUG -MD -MT mlir/lib/Dialect/Rock/Transforms/CMakeFiles/obj.MLIRRockTransforms.dir/ViewToTransform.cpp.o -MF mlir/lib/Dialect/Rock/Transforms/CMakeFiles/obj.MLIRRockTransforms.dir/ViewToTransform.cpp.o.d -o mlir/lib/Dialect/Rock/Transforms/CMakeFiles/obj.MLIRRockTransforms.dir/ViewToTransform.cpp.o -c /mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/mlir/lib/Dialect/Rock/Transforms/ViewToTransform.cpp
In file included from /mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/mlir/lib/Dialect/Rock/Transforms/ViewToTransform.cpp:14:
/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/mlir/include/mlir/Conversion/TosaToRock/TosaToRock.h:21:10: fatal error: 'mlir/Conversion/RocMLIRPasses.h.inc' file not found
21 | #include "mlir/Conversion/RocMLIRPasses.h.inc"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
[20/299] Building CXX object external/llvm-project/llvm/tools/mlir/lib/Dialect/Linalg/Transforms/CMakeFiles/obj.MLIRLinalgTransforms.dir/ElementwiseOpFusion.cpp.o
ninja: build stopped: subcommand failed.

[2948/3182] Building CXX object mlir/lib/Dialect/Rock/Transforms/CMakeFiles/obj.MLIRRockTransforms.dir/CheckResidency.cpp.o
FAILED: mlir/lib/Dialect/Rock/Transforms/CMakeFiles/obj.MLIRRockTransforms.dir/CheckResidency.cpp.o
/opt/rocm/llvm/bin/clang++ -DGTEST_HAS_RTTI=0 -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_LIBCPP_ENABLE_ASSERTIONS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/mnt/arhiv/rocm/rocm-build/build/rocmlir/mlir/lib/Dialect/Rock/Transforms -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/mlir/lib/Dialect/Rock/Transforms -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/external/llvm-project/llvm/include -I/mnt/arhiv/rocm/rocm-build/build/rocmlir/external/llvm-project/llvm/include -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/external/llvm-project/mlir/include -I/mnt/arhiv/rocm/rocm-build/build/rocmlir/external/llvm-project/llvm/tools/mlir/include -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/external/mlir-hal/mlir/include -I/mnt/arhiv/rocm/rocm-build/build/rocmlir/external/mlir-hal/include -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/external/mlir-hal/include -I/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/mlir/include -I/mnt/arhiv/rocm/rocm-build/build/rocmlir/mlir/include -fcf-protection=none -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -Werror=global-constructors -O3 -DNDEBUG -std=gnu++17 -fPIC -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_LIBCPP_ENABLE_ASSERTIONS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D_DEBUG -D_GLIBCXX_ASSERTIONS -D_LIBCPP_ENABLE_ASSERTIONS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-exceptions -funwind-tables -fno-rtti -UNDEBUG -MD -MT mlir/lib/Dialect/Rock/Transforms/CMakeFiles/obj.MLIRRockTransforms.dir/CheckResidency.cpp.o -MF mlir/lib/Dialect/Rock/Transforms/CMakeFiles/obj.MLIRRockTransforms.dir/CheckResidency.cpp.o.d -o mlir/lib/Dialect/Rock/Transforms/CMakeFiles/obj.MLIRRockTransforms.dir/CheckResidency.cpp.o -c /mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/mlir/lib/Dialect/Rock/Transforms/CheckResidency.cpp
In file included from /mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/mlir/lib/Dialect/Rock/Transforms/CheckResidency.cpp:16:
In file included from /mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/external/mlir-hal/include/mlir/Dialect/MHAL/IR/MHAL.h:27:
/mnt/arhiv/rocm/release/rocMLIR-rocm-6.1.0/external/mlir-hal/include/mlir/Dialect/MHAL/IR/MHALTypes.h:22:10: fatal error: 'mlir/Dialect/MHAL/IR/MHALTypes.h.inc' file not found
22 | #include "mlir/Dialect/MHAL/IR/MHALTypes.h.inc"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Any help please ?

Operating System

Slackware 15.0

CPU

AMD Ryzen 7 3800X 8-Core Processor

GPU

AMD Radeon RX 7900 XTX

ROCm Version

ROCm 6.1.0

ROCm Component

rocMLIR

Steps to Reproduce

No response

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

[RandUser123sa]

It's seems file it's not generated.

namespace mlir {
/// Generate the code for registering conversion passes.
#define GEN_PASS_REGISTRATION
#include "mlir/Conversion/RocMLIRPasses.h.inc"

} // namespace mlir

how I can trigger generation of this file ?

[krzysz00]

One thing I can say is that -DBUILD_FAT_LIBROCKCOMPILER=ON necessarily implies a static library build from where we're standing, so try removing that option

[RandUser123sa]

The same problem,
I try to compile the project with example :

mkdir build
cd build
cmake -G Ninja .. -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++
ninja check-rocmlir-build-only

and the same result.

[RandUser123sa]

I found how to generate the file, but does not know what cause the problem.

I cat the build.ninja file and grep for both files MHALTypes.h.inc and RocMLIRPasses.h.inc when the project compile is around 30% and manual start the process from another console: mlir-tblgen -gen-pass-decls -name RocMLIRConversion ...
and on first console the process was successful. It's seems files is generated and after that deleted from some operation.

But now come the second problem, after I do cmake --build . --target package
not all library's was in package and these that was build into package was linked wrong on build library location which also it's not added into package. For example:

    libMLIRTransformDialect.so.17git => not found
    libMLIRTransformDialectUtils.so.17git => not found
    libMLIRTensorInferTypeOpInterfaceImpl.so.17git => not found
    libMLIRMIGraphXTransforms.so.2 => /mnt/arhiv/rocm/rocm-build/build/rocmlir/lib/libMLIRMIGraphXTransforms.so.2 (0x00007f6287993000)
    libMLIRMIGraphXDialect.so.2 => /mnt/arhiv/rocm/rocm-build/build/rocmlir/lib/libMLIRMIGraphXDialect.so.2 (0x00007f62878c9000)
    libMLIRRockTransforms.so.2 => /mnt/arhiv/rocm/rocm-build/build/rocmlir/lib/libMLIRRockTransforms.so.2 (0x00007f6287781000)
    libMLIRRockTuning.so.2 => /mnt/arhiv/rocm/rocm-build/build/rocmlir/lib/libMLIRRockTuning.so.2 (0x00007f628775c000)
    libMLIRRockUtility.so.2 => /mnt/arhiv/rocm/rocm-build/build/rocmlir/lib/libMLIRRockUtility.so.2 (0x00007f6287727000)

[krzysz00]

... yes, I'm clarifying here that our build system is designed under the assumption that we will be uses a static library when other ROCm components depend on us - a shared library build is intended specifically for internal testing.

If you want -DBUILD_LIB_FATROCKCOMPILER=On to be compatible with a shared library build, I'll gladly review your patches to that effect

[krzysz00]

To be clear, the combination of CMake flags given above is not one we test or support. If you have changes we can make to our build steps that would allow that combination of flags to work, they won't be rejected unless they break the static library usecase.

I'm leaving this issue open to allow further discussion on the matter but am making no commitment to doing the work of fixing it

[RandUser123sa]

I don't think the problem are in CMake flags.

With official way that I get from project readme have the same effect.
mkdir build cd build cmake -G Ninja .. -DBUILD_FAT_LIBROCKCOMPILER=On -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++ ninja

I even try to compile it with single thread and result is the same. Generation of these files it's not triggered. I watch the directories and these files RocMLIRPasses.h.inc, RocMLIRPasses.h.inc.d, MHALTypes.h.inc, MHALTypes.h.inc.d it's not generated but as I write above if I generate them too early something is delete them.

My ninja version is 1.12.1 cmake version 3.29.3

[aquad]

This fixes it for me.

diff -aur a/mlir/lib/Dialect/Rock/Transforms/CMakeLists.txt b/mlir/lib/Dialect/Rock/Transforms/CMakeLists.txt
--- a/mlir/lib/Dialect/Rock/Transforms/CMakeLists.txt	2023-06-09 08:18:16.000000000 +0800
+++ b/mlir/lib/Dialect/Rock/Transforms/CMakeLists.txt	2024-05-26 16:26:56.644359564 +0800
@@ -23,6 +23,8 @@
   DEPENDS
   MLIRRockOpsIncGen
   MLIRRockPassIncGen
+  MLIRConversionPassIncGen
+  RocMLIRConversionPassIncGen
 
   LINK_LIBS PUBLIC
   MLIRAffineTransforms

[RandUser123sa]

Thanks, this patch fix the compilation issue.

[krzysz00]

@aquad Can you submit that as a PR please? (I'd like to make sure your name's on it so we're not just stealing your patch)