compile error

[FightingMan]

compile shell:

BASE_CONTAINER_IMAGE_NAME=nvcr.io/nvidia/tritonserver:23.11-py3-min
TENSORRTLLM_BACKEND_REPO_TAG=main
PYTHON_BACKEND_REPO_TAG=r23.11

./build.py -v --no-container-interactive --enable-logging --enable-stats --enable-tracing \
              --endpoint=http --endpoint=grpc --endpoint=sagemaker --endpoint=vertex-ai \
              --backend=ensemble --enable-gpu --endpoint=http --endpoint=grpc \
              --image=base,${BASE_CONTAINER_IMAGE_NAME} \
              --backend=tensorrtllm:${TENSORRTLLM_BACKEND_REPO_TAG} \
              --backend=python:${PYTHON_BACKEND_REPO_TAG} -j 50

errors:

...
-- Found CUDA: /usr/local/cuda (found version "12.3") 
-- Found MPI_C: /opt/hpcx/ompi/lib/libmpi.so (found version "3.1") 
-- Found MPI_CXX: /opt/hpcx/ompi/lib/libmpi.so (found version "3.1") 
-- Found MPI: TRUE (found version "3.1")  
-- Using MPI_INCLUDE_PATH: /opt/hpcx/ompi/include;/opt/hpcx/ompi/include/openmpi;/opt/hpcx/ompi/include/openmpi/opal/mca/hwloc/hwloc201/hwloc/include;/opt/hpcx/ompi/include/openmpi/opal/mca/event/libevent2022/libevent;/opt/hpcx/ompi/include/openmpi/opal/mca/event/libevent2022/libevent/include
-- Using MPI_LIBRARIES: /opt/hpcx/ompi/lib/libmpi.so
-- ========================= Importing and creating target nvinfer ==========================
-- Looking for library nvinfer
-- Library that was found /usr/local/tensorrt/targets/x86_64-linux-gnu/lib/libnvinfer.so
-- ==========================================================================================
-- Building for TensorRT version: 8.6.1.6+cuda12.0.1.011, library version: 9
-- USE_CXX11_ABI: 
-- CMAKE_SYSTEM_PROCESSOR: x86_64
CMake Deprecation Warning at build/_deps/json-src/CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Using the multi-header code from /tmp/tritonbuild/tensorrtllm/build/_deps/json-src/include/
-- Configuring done (743.1s)
CMake Warning (dev) in build/_deps/repo-backend-src/CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "kernel_library_new".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Error at build/_deps/repo-backend-src/CMakeLists.txt:212 (target_link_libraries):
  Target "triton-backend-utils" links to:

    CUDA::cupti

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.



-- Generating done (0.0s)
CMake Generate step failed.  Build files cannot be regenerated correctly.
error: build failed

env:

os:CentOS Linux release 8.2.2.2004 (Core) 
python:Python 3.11.4
gcc:gcc (GCC) 8.5.0 20210514
nvidia:NVIDIA-SMI 545.23.06              Driver Version: 545.23.06    CUDA Version: 12.3  A10

[krishung5]

Hi @FightingMan, there is some bug with the latest cmake version, and will be fixed in the next release of cmake: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/9054. As a WAR, you could cherry-pick this commit to the r23.11 Triton Server repo, which downgrades the cmake version:

• Set CMake version to 3.27.7 server#6675
  This should unblock you from building the backends.

[FightingMan]

Hi @FightingMan, there is some bug with the latest cmake version, and will be fixed in the next release of cmake: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/9054. As a WAR, you could cherry-pick this commit to the r23.11 Triton Server repo, which downgrades the cmake version:

• Set CMake version to 3.27.7 server#6675
  This should unblock you from building the backends.

thank you, I'll try it, it seems the bug has been fixed in main branch of tritonserver ?

[FightingMan]

@krishung5 tested it on main branch, works fine for me, thanks

[schetlur-nv]

@FightingMan thanks for confirming! I will close the issue now, but feel free to leave a comment here if you'd like us to re-open it.