Compile bug: Missing link with cublasLt_static when GGML_STATIC is enabled

[matiaslin]

Git commit

$git rev-parse HEAD

da54f9f

Operating systems

Linux

GGML backends

CUDA

Problem description & steps to reproduce

When compiling llama-cli with GGML_STATIC=ON, we face some undefined references linking issues as shown in the Relevant log output.

This behavior was introduced after we removed linking with CUDA::cublasLt_static in 48b86c4.

Using:

• gcc (GCC) 11.3.0
• cmake version 3.24.1
• nvcc: NVIDIA (R) Cuda compiler driver
  Copyright (c) 2005-2024 NVIDIA Corporation
  Built on Thu_Mar_28_02:18:24_PDT_2024
  Cuda compilation tools, release 12.4, V12.4.131
  Build cuda_12.4.r12.4/compiler.34097967_0

First Bad Commit

48b86c4

Compile command

# set your own gccpath

mkdir -p repro_build && cd repro_build

cmake .. \
 -DLLAMA_BUILD_EXAMPLES=OFF \
 -DLLAMA_BUILD_TESTS=ON \
 -DLLAMA_BUILD_TOOLS=ON \
 -DLLAMA_CURL=OFF \
 -DGGML_STATIC=ON -DBUILD_SHARED_LIBS=OFF \
 -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
 -DCMAKE_CXX_COMPILER=$gccpath/bin/g++ -DCMAKE_C_COMPILER=$gccpath/bin/gcc \
 -DGGML_CUDA=ON -DGGML_CUDA_NO_VMM=1

make llama-cli -j8 || exit -1

Relevant log output

ld: /usr/local/cuda/lib64/libcublas_static.a(batch_gemmEx.cpp.o):(.text._ZL18cublasLtLegacyGemmP13cublasContextP11CUstream_stP14cublasStatus_t10gemmType_t17cublasOperation_tS6_iiiPKvS8_S8_14cudaDataType_tiS8_S9_iPvS9_i12shapeTypeC_tiibmmm.constprop.39+0xb5): undefined reference to `cublasLtLegacyGemmBII'
ld: /usr/local/cuda/lib64/libcublas_static.a(batch_gemmEx.cpp.o):(.text._ZL18cublasLtLegacyGemmP13cublasContextP11CUstream_stP14cublasStatus_t10gemmType_t17cublasOperation_tS6_iiiPKvS8_S8_14cudaDataType_tiS8_S9_iPvS9_i12shapeTypeC_tiibmmm.constprop.39+0x138): undefined reference to `cublasLtLegacyGemmHHH'
ld: /usr/local/cuda/lib64/libcublas_static.a(batch_gemmEx.cpp.o):(.text._ZL18cublasLtLegacyGemmP13cublasContextP11CUstream_stP14cublasStatus_t10gemmType_t17cublasOperation_tS6_iiiPKvS8_S8_14cudaDataType_tiS8_S9_iPvS9_i12shapeTypeC_tiibmmm.constprop.39+0x1c0): undefined reference to `cublasLtLegacyGemmZZZ'
ld: /usr/local/cuda/lib64/libcublas_static.a(batch_gemmEx.cpp.o):(.text._ZL18cublasLtLegacyGemmP13cublasContextP11CUstream_stP14cublasStatus_t10gemmType_t17cublasOperation_tS6_iiiPKvS8_S8_14cudaDataType_tiS8_S9_iPvS9_i12shapeTypeC_tiibmmm.constprop.39+0x245): undefined reference to `cublasLtLegacyGemmDDD'
...
error: ld returned 1 exit status