Compile with OpenBlas libraries #23

jd-lara · 2019-10-26T22:48:33Z

I am trying to compile SUNDIALS with LAPACK enabled using the OpenBlas distribution. OpenBlas automatically exports in the same shared library the lapack and blas API so the build seems to be unable to find a working version of the libraries

I am currently trying to update the BinaryBuilder for Sundials v5.0.0 and If I keep this configuration for Sundials I get that LAPACK isn't functional.

-- Looking for LAPACK libraries... OK
-- Checking if LAPACK works... FAILED
CMake Warning at config/SundialsCMakeMacros.cmake:83 (message):
  ------------------------------------------------------------------------

  WARNING: LAPACK not functional

  Blas/Lapack support will not be provided

  ------------------------------------------------------------------------
Call Stack (most recent call first):
  CMakeLists.txt:914 (print_warning)

Any ideas on how to make it work?

This is my current CMAKE configuration

CMAKE_FLAGS="-DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_TOOLCHAIN_FILE=/opt/$target/$target.toolchain"
CMAKE_FLAGS="${CMAKE_FLAGS} -DCMAKE_BUILD_TYPE=Release -DEXAMPLES_ENABLE_C=OFF"
CMAKE_FLAGS="${CMAKE_FLAGS} -DKLU_ENABLE=ON -DKLU_INCLUDE_DIR=$prefix/include -DKLU_LIBRARY_DIR=$prefix/lib"
CMAKE_FLAGS="${CMAKE_FLAGS} -DLAPACK_ENABLE=ON -DLAPACK_LIBRARIES=$LIBBLAS"

if [[ ${nbits} == 32 ]]; then
    echo "***   32-bit BUILD   ***"
    LIBBLAS="$prefix/lib/libopenblas.so"
    cmake ${CMAKE_FLAGS} -DSUNDIALS_INDEX_TYPE=int32_t ..
else
    echo "***   64-bit BUILD   ***"
    LIBBLAS="$prefix/lib/libopenblas64_.so"
    cmake ${CMAKE_FLAGS} ..
fi

The text was updated successfully, but these errors were encountered:

balos1 · 2019-10-28T23:57:12Z

Does this occur with both the 32-bit build and the 64-bit build? SUNDIALS 5.0.0 does not yet work with 64-bit BLAS/LAPACK. What version of openblas is this with? We test against openblas 0.3.0 as part of our CI testing and it builds successfully.

jd-lara · 2019-10-29T17:50:54Z

Yes in both versions, I used already compiled OpenBlass 0.3.0.

jd-lara · 2019-12-08T22:36:23Z

bump @balos1

balos1 · 2019-12-09T16:53:47Z

@jd-lara Per our email conversation, isn't this resolved?

I noticed in your script posted above that you use the variable $LIBBLAS before defining it. So -DLAPACK_LIBRARIES=$LIBBLAS will expand to just “-DLAPACK_LIBRARIES=”, thus the path to BLAS is missing.

I recall that fixing the bug in the script fixed the issue for all builds except when trying to use 64-bit openblas, but we do not support that yet (we will in the next minor/patch release).

smaret · 2019-12-17T14:46:06Z

I get the same error message as @jd-lara on Darwin. I used the following cmake options:

-DLAPACK_ENABLE=ON -DSUNDIALS_INDEX_TYPE=int32_t -DLAPACK_LIBRARIES=~/lib/libopenblas.dylib

@balos1 What options do you use in your CI testing?

smaret · 2019-12-17T15:03:29Z

I found the problem: the path to the lib (~/lib) does not seem to be resolved by cmake. Replacing it by the full path to the lib solves the issue.

Still, I find the message from cmake is a bit confusing: it says that LAPACK isn't functional although the real problem is that it is not found. Using -DLAPACK_LIBRARIES=/nonexistent/path/foo.dylib gives the same error message.

balos1 · 2019-12-17T17:17:21Z

@smaret Yes, I agree that the message is not very clear. The message(s) will be updated in the next release.

jd-lara · 2020-01-21T18:18:37Z

@balos1 I have updated the building code to Sundials 5.1 but still getting some errors. After running this configuration on the CMAKE

CMAKE_FLAGS=(-DCMAKE_INSTALL_PREFIX=${prefix} -DCMAKE_TOOLCHAIN_FILE="${CMAKE_TARGET_TOOLCHAIN}")
CMAKE_FLAGS+=(-DCMAKE_BUILD_TYPE=Release -DEXAMPLES_ENABLE_C=OFF)
CMAKE_FLAGS+=(-DKLU_ENABLE=ON -DKLU_INCLUDE_DIR="$prefix/include" -DKLU_LIBRARY_DIR="$libdir")
CMAKE_FLAGS+=(-DLAPACK_ENABLE=ON)
CMAKE_FLAGS+=(-DLAPACK_LIBRARIES="${libdir}/libopenblas64_.${dlext}")

The output of the cmake command is as follows

Re-run cmake no build system arguments
Appending C release flags
-- Using int64_t for indices
Appending Fortran release flags
-- Determining Fortran name-mangling scheme... FAILED
-- Looking for LAPACK libraries... OK
-- Checking if LAPACK works... FAILED
CMake Warning at config/SundialsCMakeMacros.cmake:83 (message):
  ------------------------------------------------------------------------

  WARNING: LAPACK not functional

  Blas/Lapack support will not be provided

  ------------------------------------------------------------------------
Call Stack (most recent call first):
  CMakeLists.txt:933 (print_warning)


-- Looking for KLU libraries...
-- Checking if KLU works... OK
-- Added NVECTOR_SERIAL module
-- Added NVECTOR_MANYVECTOR module
-- Added SUNMATRIX_BAND module
-- Added SUNMATRIX_DENSE module
-- Added SUNMATRIX_SPARSE module
-- Added SUNLINSOL_BAND module
-- Added SUNLINSOL_DENSE module
-- Added SUNLINSOL_PCG module
-- Added SUNLINSOL_SPBCGS module
-- Added SUNLINSOL_SPFGMR module
-- Added SUNLINSOL_SPGMR module
-- Added SUNLINSOL_SPTFQMR module
-- Added SUNLINSOL_KLU module
-- Added SUNNONLINSOL_NEWTON module
-- Added SUNNONLINSOL_FIXEDPOINT module
-- Added ARKODE module
-- Added CVODE module
-- Added CVODES module
-- Added IDA module
-- Added IDAS module
-- Added KINSOL module
-- Configuring done
-- Generating done
-- Build files have been written to: /workspace/srcdir/sundials-5.1.0/build

I patched the files used to determine the Fortran name mangling unsuccessfully. In 3.1 the name mangling used to be determined properly but not anymore. Any hints on how to address it?

balos1 · 2020-01-22T17:48:28Z

@jd-lara Can you navigate to the /workspace/srcdir/sundials-5.1.0/build/LapackTest directory, and run make and post the output?

jd-lara · 2020-01-22T18:44:26Z

@balos1 This is the output I am getting consistently where the name mangling scheme failed and gave me this output

sandbox:${WORKSPACE}/srcdir/sundials-5.1.0/build/LapackTest # make
/usr/bin/cmake -S/workspace/srcdir/sundials-5.1.0/build/LapackTest -B/workspace/srcdir/sundials-5.1.0/build/LapackTest --check-build-system CMakeFiles/Makefile.cmake 0
Re-run cmake missing file: CMakeFiles/Makefile.cmake
-- The C compiler identification is GNU 4.8.5
-- Check for working C compiler: /opt/bin/cc
-- Check for working C compiler: /opt/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /workspace/srcdir/sundials-5.1.0/build/LapackTest
/usr/bin/cmake -E cmake_progress_start /workspace/srcdir/sundials-5.1.0/build/LapackTest/CMakeFiles /workspace/srcdir/sundials-5.1.0/build/LapackTest/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/workspace/srcdir/sundials-5.1.0/build/LapackTest'
make -f CMakeFiles/ltest.dir/build.make CMakeFiles/ltest.dir/depend
make[2]: Entering directory '/workspace/srcdir/sundials-5.1.0/build/LapackTest'
cd /workspace/srcdir/sundials-5.1.0/build/LapackTest && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /workspace/srcdir/sundials-5.1.0/build/LapackTest /workspace/srcdir/sundials-5.1.0/build/LapackTest /workspace/srcdir/sundials-5.1.0/build/LapackTest /workspace/srcdir/sundials-5.1.0/build/LapackTest /workspace/srcdir/sundials-5.1.0/build/LapackTest/CMakeFiles/ltest.dir/DependInfo.cmake --color=
Dependee "/workspace/srcdir/sundials-5.1.0/build/LapackTest/CMakeFiles/ltest.dir/DependInfo.cmake" is newer than depender "/workspace/srcdir/sundials-5.1.0/build/LapackTest/CMakeFiles/ltest.dir/depend.internal".
Dependee "/workspace/srcdir/sundials-5.1.0/build/LapackTest/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/workspace/srcdir/sundials-5.1.0/build/LapackTest/CMakeFiles/ltest.dir/depend.internal".
Scanning dependencies of target ltest
make[2]: Leaving directory '/workspace/srcdir/sundials-5.1.0/build/LapackTest'
make -f CMakeFiles/ltest.dir/build.make CMakeFiles/ltest.dir/build
make[2]: Entering directory '/workspace/srcdir/sundials-5.1.0/build/LapackTest'
[ 50%] Building C object CMakeFiles/ltest.dir/ltest.o
/opt/bin/cc   -O3 -DNDEBUG   -o CMakeFiles/ltest.dir/ltest.o   -c /workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:2:19: error: ‘SUNDIALS_F77_FUNC’ declared as function returning a function
 #define dcopy_f77 SUNDIALS_F77_FUNC(dcopy, DCOPY)
                   ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:4:13: note: in expansion of macro ‘dcopy_f77’
 extern void dcopy_f77(int *n, const double *x, const int *inc_x, double *y, const int *inc_y);
             ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:4:1: warning: parameter names (without types) in function declaration [enabled by default]
 extern void dcopy_f77(int *n, const double *x, const int *inc_x, double *y, const int *inc_y);
 ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:3:20: error: ‘SUNDIALS_F77_FUNC’ declared as function returning a function
 #define dgetrf_f77 SUNDIALS_F77_FUNC(dgetrf, DGETRF)
                    ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:5:13: note: in expansion of macro ‘dgetrf_f77’
 extern void dgetrf_f77(const int *m, const int *n, double *a, int *lda, int *ipiv, int *info);
             ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:5:1: warning: parameter names (without types) in function declaration [enabled by default]
 extern void dgetrf_f77(const int *m, const int *n, double *a, int *lda, int *ipiv, int *info);
 ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c: In function ‘main’:
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:2:37: error: ‘dcopy’ undeclared (first use in this function)
 #define dcopy_f77 SUNDIALS_F77_FUNC(dcopy, DCOPY)
                                     ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:9:1: note: in expansion of macro ‘dcopy_f77’
 dcopy_f77(&n, &x, &n, &y, &n);
 ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:2:37: note: each undeclared identifier is reported only once for each function it appears in
 #define dcopy_f77 SUNDIALS_F77_FUNC(dcopy, DCOPY)
                                     ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:9:1: note: in expansion of macro ‘dcopy_f77’
 dcopy_f77(&n, &x, &n, &y, &n);
 ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:2:44: error: ‘DCOPY’ undeclared (first use in this function)
 #define dcopy_f77 SUNDIALS_F77_FUNC(dcopy, DCOPY)
                                            ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:9:1: note: in expansion of macro ‘dcopy_f77’
 dcopy_f77(&n, &x, &n, &y, &n);
 ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:3:38: error: ‘dgetrf’ undeclared (first use in this function)
 #define dgetrf_f77 SUNDIALS_F77_FUNC(dgetrf, DGETRF)
                                      ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:10:1: note: in expansion of macro ‘dgetrf_f77’
 dgetrf_f77(&n, &n, &x, &n, &n, &n);
 ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:3:46: error: ‘DGETRF’ undeclared (first use in this function)
 #define dgetrf_f77 SUNDIALS_F77_FUNC(dgetrf, DGETRF)
                                              ^
/workspace/srcdir/sundials-5.1.0/build/LapackTest/ltest.c:10:1: note: in expansion of macro ‘dgetrf_f77’
 dgetrf_f77(&n, &n, &x, &n, &n, &n);
 ^
make[2]: *** [CMakeFiles/ltest.dir/build.make:66: CMakeFiles/ltest.dir/ltest.o] Error 1
make[2]: Leaving directory '/workspace/srcdir/sundials-5.1.0/build/LapackTest'
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/ltest.dir/all] Error 2
make[1]: Leaving directory '/workspace/srcdir/sundials-5.1.0/build/LapackTest'
make: *** [Makefile:87: all] Error 2

The output of the FortranTest is as follows:

sandbox:${WORKSPACE}/srcdir/sundials-5.1.0/build/FortranTest # make
/usr/bin/cmake -S/workspace/srcdir/sundials-5.1.0/build/FortranTest -B/workspace/srcdir/sundials-5.1.0/build/FortranTest --check-build-system CMakeFiles/Makefile.cmake 0
Re-run cmake missing file: CMakeFiles/Makefile.cmake
-- The Fortran compiler identification is GNU 4.8.5
-- Check for working Fortran compiler: /opt/bin/gfortran
-- Check for working Fortran compiler: /opt/bin/gfortran  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /opt/bin/gfortran supports Fortran 90
-- Checking whether /opt/bin/gfortran supports Fortran 90 -- yes
-- Configuring done
-- Generating done
-- Build files have been written to: /workspace/srcdir/sundials-5.1.0/build/FortranTest
/usr/bin/cmake -E cmake_progress_start /workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles /workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/workspace/srcdir/sundials-5.1.0/build/FortranTest'
make -f CMakeFiles/flib.dir/build.make CMakeFiles/flib.dir/depend
make[2]: Entering directory '/workspace/srcdir/sundials-5.1.0/build/FortranTest'
cd /workspace/srcdir/sundials-5.1.0/build/FortranTest && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /workspace/srcdir/sundials-5.1.0/build/FortranTest /workspace/srcdir/sundials-5.1.0/build/FortranTest /workspace/srcdir/sundials-5.1.0/build/FortranTest /workspace/srcdir/sundials-5.1.0/build/FortranTest /workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/flib.dir/DependInfo.cmake --color=
Dependee "/workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/flib.dir/DependInfo.cmake" is newer than depender "/workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/flib.dir/depend.internal".
Dependee "/workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/flib.dir/depend.internal".
Scanning dependencies of target flib
make[2]: Leaving directory '/workspace/srcdir/sundials-5.1.0/build/FortranTest'
make -f CMakeFiles/flib.dir/build.make CMakeFiles/flib.dir/build
make[2]: Entering directory '/workspace/srcdir/sundials-5.1.0/build/FortranTest'
[ 25%] Building Fortran object CMakeFiles/flib.dir/flib.f.o
/opt/bin/gfortran   -O3 -DNDEBUG -O3   -c /workspace/srcdir/sundials-5.1.0/build/FortranTest/flib.f -o CMakeFiles/flib.dir/flib.f.o
[ 50%] Linking Fortran static library libflib.a
/usr/bin/cmake -P CMakeFiles/flib.dir/cmake_clean_target.cmake
/usr/bin/cmake -E cmake_link_script CMakeFiles/flib.dir/link.txt --verbose=1
/opt/bin/ar qc libflib.a  CMakeFiles/flib.dir/flib.f.o
/opt/bin/ranlib libflib.a
make[2]: Leaving directory '/workspace/srcdir/sundials-5.1.0/build/FortranTest'
[ 50%] Built target flib
make -f CMakeFiles/ftest.dir/build.make CMakeFiles/ftest.dir/depend
make[2]: Entering directory '/workspace/srcdir/sundials-5.1.0/build/FortranTest'
cd /workspace/srcdir/sundials-5.1.0/build/FortranTest && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /workspace/srcdir/sundials-5.1.0/build/FortranTest /workspace/srcdir/sundials-5.1.0/build/FortranTest /workspace/srcdir/sundials-5.1.0/build/FortranTest /workspace/srcdir/sundials-5.1.0/build/FortranTest /workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/ftest.dir/DependInfo.cmake --color=
Dependee "/workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/ftest.dir/DependInfo.cmake" is newer than depender "/workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/ftest.dir/depend.internal".
Dependee "/workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles/ftest.dir/depend.internal".
Scanning dependencies of target ftest
make[2]: Leaving directory '/workspace/srcdir/sundials-5.1.0/build/FortranTest'
make -f CMakeFiles/ftest.dir/build.make CMakeFiles/ftest.dir/build
make[2]: Entering directory '/workspace/srcdir/sundials-5.1.0/build/FortranTest'
[ 75%] Building Fortran object CMakeFiles/ftest.dir/ftest.f.o
/opt/bin/gfortran   -O3 -DNDEBUG -O3   -c /workspace/srcdir/sundials-5.1.0/build/FortranTest/ftest.f -o CMakeFiles/ftest.dir/ftest.f.o
[100%] Linking Fortran executable ftest
/usr/bin/cmake -E cmake_link_script CMakeFiles/ftest.dir/link.txt --verbose=1
/opt/bin/gfortran   -O3 -DNDEBUG -O3 CMakeFiles/ftest.dir/ftest.f.o  -o ftest libflib.a 
make[2]: Leaving directory '/workspace/srcdir/sundials-5.1.0/build/FortranTest'
[100%] Built target ftest
make[1]: Leaving directory '/workspace/srcdir/sundials-5.1.0/build/FortranTest'
/usr/bin/cmake -E cmake_progress_start /workspace/srcdir/sundials-5.1.0/build/FortranTest/CMakeFiles 0

jd-lara · 2020-01-27T18:46:12Z

@balos1 we have figure out that this is an issue with gfortran in the toolchain. I will close the issue. Thanks for the help.

balos1 · 2020-01-27T18:55:06Z

@jd-lara Are there fixes we need to incorporate into SUNDIALS?

jd-lara · 2020-01-27T19:03:26Z

Not for the Linux and MacOS versions. If you are interested in the process we have followed you can see it here JuliaPackaging/Yggdrasil#421

We are working on an error in Windows with the LAPACK linking. Here are the error and the build log.

https://dev.azure.com/JuliaPackaging/Yggdrasil/_build/results?buildId=1183&view=logs&j=6c9588bd-a8dc-5eaa-bb0f-0cab3cd64f67&t=af0a3291-65cd-5bc5-3d11-ffbde54b948f&l=3867

I was thinking about opening a different issue for this.

This was referenced Jan 22, 2020

[Sundials 3.1] Build with OpenBLAS support JuliaPackaging/Yggdrasil#378

Merged

Use sundials 3 from artifacts. SciML/Sundials.jl#227

Closed

update for sundials 5.1 JuliaPackaging/Yggdrasil#421

Closed

jd-lara closed this as completed Jan 27, 2020

AnHeuermann mentioned this issue May 15, 2020

[WIP] Upgrade SUNDIALS to v5.2.0 OpenModelica/OpenModelica#796

Closed

balos1 added the archos-windows label May 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compile with OpenBlas libraries #23

Compile with OpenBlas libraries #23

jd-lara commented Oct 26, 2019 •

edited

Loading

balos1 commented Oct 28, 2019 •

edited

Loading

jd-lara commented Oct 29, 2019

jd-lara commented Dec 8, 2019

balos1 commented Dec 9, 2019 •

edited

Loading

smaret commented Dec 17, 2019 •

edited

Loading

smaret commented Dec 17, 2019 •

edited

Loading

balos1 commented Dec 17, 2019

jd-lara commented Jan 21, 2020 •

edited

Loading

balos1 commented Jan 22, 2020

jd-lara commented Jan 22, 2020 •

edited

Loading

jd-lara commented Jan 27, 2020

balos1 commented Jan 27, 2020

jd-lara commented Jan 27, 2020 •

edited

Loading

Compile with OpenBlas libraries #23

Compile with OpenBlas libraries #23

Comments

jd-lara commented Oct 26, 2019 • edited Loading

balos1 commented Oct 28, 2019 • edited Loading

jd-lara commented Oct 29, 2019

jd-lara commented Dec 8, 2019

balos1 commented Dec 9, 2019 • edited Loading

smaret commented Dec 17, 2019 • edited Loading

smaret commented Dec 17, 2019 • edited Loading

balos1 commented Dec 17, 2019

jd-lara commented Jan 21, 2020 • edited Loading

balos1 commented Jan 22, 2020

jd-lara commented Jan 22, 2020 • edited Loading

jd-lara commented Jan 27, 2020

balos1 commented Jan 27, 2020

jd-lara commented Jan 27, 2020 • edited Loading

jd-lara commented Oct 26, 2019 •

edited

Loading

balos1 commented Oct 28, 2019 •

edited

Loading

balos1 commented Dec 9, 2019 •

edited

Loading

smaret commented Dec 17, 2019 •

edited

Loading

smaret commented Dec 17, 2019 •

edited

Loading

jd-lara commented Jan 21, 2020 •

edited

Loading

jd-lara commented Jan 22, 2020 •

edited

Loading

jd-lara commented Jan 27, 2020 •

edited

Loading