ADD: Interface to the SPRAL linear solver

.coin-or/projDesc.xml

@@ -199,7 +199,7 @@
 			</otherPackage>
 
 			<otherPackage>
-				<packageName>A sparse linear solver (MA27, MA57, HSL_MA86, HSL_MA97, WSMP, Pardiso, MUMPS)
+				<packageName>A sparse linear solver (MA27, MA57, HSL_MA86, HSL_MA97, WSMP, Pardiso, MUMPS, SPRAL)
                                 </packageName>
 				<packageURL>
 

COMPILE.md

@@ -0,0 +1,88 @@
+This is a guide detailing the compilation of Ipopt with SPRAL as a linear solver.
+It was developed assuming a standard installation of Ubuntu 18.04 LTS.
+To begin, first, compile the [LANL ANSI version of SPRAL](https://github.com/lanl-ansi/spral) using the [compilation suggestions described therein](https://github.com/lanl-ansi/spral/blob/master/COMPILE.md).
+
+## Cloning the Repository
+First, create a directory where Ipopt will be compiled from source (not via `coinbrew`), e.g.,
+```bash
+mkdir -p ${HOME}/Software
+```
+The remainder of this guide assumes such a directory has been created.
+Then, clone the Ipopt repository via
+```bash
+cd ${HOME}/Software
+git clone https://github.com/lanl-ansi/Ipopt.git
+```
+
+## Rebuilding Configuration Files (optional)
+To rebuild configuration files for Ipopt, if needed (e.g., during development), execute
+```bash
+cd ${HOME}/Software/Ipopt
+git clone https://github.com/coin-or-tools/BuildTools.git
+export COIN_AUTOTOOLS_DIR="${HOME}/local2"
+./BuildTools/install_autotools.sh
+./BuildTools/run_autotools
+```
+If you are not modifying the directory or source structure of Ipopt, this step is not required.
+
+## Compilation with SPRAL
+### Multicore CPUs Only
+To compile Ipopt with SPRAL (CPU support only), specify `${SPRALDIR}` as the directory containing `lib/libspral.a`, then execute
+```bash
+cd ${HOME}/Software/Ipopt
+mkdir build
+cd build
+../configure --prefix=${PWD} --with-spral="-L${SPRALDIR}/lib -L${METISDIR}/lib \
+    -lspral -lgfortran -lhwloc -lm -lcoinmetis -lopenblas -lstdc++ -fopenmp" \
+    --with-lapack-lflags="-llapack -lopenblas"
+make && make install
+```
+
+### Multicore CPUs and NVIDIA GPUs
+To compile with GPU support, execute
+```bash
+cd ${HOME}/Software/Ipopt
+mkdir build
+cd build
+../configure --prefix=${PWD} --with-spral="-L${SPRALDIR}/lib -L${METISDIR}/lib \
+    -lspral -lgfortran -lhwloc -lm -lcoinmetis -lopenblas -lstdc++ -fopenmp \
+    -lcudadevrt -lcudart -lcuda -lcublas" --with-lapack-lflags="-llapack -lopenblas"
+make && make install
+```
+
+## Usage
+Ensure the following environment variables are set when using the SPRAL library:
+```bash
+export OMP_CANCELLATION=TRUE
+export OMP_NESTED=TRUE
+export OMP_PROC_BIND=TRUE
+```
+
+## Testing
+Within the `build` directory created above, the `examples/ScalableProblems` directory contains a set of scalable test problems.
+After compilation of Ipopt, these examples can be compiled via
+```bash
+cd ${HOME}/Software/Ipopt/build
+cd examples/ScalableProblems && make
+```
+As an example, if Ipopt was compiled with SPRAL support, try creating a file named `ipopt.opt` in this directory with the contents
+```
+linear_solver spral
+spral_use_gpu no
+```
+Then, solve a test problem, e.g.,
+```bash
+time ./solve_problem MBndryCntrl1 768
+```
+If SPRAL was compiled with GPU support, next try modifying the `ipopt.opt` file to contain
+```
+linear_solver spral
+spral_use_gpu yes
+```
+Then, solve the same test problem, e.g.,
+```bash
+time ./solve_problem MBndryCntrl1 768
+```
+The `real` time required by the solver should typically decrease on very large, dense problems, compared with a solve using `spral_use_gpu no`.
+If this is not the case, ensure your GPU is actually being recognized by SPRAL.
+This issue is briefly discussed near the end of the associated  [SPRAL compilation guide](https://github.com/lanl-ansi/spral/blob/master/COMPILE.md#multicore-cpus-and-nvidia-gpus-optional).

LICENSE.md

@@ -0,0 +1,13 @@
+Copyright (c) 2020. Triad National Security, LLC. All rights reserved.
+
+This program was produced under U.S. Government contract 89233218CNA000001 for Los Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC for the U.S. Department of Energy/National Nuclear Security Administration. All rights in the program are reserved by Triad National Security, LLC, and the U.S. Department of Energy/National Nuclear Security Administration. The Government is granted for itself and others acting on its behalf a nonexclusive, paid-up, irrevocable worldwide license in this material to reproduce, prepare derivative works, distribute copies to the public, perform publicly and display publicly, and to permit others to do so.
+
+This program is open source under the BSD-3 License.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+3. Neither the name of the the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

configure

@@ -667,6 +667,8 @@ BIT32FCOMMENT
 BITS_PER_POINTER
 HAVE_WSMP_FALSE
 HAVE_WSMP_TRUE
+HAVE_SPRAL_FALSE
+HAVE_SPRAL_TRUE
 HAVE_PARDISO_FALSE
 HAVE_PARDISO_TRUE
 HAVE_MA28_FALSE
@@ -881,6 +883,7 @@ with_hsl
 with_hsl_lflags
 with_hsl_cflags
 with_pardiso
+with_spral
 with_wsmp
 enable_inexact_solver
 enable_java
@@ -1608,6 +1611,7 @@ Optional Packages:
                           directories.)
   --with-pardiso          specify Pardiso library (>= 4.0) from
                           pardiso-project.org
+  --with-spral            specify SPRAL library
   --with-wsmp             specify WSMP library
 
 Some influential environment variables:
@@ -24258,6 +24262,107 @@ else
 fi
 
 
+#########
+# SPRAL #
+#########
+
+
+# Check whether --with-spral was given.
+if test "${with_spral+set}" = set; then :
+  withval=$with_spral; have_spral=yes; spral_lflags=$withval
+else
+  have_spral=no
+fi
+
+
+if test "$have_spral" = "yes"; then
+
+  ac_save_LIBS="$LIBS"
+  LIBS="$spral_lflags $LIBS"
+
+
+  spral_ssids_analyse_namemangling=unknown
+
+  for ac_extra in "no extra underscore" ; do
+    for ac_case in "lower case" "upper case" ; do
+      for ac_trail in "underscore" "no underscore" ; do
+        case $ac_case in
+          "lower case")
+            ac_name=spral_ssids_analyse
+            ;;
+          "upper case")
+            ac_name=SPRAL_SSIDS_ANALYSE
+            ;;
+        esac
+        if test "$ac_trail" = underscore ; then
+          ac_name=${ac_name}_
+        fi
+        { $as_echo "$as_me:${as_lineno-$LINENO}: checking for function $ac_name in $LIBS" >&5
+$as_echo_n "checking for function $ac_name in $LIBS... " >&6; }
+        cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h.  */
+
+/* Override any GCC internal prototype to avoid an error.
+   Use char because int might match the return type of a GCC
+   builtin and then its argument prototype would still apply.  */
+#ifdef __cplusplus
+extern "C"
+#endif
+char $ac_name ();
+#ifdef F77_DUMMY_MAIN
+
+#  ifdef __cplusplus
+     extern "C"
+#  endif
+   int F77_DUMMY_MAIN() { return 1; }
+
+#endif
+int
+main ()
+{
+return $ac_name ();
+  ;
+  return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+  spral_ssids_analyse_namemangling="${ac_case}, ${ac_trail}, ${ac_extra}"
+           ac_success=yes
+else
+  ac_success=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+    conftest$ac_exeext conftest.$ac_ext
+        { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_success" >&5
+$as_echo "$ac_success" >&6; }
+        if test $ac_success = yes ; then
+          break 3
+        fi
+      done
+    done
+  done
+  LIBS=$ac_save_LIBS
+
+  if test $ac_success = yes ; then
+    IPOPTLIB_LFLAGS="$spral_lflags $IPOPTLIB_LFLAGS"
+
+$as_echo "#define IPOPT_HAS_SPRAL 1" >>confdefs.h
+
+
+    else as_fn_error $? "Symbol ssids_solve not found with SPRAL flags $spral_lflags." "$LINENO" 5
+  fi
+
+fi
+
+ if test $have_spral = yes; then
+  HAVE_SPRAL_TRUE=
+  HAVE_SPRAL_FALSE='#'
+else
+  HAVE_SPRAL_TRUE='#'
+  HAVE_SPRAL_FALSE=
+fi
+
+
 ########
 # WSMP #
 ########
@@ -24898,7 +25003,7 @@ else
 JAVA_TEST=Test.java
 CLASS_TEST=Test.class
 cat << \EOF > $JAVA_TEST
-/* #line 24901 "configure" */
+/* #line 25006 "configure" */
 public class Test {
 }
 EOF
@@ -25384,7 +25489,7 @@ else
 JAVA_TEST=Test.java
 CLASS_TEST=Test.class
 cat << \EOF > $JAVA_TEST
-/* #line 25387 "configure" */
+/* #line 25492 "configure" */
 public class Test {
 }
 EOF
@@ -25418,7 +25523,7 @@ JAVA_TEST=Test.java
 CLASS_TEST=Test.class
 TEST=Test
 cat << \EOF > $JAVA_TEST
-/* [#]line 25421 "configure" */
+/* [#]line 25526 "configure" */
 public class Test {
 public static void main (String args[]) {
         System.exit (0);
@@ -26434,6 +26539,10 @@ if test -z "${HAVE_PARDISO_TRUE}" && test -z "${HAVE_PARDISO_FALSE}"; then
   as_fn_error $? "conditional \"HAVE_PARDISO\" was never defined.
 Usually this means the macro was only invoked conditionally." "$LINENO" 5
 fi
+if test -z "${HAVE_SPRAL_TRUE}" && test -z "${HAVE_SPRAL_FALSE}"; then
+  as_fn_error $? "conditional \"HAVE_SPRAL\" was never defined.
+Usually this means the macro was only invoked conditionally." "$LINENO" 5
+fi
 if test -z "${HAVE_WSMP_TRUE}" && test -z "${HAVE_WSMP_FALSE}"; then
   as_fn_error $? "conditional \"HAVE_WSMP\" was never defined.
 Usually this means the macro was only invoked conditionally." "$LINENO" 5

configure.ac

@@ -147,6 +147,25 @@ fi
 
 AM_CONDITIONAL([HAVE_PARDISO],[test "$have_pardiso_mkl$have_pardiso_project" != nono])
 
+#########
+# SPRAL #
+#########
+
+AC_ARG_WITH([spral],
+            AC_HELP_STRING([--with-spral],[specify SPRAL library]),
+            [have_spral=yes; spral_lflags=$withval],
+            [have_spral=no])
+
+if test "$have_spral" = "yes"; then
+  AC_COIN_TRY_LINK([spral_ssids_analyse],[$spral_lflags],[],
+    [IPOPTLIB_LFLAGS="$spral_lflags $IPOPTLIB_LFLAGS"
+     AC_DEFINE(IPOPT_HAS_SPRAL,1,[Define to 1 if SPRAL is available])
+    ],
+    [AC_MSG_ERROR([Symbol ssids_solve not found with SPRAL flags $spral_lflags.])])
+fi
+
+AM_CONDITIONAL([HAVE_SPRAL],[test $have_spral = yes])
+
 ########
 # WSMP #
 ########

doc/install.dox

@@ -339,6 +339,39 @@ that is included in Intel MKL, it is sufficient to ensure
 that MKL is used for the linear algebra routines (Blas/Lapack),
 see \ref EXTERNALCODE_LINALG.
 
+\subsection DOWNLOAD_SPRAL SPRAL (Sparse Parallel Robust Algorithms Library)
+
+To compile \Ipopt with the linear solver SPRAL, the [LANL ANSI version of
+SPRAL](https://github.com/lanl-ansi/spral) should first be downloaded and
+compiled according to the associated <a
+href="https://github.com/lanl-ansi/spral/blob/master/COMPILE.md">SPRAL
+compilation guide</a>. After compilation of SPRAL, while configuring \Ipopt,
+you need to specify the link flags for the library with the `--with-spral`
+flag, including required additional libraries and flags. For example, if you
+want to compile \Ipopt with a purely CPU-parallel version of SPRAL (located in
+`$HOME/lib`) on a GNU/Linux system, you should add the following flags to the
+`configure` command:
+\verbatim
+--with-spral="-L${HOME}/lib -lspral -lgfortran -lhwloc -lm -lcoinmetis \
+              -lopenblas -lstdc++ -fopenmp"
+\endverbatim
+For a CPU and graphics processing unit- (GPU-) parallel version of SPRAL
+(located in `$HOME/lib`), the following flags should be used, instead:
+\verbatim
+--with-spral="-L${HOME}/lib -lspral -lgfortran -lhwloc -lm -lcoinmetis \
+              -lopenblas -lstdc++ -fopenmp -lcudadevrt -lcudart -lcuda -lcublas"
+\endverbatim
+The above assume a SPRAL compilation using OpenBLAS libraries. For best
+performance, use SPRAL together with linear algebra routines from the Intel MKL
+(see \ref EXTERNALCODE_LINALG).
+
+Finally, ensure the following environment variables are employed when using SPRAL
+\verbatim
+export OMP_CANCELLATION=TRUE
+export OMP_NESTED=TRUE
+export OMP_PROC_BIND=TRUE
+\endverbatim
+
 \subsection DOWNLOAD_WSMP WSMP (Watson Sparse Matrix Package)
 
 If you would like to compile \Ipopt with WSMP, you need to obtain the
@@ -399,7 +432,6 @@ to `libhsl.so`.
 If you have problems compiling this feature, you can disable this
 by specifying `--disable-linear-solver-loader` for the `configure` script.
 
-
 \section GETIPOPT Getting the Ipopt Code
 
 \Ipopt is available from the COIN-OR group at GitHub. You can either

doc/main.dox

@@ -143,6 +143,8 @@ required:
         or Intel MKL. Note that current versions from Pardiso Project typically
         offer much better performance than the one from Intel MKL.
 
+    -   [SPRAL](https://github.com/lanl-ansi/spral) (Sparse Parallel Robust Algorithms Library)
+
     -   [WSMP](http://researcher.ibm.com/view_project.php?id=1426) (Watson Sparse Matrix Package)
 
     You must include at least one of the linear solvers above in order
@@ -164,8 +166,8 @@ required:
     this in mind, particularly when you are comparing
     \Ipopt with other optimization codes.
 
-    If you are compiling MA57, HSL_MA77, HSL_MA86, HSL_MA97, or MUMPS
-    within the \Ipopt build system, you should also include
+    If you are compiling MA57, HSL_MA77, HSL_MA86, HSL_MA97, MUMPS,
+    or SPRAL within the \Ipopt build system, you should also include
     the METIS linear system ordering package.
 
     Interfaces to other linear solvers might be added in the future; if