Documentation updates

docs/Documentation/Applications/Plexos/run_plexos.md

@@ -7,10 +7,10 @@ grand_parent: Applications
 Please follow the [setup instructions](setup_plexos.md) before running the examples. Example scripts for new users are available within the [master branch](https://github.com/NREL/HPC/tree/master/applications/plexos).
 
 !!! note
-    Sometimes newer modules may be availabe in a `test` directory which is hidden by default from the general user base. This obscured release is done to iron out any bugs that may arise during the installation and use of the module while avoiding breaking users existing jobs and workflows. You can use these test modules by running
+    Sometimes newer modules may be available in a `test` directory which is hidden by default from the general user base. This obscured release is done to iron out any bugs that may arise during the installation and use of the module while avoiding breaking users existing jobs and workflows. You can use these test modules by running
 
     ```bash
-    module use /nopt/nrel/apps/modules/test/modulefiles
+    module use /nopt/nrel/apps/software/plexos/modules/test
     module avail
     ```
 

docs/Documentation/Applications/Plexos/setup_plexos.md

@@ -63,7 +63,10 @@ Before we can run PLEXOS, we need to create a license file on the cluster. For t
       <LicenseServerRequestCount />
     </XmlRegistryRoot>'   > ~/.config/PLEXOS/EE_reg.xml
     ```
-## Conda environment for PLEXOS with Python and R
+## Optional: Conda environment for PLEXOS with Python and R
+
+!!! note
+    The following instructions are NOT required for only running PLEXOS. One only needs to load the relevant Gurobi and PLEXOS modules to run a PLEXOS XML database. Users may combine these runs with conda, Julia, or other software simply by loading the relevant modules and activating the appropriate conda and Julia environments.
 
 1. We need to load a few modules and create the requisite conda environment. First, we need to create a conda environment for PLEXOS.
     ```bash
@@ -127,7 +130,7 @@ Before we can run PLEXOS, we need to create a license file on the cluster. For t
     pip install numpy pandas
     ```
 
-## Loading an existing PLEXOS environment
+### Loading an existing PLEXOS environment
 
 If you have successfully followed all the instructions in the previous subsection and installed PLEXOS, you can simply load the following modules and activate the conda environment
 
@@ -137,7 +140,7 @@ module load comp-intel intel-mpi mkl conda
 conda activate plex1
 ```
 
-## Using your own version of R and Python
+### Using your own version of R and Python
 
 This section is in regards to Point 3 in [setting up the PLEXOS environment](#conda-environment-for-plexos-with-python-and-r).
 The following R libraries will need to be installed manually in this case.

docs/Documentation/Applications/gurobi.md

@@ -1,4 +1,4 @@
-# Using the Gurobi Optimizer Solvers
+# Gurobi
 
 *Gurobi Optimizer is a suite of solvers for mathematical programming.*
 
@@ -23,12 +23,12 @@ documentation](https://www.gurobi.com/documentation/).
 
 ## Available Modules
 
-| Kestrel         | Eagle           | Swift           |
-|:---------------:|:---------------:|:---------------:|
-| gurobi/11.0.0   |||
-| gurobi/10.0.2   | gurobi/10.0.2   ||
-| gurobi/10.0.1   | gurobi/10.0.1   ||
-| gurobi/9.5.1    | gurobi/9.5.1    | gurobi/9.5.1    |
+| Kestrel         | Swift           |
+|:---------------:|:---------------:|
+| gurobi/11.0.0   ||
+| gurobi/10.0.2   ||
+| gurobi/10.0.1   ||
+| gurobi/9.5.1    | gurobi/9.5.1    |
 
 
 ## Gurobi and MATLAB

docs/Documentation/Applications/idaes_solvers.md

@@ -0,0 +1,88 @@
+---
+layout: main
+title: IDAES Solvers
+parent: Applications
+
+---
+
+# IDAES Solvers
+
+*Institute for Design of Advanced Energy Systems (IDAES) Solvers are a collection of pre-compiled optimizer binaries with efficient linear algebra solvers that enable solving a variety of MINLP problems.*
+
+Available optimizers include:
+
+1. [Bonmin](#bonmin)
+2. [CBC](#cbc)
+3. [CLP](#clp)
+4. [Couenne](#couenne)
+5. [IPOPT + HSL](#ipopt-hsl)
+
+## Available Modules
+
+!!! info
+    IDAES solvers are currently not available on GPU compute nodes.
+
+| Kestrel (CPU nodes)                  |
+|:------------------------------------:|
+| idaes_solvers/3.4.0-cray-libsci      |
+| idaes_solvers/3.4.0-intel-oneapi-mkl |
+| idaes_solvers/3.4.0-netlib-lapack    |
+
+### v3.4.0
+
+IDAES Solvers v3.4.0 contains the following optimizer versions
+
+| Optimizer   | Version |
+|:-----------:|:-------:|
+| Bonmin      | 1.8.8   |
+| CBC         | 2.10.10 |
+| CLP         | 1.17.8  |
+| Couenne     | 0.5.8   |
+| IPOPT + HSL | 3.13.2  |
+
+!!! note
+    IPOPT is available with performant HSL MA27, MA57, and MA97 linear solvers. These have been shown to perform better than the default MUMPS solver for a variety of renewable energy optimization problems. Please see documentation [here](ipopt.md#ipopt--hsl).
+
+### Usage
+
+Users can run any of the IDAES solvers simply by loading the appropriate module, e.g.,
+
+```bash
+module load idaes_solvers/3.4.0-cray-libsci # OR 
+module load idaes_solvers/3.4.0-netlib-lapack # OR
+module load idaes_solvers/3.4.0-intel-oneapi-mkl
+```
+
+## Bonmin
+
+Bonmin (Basic Open-source Nonlinear Mixed Integer) is an open source solver that leverages CBC and IPOPT to solve general mixed integer nonlinear programs (MINLP). 
+Please refer to the Bonmin [documentation here](https://coin-or.github.io/Bonmin/)
+
+## CBC
+
+COIN-OR Branch and Cut (CBC) solver is an opensource optimizer for solving mixed integer programs (MIP). Please refer to the [documentation here](https://coin-or.github.io/Cbc/intro.html) for more details.
+
+## CLP
+
+COIN-OR Linear Program (CLP) is an open-source solver for solving linear programs. Please refer to the [documentaion here](https://coin-or.github.io/Clp/) for further details. 
+
+## Couenne
+
+Convex Over and Under Envelopes for Nonlinear Estimation (Couenne) is an open-source mixed integer nonlinear programming (MINLP) global optimization solver. Please visit the [following website](https://github.com/coin-or/Couenne) for more details regarding the solver.
+
+## IPOPT + HSL
+
+Interior Point Optimizer (IPOPT) is an open-source nonlinear optimizer.
+[Harwell Subroutine Library (HSL)](https://licences.stfc.ac.uk/product/coin-hsl) is a collection of efficient linear solvers used by IPOPT.
+HSL solvers have been demonstrated to be more performant than the default [MUMPS](https://mumps-solver.org/index.php) (Multifrontal Massively Parallel sparse direct Solver) solver that comes with IPOPT, and are highly recommended.
+
+IPOPT that is distributed as part of IDAES solvers comes pre-compiled with 3 HSL solvers:
+
+1. **MA27** is a serial linear solver suitable for small problems
+2. **MA57** has threaded BLAS operations and is suitable for small to medium-sized problems.
+3. **MA97** is a parallel direct linear solver for sparse symmetric systems. It is more suitable for medium and large problem sizes. Users will may see worse performance on small problems when compared to MA27 and MA57.
+
+All three solvers produce repeatable answers unlike their sibling MA86.
+
+!!! info
+    For additional details regarding IPOPT on Kestrel, e.g., building a custom version, please visit [here](./ipopt.md). Please click here for additional details regarding [HSL](../Development/Libraries/hsl.md) solvers on Kestrel.

docs/Documentation/Applications/index.md

@@ -10,12 +10,17 @@ The following are packages that the NREL Computational Science Center supports:
 | :---------- | :--------- | 
 | [Ansys](./ansys.md)       | Enables modeling, simulation, and visualization of flow, turbulence, heat transfer and reactions for industrial applications | 
 | [AMR-Wind](./amrwind.md)       | A massively parallel, block-structured adaptive-mesh, incompressible flow solver for wind turbine and wind farm simulations | 
+| [Bonmin](./idaes_solvers.md#bonmin) | Open source solver that leverages CBC and IPOPT to solve general mixed integer nonlinear programs (MINLP) |
+| [CBC](./idaes_solvers.md#cbc) | Open-source optimizer for solving mixed integer programs (MIP) |
+| [CLP](./idaes_solvers.md#clp) | Open-source linear program solver |
 | [COMSOL](./comsol.md)      | Multiphysics simulation environment | 
-| Converge | HPC CFD+, focused on engine modeling and simulation | 
+| Converge | HPC CFD+, focused on engine modeling and simulation |
+| [Couenne](./idaes_solvers.md#couenne) | Open-source mixed integer nonlinear programming (MINLP) global optimization solver |
 | [FEniCS](./fenics.md) | Solving partial differential equations by the finite element method | 
 | [GAMS](./gams.md) | High-level modeling system for mathematical programming and optimization | 
 | [Gaussian](./gaussian.md) | Program for calculating molecular electronic structure and reactivity | 
-| [Gurobi](./gurobi.md) | Solver for mathematical programming | 
+| [Gurobi](./gurobi.md) | Solver for mathematical programming |
+| [IPOPT](./ipopt.md) | Open-source interior point nonlinear optimizer |
 | [LAMMPS](./lammps.md) | Open-source classical molecular dynamics program designed for massively parallel systems | 
 | [MATLAB](./Matlab/index.md) | General technical computing framework | 
 | [OpenFOAM](./openfoam.md) | Software for computational fluid dynamics | 
@@ -24,4 +29,4 @@ The following are packages that the NREL Computational Science Center supports:
 | [<nobr>STAR-CCM+</nobr>](./starccm.md) | Engineering simulation package from CD-adapco for solving problems involving flow of fluids or solids, heat transfer, and stress | 
 | [VASP](./vasp.md) | Atomic scale materials modeling, e.g., electronic structure calculations and quantum-mechanical molecular dynamics, from first principles | 
 | [WRF](./wrf.md) | Mesoscale numerical weather prediction system designed for both atmospheric research and operational forecasting applications | 
-
+| [Xpress](./xpressmp.md) | Optimization algorithms and technologies to solve linear, mixed integer and non-linear problems |

docs/Documentation/Development/Libraries/ipopt.md → docs/Documentation/Applications/ipopt.md

@@ -8,28 +8,33 @@ parent: Libraries
 grand_parent: Development
 ---
 
-# Ipopt
+# IPOPT
 
-*Ipopt (Interior Point OPTimizer, pronounced "Eye-Pea-Opt") is an open-source non-linear optimizer using the interior point method.*
+*IPOPT (Interior Point OPTimizer, pronounced "Eye-Pea-Opt") is an open-source non-linear optimizer using the interior point method.*
 
-Ipopt is commonly used in solving power flow, e.g., AC Optimal Power Flow, and controls problems. Please refer to their [project website](https://github.com/coin-or/Ipopt) for the source code. The documentation can be found [here](https://coin-or.github.io/Ipopt/index.html).
+IPOPT is commonly used in solving power flow, e.g., AC Optimal Power Flow, and controls problems. Please refer to their [project website](https://github.com/coin-or/Ipopt) for the source code. The documentation can be found [here](https://coin-or.github.io/Ipopt/index.html).
 
-## Installation
+!!! info
+    IPOPT with HSL linear solvers is available as a module on Kestrel. Please see [IDAES Solvers](./idaes_solvers.md) for additional details.
 
-The default installation instructions can be found in the [Ipopt documentation here](https://coin-or.github.io/Ipopt/INSTALL.html). The remainder of the page describes what has worked for NREL HPC users.
 
-### Eagle
+## Installation from source
 
-We will use COIN-OR's [coinbrew](https://github.com/coin-or/coinbrew) repo to build Ipopt along with the dependencies ASL, HSL and Mumps libraries. 
+!!! warning
+    The following installation instructions are for NREL's older cluster Eagle. We recommend users use the [IPOPT modules available on Kestrel](./idaes_solvers.md). Users will have to tweak the instructions below to build IPOPT from source on Kestrel.  
+
+The default installation instructions can be found in the [IPOPT documentation here](https://coin-or.github.io/Ipopt/INSTALL.html). The remainder of the page describes what has worked for NREL HPC users.
+
+We will use COIN-OR's [coinbrew](https://github.com/coin-or/coinbrew) repo to build IPOPT along with the dependencies ASL, HSL and Mumps libraries. 
 
 !!! note
-    Follow the [instructions to setup the environment](hsl.md#Eagle) for HSL before proceeding with the steps below.
+    Follow the [instructions to setup the environment](../Development/Libraries/hsl.md#hpc) for HSL before proceeding with the steps below.
 
 1. `module load gcc/8.4.0 mkl`
 2. Clone (or download) the [coinbrew](https://github.com/coin-or/coinbrew) repo. If you download the repo you may have to change the permissions on the `coinbrew` *script* before using it: `chmod u+x coinbrew/coinbrew`
 3. `cd` into the directory
 4. `./coinbrew fetch Ipopt@stable/3.13`
-    * This fetches the branch `stable/3.13` of the Ipopt repository as well as the dependencies COIN-OR repositories `ThirdParty-ASL`, `ThirdParty-HSL` and `ThirdParty-Mumps` (other versions of Ipopt can also be downloaded in this manner)
+    * This fetches the branch `stable/3.13` of the IPOPT repository as well as the dependencies COIN-OR repositories `ThirdParty-ASL`, `ThirdParty-HSL` and `ThirdParty-Mumps` (other versions of IPOPT can also be downloaded in this manner)
 5. `cd ThirdParty/HSL`
 6. Copy the HSL source code to the current directory and unpack it
 7. Create a link called `coinhsl` that points to the HSL source code (or rename the directory)
@@ -40,41 +45,41 @@ We will use COIN-OR's [coinbrew](https://github.com/coin-or/coinbrew) repo to bu
     ./coinbrew build Ipopt --disable-java --prefix="${MYAPPS}" --with-metis-cflags="-I${MYINC}" --with-metis-lflags="-L${MYLIB} -lmetis" --with-lapack-lflags="-L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_rt -lpthread -lm -ldl" --with-lapack-cflags="-m64 -I${MKLROOT}/include" ADD_CFLAGS="-march=skylake-avx512" ADD_FCFLAGS="-march=skylake-avx512" ADD_FFLAGS="-march=skylake-avx512"
     ```
 
-    * `build Ipopt` tells `coinbrew` to configure and build Ipopt and its dependencies
-    * `--disable-java` says to build Ipopt without the java interface
+    * `build Ipopt` tells `coinbrew` to configure and build IPOPT and its dependencies
+    * `--disable-java` says to build IPOPT without the java interface
     * `--prefix` says to install the library in "${MYAPPS}"
     * `--with-metis-cflags` gives the compiler the location of the metis header "metis.h"
     * `--with-metis-lflags` gives the linker the location and name of the metis library
     * `--with-lapack-lflags` gives the location of LAPACK and BLAS libraries as well as the needed linker lines.  Here we are using Intel's single dynamic library interface (google "mkl single dynamic library" for more details on this).
-    * `ADD_CFLAGS`, `ADD_FCFLAGS` and `ADD_FFLAGS` say to use those extra flags when compiling C and fortran code, respectively. Specifically, `-march=skylake-avx512` tells the compiler to optimize code for the skylake CPUs on Eagle which is [recommended for perfomance reasons](https://www.nrel.gov/hpc/eagle-software-libraries-mkl.html).
+    * `ADD_CFLAGS`, `ADD_FCFLAGS` and `ADD_FFLAGS` say to use those extra flags when compiling C and fortran code, respectively.
 
 !!! tip
     When linking with MKL libraries, Intel's [link line advisor](https://software.intel.com/content/www/us/en/develop/articles/intel-mkl-link-line-advisor.html) is extremely helpful.
 
 !!! note 
-    When compiling Julia with MKL libraries, the single dynamic library interface is used to link against.  This is why we are also using that linking method.  Using a different linking method will cause unusual behaviors when using Ipopt with Julia (e.g. through JuMP).
+    When compiling Julia with MKL libraries, the single dynamic library interface is used to link against.  This is why we are also using that linking method.  Using a different linking method will cause unusual behaviors when using IPOPT with Julia (e.g. through JuMP).
 
 ## Usage
 
-### Using Custom Ipopt with JuMP
+### Using Custom IPOPT with JuMP
 
 !!! note
-    When running your custom Ipopt build on Eagle, you will need to do two things:
+    When running your custom IPOPT build on Kestrel, you will need to do two things:
 
     1. Load the same MKL module you compiled against:
         ```bash
         module load mkl
         ```
-    2. Add the directory containing Ipopt and HSL libraries to your LD_LIBRARY_PATH
+    2. Add the directory containing IPOPT and HSL libraries to your LD_LIBRARY_PATH
         ```bash
         export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MYLIB}
         ```
 
 
-To use our custom installation of Ipopt with `Ipopt.jl`, we do the following:
+To use our custom installation of IPOPT with `Ipopt.jl`, we do the following:
 
-1. Open the Julia REPL and activate an environment that has Ipopt installed
-2. Tell Julia and `Ipopt.jl` the location of our Ipopt library and executable
+1. Open the Julia REPL and activate an environment that has IPOPT installed
+2. Tell Julia and `Ipopt.jl` the location of our IPOPT library and executable
     ```julia
     ENV["JULIA_IPOPT_LIBRARY_PATH"] = ENV["MYLIB"]
     ENV["JULIA_IPOPT_EXECUTABLE_PATH"] = ENV["MYBIN"]

docs/Documentation/Applications/xpressmp.md

@@ -0,0 +1,57 @@
+# Xpress Solver
+
+*FICO Xpress Optimizer provides optimization algorithms and technologies to solve linear, mixed integer and non-linear problems*
+
+For documentation, forums, and FAQs, see the [FICO
+website](https://www.fico.com/fico-xpress-optimization/docs/latest/overview.html).
+
+The Xpress solver includes algorithms that can solve
+
+* Linear Programs
+* Mixed Integer Programs
+* Quadratic Programs
+* Quadratically Constrained Quadratic Programs
+* Second Order Cone Problems
+
+Xpress solver cannot be used to solve nonlinear programs. 
+
+
+## Available Modules
+
+| Kestrel (CPU)   |
+|:---------------:|
+| xpressmp/9.0.2 |
+| xpressmp/9.2.2 |
+| xpressmp/9.2.5 |
+
+!!! info
+    Xpress is available as a module on Kestrel. Additionally, NREL has a site-wide license for Xpress to run locally on an NREL-issued computer. Please see instructions [here](https://github.nrel.gov/MSOC/fico-xpress).
+
+## Running Xpress Solver on Kestrel
+
+!!! important
+    While Xpress Solver is available as a module on Kestrel for use by all NREL-users, you MUST be a part of the `xpressmp` group on Kestrel. If you are new or have not used Xpress in a while, you can:
+
+    1. Check whether you are a part of this group by running the `groups` command from your terminal, or
+    2. Load the `xpressmp` module and run an example
+
+    If you are not a part of the `xpressmp` linux group and/or are unable to run an Xpress instance, please submit a ticket to [HPC-Help@nrel.gov](mailto:HPC-Help@nrel.gov) requesting access to Xpress on HPC systems and provide a business justification that describes how you intend to use Xpress in your workflow.
+
+Xpress solvers can be used by simply loading the module
+
+```bash
+module load xpressmp/9.2.5
+```
+
+Once the module is loaded, Xpress Solver can be used directly using the command line 
+by running the `optimizer` command.
+
+```bash
+$ optimizer
+FICO Xpress Solver 64bit v9.2.5 Nov  9 2023
+(c) Copyright Fair Isaac Corporation 1983-2023. All rights reserved
+ Optimizer v42.01.04    [/nopt/nrel/apps/software/xpressmp/9.2.5/lib/libxprs.so.42.01.04]
+[xpress kpanda] 
+```
+
+Alternatively, Xpress can now be used directly in Python or Julia by loading the necessary modules and programming environments.

docs/Documentation/Development/Languages/Julia/index.md

@@ -13,12 +13,13 @@ This section contains demos (in the form of scripts and notebooks) and how-to gu
 
 ## Available modules
 
-| Swift              | Vermilion  | Kestrel |
-|:------------------:|:----------:|:----------:|
-|julia/1.6.2-ocsfign |||  
-|julia/1.7.2-gdp7a25 |julia/1.7.2 | julia/1.7.2 |
-|                    |            | julia/1.8.5-generic-linux |
-|                    |            | julia/1.10.0-gcc |
+| Swift              | Vermilion  | Kestrel (CPU) | Kestrel (GPU) |
+|:------------------:|:----------:|:-------------:| :------------:| 
+|julia/1.6.2-ocsfign ||||
+|julia/1.7.2-gdp7a25 |julia/1.7.2 | julia/1.7.2 ||
+|                    |            | julia/1.8.5-generic-linux ||
+|                    |            | julia/1.10.0-gcc ||
+|                    |            || julia/1.10.4 |
 
 
 !!! Bug "Julia 1.9.x does not work well on Sapphire Rapids"