Trac #28175: Move optional sage optimization backends (COIN, CPLEX, G…

…urobi) to separate Cython packages to remove OptionalExtension problems `sage.numerical.backends` depends on very little from Sage. We propose to split out the optional backends (COIN, CPLEX, Gurobi) to separate Cython packages. With this ticket, sagelib no longer has the optional dependency on cbc. Instead, these new packages depend on the `sage` package (i.e., sagelib). This makes it easier to reconfigure when new solvers are installed, as sagelib does not have to be recompiled. By eliminating the use of the optional extensions mechanisms from sagelib, this will also simplify packaging binary distributions. The new packages are on PyPI: {{{ wget -P upstream https://files.pythonhosted.org/packages/29/8a/65fd90a89 0fcfb56166c4071aa7a4fb5b865d313304f60474870f0e89998/sage_numerical_backe nds_cplex-9.0b12.tar.gz wget -P upstream https://files.pythonhosted.org/packages/2c/5c/e1532bb6c de28cf86ebffd408ccb500d7b846bc29bbd1c5e479a2f18acb6/sage_numerical_backe nds_gurobi-9.0b12.tar.gz wget -P upstream https://files.pythonhosted.org/packages/17/f4/308d3e151 d82daf2c58ca26bce157fac31f763b2b60b5173243c09291e0a/sage_numerical_backe nds_coin-9.0b12.tar.gz }}} Development here: - https://github.com/mkoeppe/sage-numerical-backends-cplex - https://github.com/mkoeppe/sage-numerical-backends-coin - https://github.com/mkoeppe/sage-numerical-backends-gurobi See also: - Follow-up: #28920: Move sage optimization backend framework (`sage.numerical.backends`) to separate Cython packages - #26511 - Meta-ticket: Use Python optimization interfaces: PuLP, Pyomo, cylp... - https://groups.google.com/forum/#!topic/sage-packaging/_FLDkqD2X8c URL: https://trac.sagemath.org/28175 Reported by: mkoeppe Ticket author(s): Matthias Koeppe Reviewer(s): Dima Pasechnik
sagemath · Jan 1, 2020 · 5fa32aa · 5fa32aa
2 parents 7afa68e + 1279897
commit 5fa32aa
Show file tree

Hide file tree

Showing 28 changed files with 183 additions and 5,892 deletions.
diff --git a/build/pkgs/sage_numerical_backends_coin/checksums.ini b/build/pkgs/sage_numerical_backends_coin/checksums.ini
@@ -0,0 +1,4 @@
+tarball=sage_numerical_backends_coin-VERSION.tar.gz
+sha1=5c9708603564c55fa5fddf66ac06f68f4c6145e2
+md5=94ed7c290defc4bf16dcdb0e0c67ca35
+cksum=2355301440
diff --git a/build/pkgs/sage_numerical_backends_coin/dependencies b/build/pkgs/sage_numerical_backends_coin/dependencies
@@ -0,0 +1,5 @@
+cbc cysignals $(SAGE_SRC)/sage/numerical/backends/generic_backend.pxd $(SAGE_SRC)/sage/cpython/string.pxd $(SAGE_SRC)/sage/cpython/string_impl.h | $(SAGERUNTIME) setuptools pip cython
+
+----------
+All lines of this file are ignored except the first.
+It is copied by SAGE_ROOT/build/make/install into SAGE_ROOT/build/make/Makefile.
diff --git a/build/pkgs/sage_numerical_backends_coin/package-version.txt b/build/pkgs/sage_numerical_backends_coin/package-version.txt
@@ -0,0 +1 @@
+9.0b12
diff --git a/build/pkgs/sage_numerical_backends_coin/spkg-check b/build/pkgs/sage_numerical_backends_coin/spkg-check
@@ -0,0 +1,2 @@
+cd src
+sage-python23 setup.py test
diff --git a/build/pkgs/sage_numerical_backends_coin/spkg-install b/build/pkgs/sage_numerical_backends_coin/spkg-install
@@ -0,0 +1,15 @@
+if [ "$SAGE_LOCAL" = "" ]; then
+   echo "SAGE_LOCAL undefined ... exiting";
+   echo "Maybe run 'sage -sh'?"
+   exit 1
+fi
+
+#Install new version
+cd src
+
+sdh_pip_install .
+
+if [ $? -ne 0 ]; then
+   echo "Error installing sage_numerical_backends_coin."
+   exit 1
+fi
diff --git a/build/pkgs/sage_numerical_backends_coin/type b/build/pkgs/sage_numerical_backends_coin/type
@@ -0,0 +1 @@
+optional
diff --git a/build/pkgs/sage_numerical_backends_cplex/checksums.ini b/build/pkgs/sage_numerical_backends_cplex/checksums.ini
@@ -0,0 +1,4 @@
+tarball=sage_numerical_backends_cplex-VERSION.tar.gz
+sha1=59b5f344f5b3d2e1f2fb7dcddef208bcc99e710a
+md5=f8dcd101092634dac83097a36d64c872
+cksum=2646359461
diff --git a/build/pkgs/sage_numerical_backends_cplex/dependencies b/build/pkgs/sage_numerical_backends_cplex/dependencies
@@ -0,0 +1,5 @@
+cysignals $(SAGE_SRC)/sage/numerical/backends/generic_backend.pxd $(SAGE_SRC)/sage/cpython/string.pxd $(SAGE_SRC)/sage/cpython/string_impl.h | $(SAGERUNTIME) setuptools pip cython
+
+----------
+All lines of this file are ignored except the first.
+It is copied by SAGE_ROOT/build/make/install into SAGE_ROOT/build/make/Makefile.
diff --git a/build/pkgs/sage_numerical_backends_cplex/package-version.txt b/build/pkgs/sage_numerical_backends_cplex/package-version.txt
@@ -0,0 +1 @@
+9.0b12
diff --git a/build/pkgs/sage_numerical_backends_cplex/spkg-check b/build/pkgs/sage_numerical_backends_cplex/spkg-check
@@ -0,0 +1,2 @@
+cd src
+sage-python23 setup.py test
diff --git a/build/pkgs/sage_numerical_backends_cplex/spkg-install b/build/pkgs/sage_numerical_backends_cplex/spkg-install
@@ -0,0 +1,15 @@
+if [ "$SAGE_LOCAL" = "" ]; then
+   echo "SAGE_LOCAL undefined ... exiting";
+   echo "Maybe run 'sage -sh'?"
+   exit 1
+fi
+
+#Install new version
+cd src
+
+sdh_pip_install .
+
+if [ $? -ne 0 ]; then
+   echo "Error installing sage_numerical_backends_cplex."
+   exit 1
+fi
diff --git a/build/pkgs/sage_numerical_backends_cplex/type b/build/pkgs/sage_numerical_backends_cplex/type
@@ -0,0 +1 @@
+optional
diff --git a/build/pkgs/sage_numerical_backends_gurobi/checksums.ini b/build/pkgs/sage_numerical_backends_gurobi/checksums.ini
@@ -0,0 +1,4 @@
+tarball=sage_numerical_backends_gurobi-VERSION.tar.gz
+sha1=1a3ea79350cb6bffaff0c785bdb310f4c21475f4
+md5=31e4da1d0538422c370a0149b4da809d
+cksum=2128488628
diff --git a/build/pkgs/sage_numerical_backends_gurobi/dependencies b/build/pkgs/sage_numerical_backends_gurobi/dependencies
@@ -0,0 +1,5 @@
+cysignals $(SAGE_SRC)/sage/numerical/backends/generic_backend.pxd $(SAGE_SRC)/sage/cpython/string.pxd $(SAGE_SRC)/sage/cpython/string_impl.h | $(SAGERUNTIME) setuptools pip cython
+
+----------
+All lines of this file are ignored except the first.
+It is copied by SAGE_ROOT/build/make/install into SAGE_ROOT/build/make/Makefile.
diff --git a/build/pkgs/sage_numerical_backends_gurobi/package-version.txt b/build/pkgs/sage_numerical_backends_gurobi/package-version.txt
@@ -0,0 +1 @@
+9.0b12
diff --git a/build/pkgs/sage_numerical_backends_gurobi/spkg-check b/build/pkgs/sage_numerical_backends_gurobi/spkg-check
@@ -0,0 +1,2 @@
+cd src
+sage-python23 setup.py test
diff --git a/build/pkgs/sage_numerical_backends_gurobi/spkg-install b/build/pkgs/sage_numerical_backends_gurobi/spkg-install
@@ -0,0 +1,15 @@
+if [ "$SAGE_LOCAL" = "" ]; then
+   echo "SAGE_LOCAL undefined ... exiting";
+   echo "Maybe run 'sage -sh'?"
+   exit 1
+fi
+
+#Install new version
+cd src
+
+sdh_pip_install .
+
+if [ $? -ne 0 ]; then
+   echo "Error installing sage_numerical_backends_gurobi."
+   exit 1
+fi
diff --git a/build/pkgs/sage_numerical_backends_gurobi/type b/build/pkgs/sage_numerical_backends_gurobi/type
@@ -0,0 +1 @@
+optional
diff --git a/src/doc/en/thematic_tutorials/linear_programming.rst b/src/doc/en/thematic_tutorials/linear_programming.rst
@@ -436,117 +436,85 @@ graph, in which all the edges have a capacity of 1::
     :align: center
 
 
-Solvers
--------
+Solvers (backends)
+------------------
 
 Sage solves linear programs by calling specific libraries. The
 following libraries are currently supported:
 
-* `CBC <http://www.coin-or.org/projects/Cbc.xml>`_: A solver from
-  `COIN-OR <http://www.coin-or.org/>`_
+* `CBC <https://github.com/coin-or/Cbc>`_: A solver from
+  `COIN-OR <http://www.coin-or.org/>`_,
+  provided under the Eclipse Public License (EPL), which is an open source
+  license but incompatible with GPL. CBC and the Sage CBC backend can be
+  installed using the shell command::
 
-  Provided under the open source license CPL, but incompatible with
-  GPL. CBC can be installed using the shell command
-  ``sage -i cbc sagelib``.
+    $ sage -i -c sage_numerical_backends_coin
 
 * `CPLEX
-  <http://www-01.ibm.com/software/integration/optimization/cplex/>`_:
-  A solver from `ILOG <http://www.ilog.com/>`_
+  <https://www.ibm.com/products/ilog-cplex-optimization-studio/>`_:
+  Proprietary, but available for free for researchers and students through
+  IBM's Academic Initiative.  Since :trac:`27790`, only versions 12.8 and
+  above are supported.
 
-  Proprietary, but free for researchers and students.
+  Install CPLEX according to the instructions on the
+  website, which includes obtaining a license key.
 
-  Since :trac:`27790`, only versions 12.8 and above are supported.
+  Then find the installation directory of your ILOG CPLEX Studio installation, which contains subdirectories ``cplex``, ``doc``, ``opl``, etc.
+  Set the environment variable ``CPLEX_HOME`` to this directory; for example using the following shell command (on macOS)::
 
-* `CVXOPT <http://cvxopt.org/>`_: an LP solver from Python Software for
-  Convex Optimization, uses an interior-point method, always installed in Sage.
-
-  Licensed under the GPL.
-
-* `GLPK <http://www.gnu.org/software/glpk/>`_: A solver from `GNU
-  <http://www.gnu.org/>`_
-
-  Licensed under the GPLv3. This solver is always installed, as the default one, in Sage.
-
-* `GUROBI <http://www.gurobi.com/>`_
-
-  Proprietary, but free for researchers and students.
-
-* `PPL <http://bugseng.com/products/ppl>`_: A solver from bugSeng.
-
-  This solver provides exact (arbitrary precision) computation, always installed in Sage.
-
-  Licensed under the GPLv3.
-
-Using CPLEX or GUROBI through Sage
-----------------------------------
-
-ILOG's CPLEX and GUROBI being proprietary softwares, you must be in possession
-of several files to use it through Sage. In each case, the **expected** (it may
-change !) filename is joined.
+    $ export CPLEX_HOME=/Applications/CPLEX_Studio1210
 
-* A valid license file
-    * GUROBI : a ``.lic`` file
+  or (on Linux)::
 
-* A compiled version of the library
-    * CPLEX : ``libcplex.a``
-    * GUROBI : ``libgurobi55.so`` (or more recent)
+    $ export CPLEX_HOME=/opt/ibm/ILOG/CPLEX_Studio1210
 
-* The library file
-    * CPLEX : ``cplex.h``
-    * GUROBI : ``gurobi_c.h``
+  Now verify that the CPLEX binary that you will find in the subdirectory
+  ``cplex/bin/ARCH-OS`` starts correctly, for example::
 
-The environment variable defining the licence's path must also be set when
-running Sage. You can append to your ``.bashrc`` file one of the following :
+    $ $CPLEX_HOME/cplex/bin/x86-64_osx/cplex
+    Welcome to IBM(R) ILOG(R) CPLEX(R) Interactive Optimizer...
 
-* For GUROBI
+  This environment variable only needs to be set for the following step:
+  Install the Sage CPLEX backend using the shell command::
 
-  .. CODE-BLOCK:: bash
+    $ sage -i -c sage_numerical_backends_cplex
 
-    export GRB_LICENSE_FILE=/path/to/the/license/gurobi.lic
-
-
-As Sage also needs the files library and header files the easiest way is to
-create symbolic links to these files in the appropriate directories:
-
-* For CPLEX:
-    * ``libcplex.a`` -- in ``SAGE_ROOT/local/lib/``, type:
-
-      .. CODE-BLOCK:: shell-session
-
-        $ ln -s /path/to/lib/libcplex.a .
-
-    * ``cplex.h`` -- in ``SAGE_ROOT/local/include/``, type:
-
-      .. CODE-BLOCK:: shell-session
-
-        $ ln -s /path/to/include/cplex.h .
-
-    *  ``cpxconst.h`` (if it exists) -- in ``SAGE_ROOT/local/include/``, type:
-
-      .. CODE-BLOCK:: shell-session
+* `CVXOPT <http://cvxopt.org/>`_: an LP solver from Python Software for
+  Convex Optimization, uses an interior-point method, always installed in Sage.
 
-        $ ln -s /path/to/include/cpxconst.h .
+  Licensed under the GPL.
 
-* For GUROBI
+* `GLPK <http://www.gnu.org/software/glpk/>`_: A solver from `GNU
+  <http://www.gnu.org/>`_
 
-    * ``libgurobi56.so`` -- in ``SAGE_ROOT/local/lib/``, type:
+  Licensed under the GPLv3. This solver is always installed, as the default one, in Sage.
 
-      .. CODE-BLOCK:: shell-session
+* `Gurobi <https://www.gurobi.com/>`_:
+  Proprietary, but available for free for researchers and students via Gurobi's
+  Academic Program.
 
-        $ ln -s /path/to/lib/libgurobi56.so libgurobi.so
+  Install Gurobi according to the instructions on the website,
+  which includes obtaining a license key.  The installation should make the
+  interactive Gurobi shell ``gurobi.sh`` available in your ``PATH``.
+  Verify this by typing the shell command ``gurobi.sh``::
 
-    * ``gurobi_c.h`` -- in ``SAGE_ROOT/local/include/``, type:
+    $ gurobi.sh
+    Python 3.7.4 (default, Aug 27 2019, 11:27:39)
+    ...
+    Gurobi Interactive Shell (mac64), Version 9.0.0
+    Copyright (c) 2019, Gurobi Optimization, LLC
+    Type "help()" for help
+    gurobi>
 
-      .. CODE-BLOCK:: shell-session
+  If this does not work, adjust your ``PATH`` or create symbolic links so
+  that ``gurobi.sh`` is found.
 
-        $ ln -s /path/to/include/gurobi_c.h .
+  Now install the Sage Gurobi backend using the shell command::
 
-**It is very important that the names of the symbolic links in Sage's folders
-be precisely as indicated. If the names differ, Sage will not notice that
-the files are present**
+    $ sage -i -c sage_numerical_backends_gurobi
 
-Once this is done, Sage is to be asked to notice the changes by running:
+* `PPL <http://bugseng.com/products/ppl>`_: A solver from bugSeng.
 
-.. CODE-BLOCK:: shell-session
+  This solver provides exact (arbitrary precision) computation, always installed in Sage.
 
-    $ make
+  Licensed under the GPLv3.
diff --git a/src/module_list.py b/src/module_list.py
@@ -1056,27 +1056,6 @@ def uname_specific(name, value, alternative):
     Extension("sage.numerical.backends.interactivelp_backend",
               ["sage/numerical/backends/interactivelp_backend.pyx"]),
 
-    OptionalExtension("sage.numerical.backends.gurobi_backend",
-              ["sage/numerical/backends/gurobi_backend.pyx"],
-              libraries = ["gurobi"],
-              condition = os.path.isfile(SAGE_INC + "/gurobi_c.h") and
-                  os.path.isfile(SAGE_LOCAL + "/lib/libgurobi.so")),
-
-    OptionalExtension("sage.numerical.backends.cplex_backend",
-              ["sage/numerical/backends/cplex_backend.pyx"],
-              libraries = ["cplex"],
-              condition = os.path.isfile(SAGE_INC + "/cplex.h") and
-                  os.path.isfile(SAGE_LOCAL + "/lib/libcplex.a")),
-
-    OptionalExtension("sage.numerical.backends.coin_backend",
-              ["sage/numerical/backends/coin_backend.pyx"],
-              language = 'c++',
-              libraries = ["Cbc", "CbcSolver", "Cgl", "Clp", "CoinUtils",
-                           "OsiCbc", "OsiClp", "Osi"] + lapack_libs,
-              library_dirs = lapack_library_dirs,
-              include_dirs = lapack_include_dirs,
-              package = 'cbc'),
-
     ################################
     ##
     ## sage.plot