Make sagelib installation directory more flexible by creating a module src/sage/env_config.py from src/sage/env_config.py.in, defining variables for use in sage.env

[mkoeppe]

This is a follow-up from #28225 - "Allow sage to run in the absence of sage-env". The present ticket also allows $SAGE_LOCAL/bin/python to run and import sage.all in the absence of the shell script sage-env. (Getting a fully functional sage.all is future work.)

This ticket also makes sagelib more independent from the environment variables set by src/bin/sage-env (local/bin/sage-env), and to remove assumptions regarding install locations of sagelib relative to $SAGE_LOCAL.

• To support users who want to install an experimental version of sagelib in other install locations, such as in a user site packages directory.

• For making sagelib available in a user's venv, as in the following example:
  Without this ticket:

  $ sage -python -m venv --system-site-packages ~/personal-sage-venv/
  $ source ~/personal-sage-venv/bin/activate
  (personal-sage-venv) $ python
  >>> import sage.env
  >>> sage.env.SAGE_LOCAL
  '/Users/mkoeppe/personal-sage-venv'                                     # wrong
  >>> import sage.all
  RuntimeError: You must get the file local/bin/sage-maxima.lisp
  

  With this ticket:

  >>> sage.env.SAGE_LOCAL
  '/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local'
  >>> import sage.all
  >>> sage.all.maxima('1')
  1
  

• To support Add configure option --with-sage-venv=SAGE_VENV to create venv there instead of in SAGE_LOCAL #29013/spkg-configure.m4 for python3 #27824: spkg-configure.m4 for python3

• To make it possible for distributions to use a stock src/sage/env.py (providing a custom src/sage/env_conf.py).

In a first step, we just set SAGE_LOCAL and MAXIMA (to the location of the maxima binary), but also SAGE_ROOT could be added (for developer conveniences such as sage.misc.edit_module; see also #25486); and later packages' spkg-configure might be setting MATHJAX_DIR etc.

In a follow-up ticket, src/sage/env_config.py would actually be generated by src/setup.py, not configure.

Ticket #29038 provides an alternative implementation that uses an standalone installed Python module sage_conf, rather than writing into src/sage

Related:

• Meta-ticket: Split sage-env into 5 to clean up sage configuration #21707: Split sage-env into sage-build-env and sage-env
• Add a plain-text config file that feeds into sage.env #22652: Add a plain-text config file that feeds into sage.env
• Meta-ticket: Make sagelib a pip-installable Python source package, listed on PyPI #21507: Task ticket: Make sagelib a pip-installable Python source package, listed on PyPI

CC: @kiwifb @antonio-rojas @isuruf @embray @infinity0 @timokau @jdemeyer @dimpase @jhpalmieri

Component: build

Author: Matthias Koeppe

Branch/Commit: u/mkoeppe/create_module_src_sage_env_config_py_from_src_sage_env_config_py_in__defining_variables_for_use_in_sage_env @ 7a43544

Reviewer: Dima Pasechnik, Erik Bray

Issue created by migration from https://trac.sagemath.org/ticket/29022

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1 +1,12 @@
+This is a step toward #25486: "`$SAGE_LOCAL/bin/sage` should work in an environment without `SAGE_*` variables; and also useful for #29013.
 
+In a first step, we just set `SAGE_LOCAL`, but later packages' `spkg-configure` might be setting `MATHJAX_DIR` etc. 
+
+The hope is that distributions would be able to use a stock `src/sage/env.py` and a custom `src/sage/env_conf.py`.
+
+In a follow-up ticket, `src/sage/env_config.py` would actually be generated by `src/setup.py`, not `configure`.
+
+Related:
+- #21707: Split sage-env into sage-build-env and sage-env
+- #22652: Add a plain-text config file that feeds into sage.env
+

[mkoeppe]

Branch: u/mkoeppe/create_module_src_sage_env_config_py_from_src_sage_env_config_py_in__defining_variables_for_use_in_sage_env

[mkoeppe]

Commit: bba08ec

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

New commits:

bba08ec

Create module src/sage/env_config.py from src/sage/env_config.py.in, defining SAGE_LOCAL

[kiwifb]

comment:5

Interesting concept. But your branch won't achieve much since SAGE_LOCAL is not None. In fact the problem of the value of SAGE_LOCAL is solved in most case by the current setting of os.path.abspath(sys.prefix). Did you mean to set SAGE_ROOT?

[mkoeppe]

comment:6

Replying to @kiwifb:

Interesting concept. But your branch won't achieve much since SAGE_LOCAL is not None. In fact the problem of the value of SAGE_LOCAL is solved in most case by the current setting of os.path.abspath(sys.prefix).

1. With Add configure option --with-sage-venv=SAGE_VENV to create venv there instead of in SAGE_LOCAL #29013, os.path.abspath(sys.prefix) is the location of the venv, and not equal to SAGE_LOCAL.

2. If one starts Sage's Python directly, without going through sage-env, then the environment variable SAGE_LOCAL is not set; and in this case it is retrieved from env_config.py.

[mkoeppe]

comment:7

But yes, let's add SAGE_ROOT to it.

[kiwifb]

comment:8

I misunderstood the way the code works in var. I thought your value from env_config.py wouldn't be used.
For your points

1. OK - I get it, that's a problem

2. I am perfectly happy with it being set as os.path.abspath(sys.prefix) if sage-env is not loaded as it is now. But your alternative is acceptable and of course solves your issue at (1).

I removed sage-env completely from sage-on-gentoo a number of releases ago and it does work nicely with very little patching (I upstreamed most of it). There is only one place now where I have to replace a call to SAGE_ROOT at runtime (in sage.misc.copying). I'd say sage works at 99.99% out of the box without needing sage-env, in particular you should be able to call it directly from python.

However

1. your venv work may introduce a few disparities

2. being able to override some default from a standardised configuration file instead of patching for your distro values is very appealing.

I don't particularly need SAGE_ROOT to be set (in fact None suits me just fine since it doesn't make much sense on a distro), it was just my misconception on what you were doing.

[mkoeppe]

comment:9

Replying to @kiwifb:

I don't particularly need SAGE_ROOT to be set (in fact None suits me just fine since it doesn't make much sense on a distro), it was just my misconception on what you were doing.

Yes, I agree that SAGE_ROOT is not very important, but it's still used for developer convenience functions like those in sage.misc.edit_module.

[kiwifb]

comment:10

Replying to @mkoeppe:

Replying to @kiwifb:

I don't particularly need SAGE_ROOT to be set (in fact None suits me just fine since it doesn't make much sense on a distro), it was just my misconception on what you were doing.

Yes, I agree that SAGE_ROOT is not very important, but it's still used for developer convenience functions like those in sage.misc.edit_module.

SAGE_ROOT is mentioned in the documentation string for edit_devel but is otherwise absent. The bit in documentation should have been amended at the same time as c352b1d#diff-141c6e649507c15775d157f417db1e26 I guess.

Almost all of the SAGE_ROOT stuff left pertains to packaging I think.

[mkoeppe]

comment:11

Note SAGE_SRC is defined through SAGE_ROOT.

[kiwifb]

comment:12

Replying to @mkoeppe:

Note SAGE_SRC is defined through SAGE_ROOT.

var('SAGE_SRC',            join(SAGE_ROOT, 'src'), SAGE_LIB)

I made sure of that :) - so if SAGE_ROOT is None, SAGE_SRC is set to SAGE_LIB. I guess you may prefer the SAGE_ROOT based value. But on a distro it will give me a read only view of the code (which is the best you should hope for if you install stuff with your distro package manager).

[embray]

comment:13

I don't see why it needs to explicitly be a Python module. I think my proposal in #22652 is simpler, because the same file could be used both for configuring the shell environment, and can be read by the Python code.

To me this just seems like extra cruft.

[embray]

comment:14

Replying to @mkoeppe:

Replying to @kiwifb:

Interesting concept. But your branch won't achieve much since SAGE_LOCAL is not None. In fact the problem of the value of SAGE_LOCAL is solved in most case by the current setting of os.path.abspath(sys.prefix).

1. With Add configure option --with-sage-venv=SAGE_VENV to create venv there instead of in SAGE_LOCAL #29013, os.path.abspath(sys.prefix) is the location of the venv, and not equal to SAGE_LOCAL.

sys.prefix may not be, but even in a venv sysconfig.get_config_var('prefix') will always give you the original install prefix for Python, which when using Sage's Python will be SAGE_LOCAL.