sage_bootstrap: Add command "sage -package list --output={install-requires,requirements,pipfile,debian,....}"

[mkoeppe]

We extend the sage_bootstrap package to add the commands:

• sage -package list --output=install-requires PACKAGES... (for sagelib: Generate pyproject.toml and setup.cfg [install_requires], requirements.txt, Pipfile, and src/Pipfile #30913)
• sage -package list --output=requirements PACKAGES...
• sage -package list --output=pipfile PACKAGES...
• sage -package list --output=debian PACKAGE... etc. (taking over the functionality of the shell script build/bin/sage-get-system-packages.)

They generate text output in the format of setup.cfg [options] install_requires=, requirements.txt, and Pipfile; or a list of system packages

bootstrap can use these commands to generate these files for build/pkgs/sagelib/src/ or src. They are deleted by make bootstrap-clean.

---

References:

• https://pip.pypa.io/en/stable/user_guide/#requirements-files
• https://pipenv-fork.readthedocs.io/en/latest/advanced.html
• https://packaging.python.org/discussions/install-requires-vs-requirements/
• https://pypi.org/project/install-requires/

CC: @seblabbe @dimpase @embray @vbraun @kiwifb @tobihan @tobiasdiez

Component: build

Keywords: sd111

Issue created by migration from https://trac.sagemath.org/ticket/29041

[mkoeppe]

Branch: u/mkoeppe/at___bootstrap_time__generate_src_requirements_txt__src_constraints_txt__src_setup_cfg__install_requires__from_build_pkgs

[mkoeppe]

comment:2

This is what it could look like - just for a small handful of packages for now

---

New commits:

cfcc9fd

bootstrap: generate src/requirements.txt, src/constraints.txt, src/setup.cfg [install_requires] from build/pkgs

[mkoeppe]

Commit: cfcc9fd

[mkoeppe]

comment:3

The apt-get line in the install manual could be generated in the same way.

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

comment:5

Replying to @mkoeppe:

The apt-get line in the install manual could be generated in the same way.

This is #29053.

[dimpase]

comment:6

I see that only few packages, e.g. pillow, are mentioned in the branch. Are they there as an example?

[mkoeppe]

comment:7

Yes, it's an initial example. More can be added on this ticket and/or on follow up tickets.

It may be the most efficient to first put this infrastructure (and the one for Debian) in place and then do invidual packages one by one to determine useful version bounds.

[dimpase]

comment:8

how is it going to play out with version constraints in spkg-configure.m4 files?

[mkoeppe]

comment:9

So far there are no spkg-configure.m4 files for any Python packages, and I would hope there will be no need for them.

[mkoeppe]

comment:10

Basically sagelib-install-requires.txt will play the same role for Python packages as spkg-configure.m4 does for non-Python packages,
and likewise sagelib-requirements.txt : package-version.txt.

[mkoeppe]

comment:11

@embray If you have time - this ticket is waiting for review.

[embray]

comment:12

Similar to what I wrote on #26964, I don't think this should be done in the bootstrap script. I don't want this script growing too far beyond its core purpose, which is simply to bootstrap the configure script.

This could be done by configure instead, using say a requirements.txt.in.

[mkoeppe]

comment:13

See my comments on #26964.

[mkoeppe]

Dependencies: #29233

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

0dc97db	Clean src/doc/en/installation/*.txt in bootstrap-clean, not misc-clean
bae1a82	Merge branch 't/29233/bootstrap____make_distclean_broken' into t/29041/at___bootstrap_time__generate_src_requirements_txt__src_constraints_txt__src_setup_cfg__install_requires__from_build_pkgs
4fb30a6	bootstrap: Add src/constraints.txt etc. to configure tarball

[sagetrac-git]

Changed commit from cfcc9fd to 4fb30a6

[sagetrac-git]

Changed commit from 4fb30a6 to f147962

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

f147962

src/bootstrap: Make executable

[sagetrac-git]

Changed commit from f147962 to 13088e3

[mkoeppe]

Changed dependencies from #29950 to #30719

[mkoeppe]

Changed commit from 13088e3 to none

[mkoeppe]

Changed branch from u/mkoeppe/at___bootstrap_time__generate_src_requirements_txt__src_constraints_txt__src_setup_cfg__install_requires__from_build_pkgs to none

[mkoeppe]

Changed author from Matthias Koeppe to none

[mkoeppe]

Changed dependencies from #30719 to #30719, #30865

[mkoeppe]

Changed work issues from redo on top of #29950 to none

[mkoeppe]

Changed keywords from none to sd111

[mkoeppe]

comment:37

Hoping we can make progress on this ticket this week - https://wiki.sagemath.org/days111

[mkoeppe]

comment:39

Rewriting tools used in the bootstrap phase in Python would also be useful for developing on Windows

[dimpase]

comment:40

hmm, what is so problematic about Windows here? Even if one talks about native, non-cygwin, it is still perfectly possible to run bash etc.

[mkoeppe]

comment:42

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[mkoeppe]

Changed dependencies from #30719, #30865 to none

[mkoeppe]

comment:44

Setting a new milestone for this ticket based on a cursory review.