Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Install script and pip packages via sage-spkg #29386

Closed
mkoeppe opened this issue Mar 21, 2020 · 35 comments · Fixed by #36747
Closed

Install script and pip packages via sage-spkg #29386

mkoeppe opened this issue Mar 21, 2020 · 35 comments · Fixed by #36747
Labels
c: build p: major / 3 t: enhancement
Milestone
sage-10.3

Comments

@mkoeppe
Copy link
Contributor

mkoeppe commented Mar 21, 2020

Follow-up from #29287.

Script packages are installed by special rules in build/make/Makefile.

This ticket installs them by sage-spkg instead. This enables script packages to participate in the staged installs (SAGE_DESTDIR) and uninstallation mechanisms.

This is useful for #29387 to clean up #27870/#29003/#29082, which introduced files in SAGE_LOCAL that do not belong to any package and are never uninstalled.

We also make the decision whether spkg-install is generated from a template spkg-install.in or provided as is in the source tree orthogonal to the distinction source=script vs. source=pip.
We simply generate spkg-install if spkg-install.in is present.

Ideally, when done, changing a package's source between script and normal and pip will no longer require running bootstrap.

CC: @dimpase @jhpalmieri @embray @orlitzky

Component: build

Work Issues: Move "dummy script package" code to sage-spkg

Author: Matthias Koeppe

Branch/Commit: u/mkoeppe/install_script_packages_via_sage_spkg @ 7a25d1d

Issue created by migration from https://trac.sagemath.org/ticket/29386
@mkoeppe mkoeppe added this to the sage-9.1 milestone Mar 21, 2020
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 14, 2020

comment:1

pushing these forward to 9.2

@mkoeppe mkoeppe modified the milestones: sage-9.1, sage-9.2 Apr 14, 2020
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jun 4, 2020

Dependencies: #29793

@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe mkoeppe modified the milestones: sage-9.2, sage-9.3 Aug 13, 2020
@embray
Copy link
Contributor

embray commented Sep 2, 2020

comment:6

+1 this would be a very nice improvement.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 24, 2020

Branch: u/mkoeppe/install_script_packages_via_sage_spkg

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 24, 2020

Changed dependencies from #29793 to #29793, #30657

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 24, 2020

Commit: fee5bc2

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 24, 2020

New commits:

f0f03c9build/bin/sage-spkg: Remove dead code for manufacturing an spkg-install script
46e5f68build/bin/sage-spkg: Get rid of USE_LOCAL_SCRIPTS (it was always yes); prepare for script packages
fee5bc2build/bin/sage-spkg: For script packages, do not poison the proxy variables

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 24, 2020

Changed commit from fee5bc2 to e4c27f6

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 24, 2020

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

9ee2110build/bin/sage-dist-helpers: Also use $sudo for storing the wheel file
d7aac84src/doc/en/developer/packaging.rst: Update sdh_... documentation
9b7c7a0build/bin/sage-pip-{install,uninstall}: Fix typo in comment
4135e8bbuild/bin/sage-pip-install: Remove an outdated comment
f2e7075Merge tag '9.2.beta13' into t/29500/install_all_python_packages_via_pip_wheel__create_pep_503_simple_repository_for_wheels
bd0fd2fMerge branch 't/29500/install_all_python_packages_via_pip_wheel__create_pep_503_simple_repository_for_wheels' into t/30657/fix_up__sage__p_
2e20f23build/bin/sage-spkg: Move SAGE_SPKG_{INST,SCRIPTS,WHEELS} setting here from build/bin/install, make them relative to SAGE_INST_LOCAL
ac09b14Merge branch 't/30657/fix_up__sage__p_' into t/29386/install_script_packages_via_sage_spkg
bf0c3dcbuild/bin/sage-spkg: Exit with error for some old-style script invocations
e4c27f6build/bin/sage-spkg: Remove error message when checksums.ini does not exist (script package)

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 24, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

3ceeaa7build/bin/sage-spkg: Fix up for normal and script packages
8c4fb4dbuild/bin/sage-spkg: Handle script packages without package-version.txt
137365abuild/make/Makefile.in: Replace 'SPKG-clean' by 'SPKG-uninstall', with deprecation
a9cbfdabuild/make/Makefile.in: Install script packages using sage-spkg
81a63dfsrc/bin/sage: Use 'make SPKG-uninstall' instead of -clean

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 24, 2020

Changed commit from e4c27f6 to 81a63df

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 24, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

3d6fa59build/make/Makefile.in: Replace 'SPKG-clean' by 'SPKG-uninstall', with deprecation, for pip packages too

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 24, 2020

Changed commit from 81a63df to 3d6fa59

@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 24, 2020

Author: Matthias Koeppe

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 25, 2020

Changed commit from 3d6fa59 to ae5ea98

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 25, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

ae5ea98build/make/Makefile.in: Make SPKG-uninstall targets phony

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 25, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

47ef805WIP: Install pip packages with sage-spkg
6c27c05build/make/Makefile.in: Another SPKG-clean -> SPKG-uninstall

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 25, 2020

Changed commit from ae5ea98 to 6c27c05

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Oct 2, 2020

Changed commit from 6c27c05 to 660d689

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Oct 2, 2020

Branch pushed to git repo; I updated commit sha1. New commits:

9750195WIP: Install pip packages with sage-spkg
be2417cMerge tag '9.2.beta14' into t/30657/fix_up__sage__p_
8ee7ee2build/make/install: Set SAGE_SPKG_WHEELS also here so that script packages still work
72c5901Merge branch 't/30657/fix_up__sage__p_' into t/29386/install_script_packages_via_sage_spkg
660d689Revert "build/make/install: Set SAGE_SPKG_WHEELS also here so that script packages still work"

@mkoeppe mkoeppe added this to the sage-9.4 milestone Feb 13, 2021
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 5, 2021

Changed dependencies from #29793, #30657 to #29793, #30657, #29013

@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 5, 2021

Work Issues: rework on top of #29013

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Apr 5, 2021

Changed dependencies from #29793, #30657, #29013 to #29013

@mkoeppe mkoeppe modified the milestones: sage-9.4, sage-9.5 Jul 19, 2021
@mkoeppe mkoeppe modified the milestones: sage-9.5, sage-9.6 Dec 14, 2021
@mkoeppe mkoeppe modified the milestones: sage-9.6, sage-9.7 Mar 5, 2022
@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jul 21, 2022

Changed commit from 660d689 to 7a25d1d

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Jul 21, 2022

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

483b65ebuild/bin/sage-spkg: Remove dead code for manufacturing an spkg-install script
445ed7ebuild/bin/sage-spkg: Get rid of USE_LOCAL_SCRIPTS (it was always yes); prepare for script packages
74f549cbuild/bin/sage-spkg: For script packages, do not poison the proxy variables
20ddd7ebuild/bin/sage-spkg: Exit with error for some old-style script invocations
25d38e9build/bin/sage-spkg: Remove error message when checksums.ini does not exist (script package)
97c61bebuild/bin/sage-spkg: Fix up for normal and script packages
1b1401dbuild/bin/sage-spkg: Handle script packages without package-version.txt
d850e60build/make/Makefile.in: Install script packages using sage-spkg
7a25d1dWIP: Install pip packages with sage-spkg

@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jul 21, 2022

Changed work issues from rework on top of #29013 to none

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jul 21, 2022

Changed dependencies from #29013 to none

@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jul 21, 2022

comment:25

Rebased and dropped all clean/uninstall changes here because that is #29097

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jul 21, 2022

Work Issues: Move "dummy script package" code to sage-spkg

@mkoeppe mkoeppe modified the milestones: sage-9.7, sage-9.8 Jul 21, 2022
@mkoeppe mkoeppe changed the title Install script packages via sage-spkg Install script and pip packages via sage-spkg Aug 12, 2022
@mkoeppe mkoeppe modified the milestones: sage-9.8, sage-9.9 Dec 31, 2022
@orlitzky orlitzky mentioned this issue Jan 9, 2022
Open
@mkoeppe mkoeppe removed this from the sage-10.0 milestone Apr 30, 2023
@vbraun vbraun closed this as completed in 6cadd84 Dec 14, 2023
@mkoeppe mkoeppe added this to the sage-10.3 milestone Dec 14, 2023
vbraun pushed a commit to vbraun/sage that referenced this issue Dec 17, 2023
sagemathgh-36900: Fix build of optional Cython modules broken in sage…
73e52a4 
…math#29386

The Cython modules depending on optional packages (such as
sage.libs.coxeter3) were not being built after sagemath#29386. This showed up in
the CI as numerous doctest failures; as reported in
sagemath#36775 (comment)

Fixed here by avoiding a clash of environment variables that prevented
the function `is_package_installed_and_updated` from functioning during
the build of sagelib.

<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->

URL: sagemath#36900
Reported by: Matthias Köppe
Reviewer(s): Dima Pasechnik
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c: build p: major / 3 t: enhancement
Projects
None yet
Milestone
sage-10.3
Development

Successfully merging a pull request may close this issue.

build/bin/sage-spkg: Add support for installing script packages mkoeppe/sage
2 participants
@embray @mkoeppe