Use pipenv to create venv for development #30371
Labels
Comments
|
comment:1
I've worked a bit on it. The current commit is a semi-working prototype, in the sense that all cython files are successfully compiled to C files if invoked via I had problems that none of the source cython files were correctly found on my system (wsl) and that a few dependencies in the current build files were not met ( New commits:
|
|
Commit: |
|
Branch: public/build/inplace |
|
comment:2
I don't really have time to look into this before the 9.2 release. But if you are looking for where include directories are determined, take a look at |
|
Branch pushed to git repo; I updated commit sha1. New commits:
|
|
comment:4
The command The code definitely needs cleanup, but otherwise is ready for a first round of review. |
|
Branch pushed to git repo; I updated commit sha1. New commits:
|
|
comment:7
this is not good. Our |
|
comment:8
And the changes removing old python2 code really need to go on a separate ticket |
|
Branch pushed to git repo; I updated commit sha1. New commits:
|
|
comment:10
Ok, I was under the assumption that the build folder is for build artifacts. If this would be the case, then I suppose it would be better to whitelist those files in the build folder that needs to be checked-in. I've now replaced the global build ignore, by a more fine-grained statement which only ignores the "temporary" files. Concerning the changes about old Python2 code, I've created #30397. |
|
comment:11
Replying to @tobiasdiez:
No, not at all. |
|
comment:13
I wasn't aware of your changes in #28925. I guess what I'm proposing is a different strategy for the source discovery. The approach in #28925 is to change the source files themselves and enrich them by some form of metadata that decides in which distribution / package they land. Here I was proposing to specify the distribution of the files in the setup.py file. I guess both strategies have pros and cons. The main reason for me was that it seems the responsibility of the setup.py file to construct the distribution; why should the source file need to care about where they are used/included? I.e classical example of separation of concerns. I think, it's also the more flexible approach since it's easy to include a source file into multiple distributions etc. Moreover, you also gain a bit of performance at build time since you don't need to parse the files for some magic comments. What do you think? |
|
comment:14
Replying to @tobiasdiez:
One more thing to be aware of is that (1) Separation of concerns is, of course, desirable. But so is locality: (2) My plan for splitting up the distribution is driven to a large extent by the dependencies on different sets of C/C++ libraries. And these are already encoded in Cython distutils directives. Also I have explored a little bit the packaging options beyond using setuptools (in part motivated because of the shortcoming (1)). See #29854 (flit), #29810 (poetry). Unfortunately currently there does not seem to be a proper mainstream solution for the source layout that we want to keep (a monolithic source tree in In any case, I am hoping to have a broader discussion about all of this on sage-devel at the beginning of the 9.3 cycle. |
|
comment:15
One could also take this as an argument to remove the distutil dependency directives and put them into the setup.py, giving you locality and separation of concerns. Anyway, my goal with this PR is not to redesign or modularize the build. I just wanted to have a working inplace build. For this moving the distribution declaration to setup.py was the easiest option, and I quite liked this also from a conceptual point of view. But if the current way is preferred, I change it back of course. |
|
comment:16
Replying to @tobiasdiez:
Ha, no, we just moved them there. Getting rid of the messy file |
|
comment:17
Replying to @tobiasdiez:
I haven't really had a chance to look at it in more detail yet, sorry. Soon... |
|
Branch pushed to git repo; I updated commit sha1. New commits:
|
|
Dependencies: #30397 |
|
Branch pushed to git repo; I updated commit sha1. New commits:
|
|
comment:484
I don't have any opinion about the code, so I am not the right person to review this, but I did get a lot of doctest failures when following the directions in the ticket: Some come from and I also see some instances of errors like I haven't looked at all of the failures to see what else is there. |
|
comment:485
Thanks for testing! Some failing tests are to be expected since pipenv is rather different to the existing way to use sage (as part of sage-the-distribution). But I think for the modularization effort this ticket gives a helpful testcase how sage performs in a "pip-installable" environment. I also had problems with gap (for the github action) and fixed it by manually setting the SAEG_LOCAL env variable. I guess there is potential for improvement as follow-up tickets but I'm not the right person to work on this as I'm not familiar with these parts of sage. |
|
comment:486
Stalled in |
|
Branch pushed to git repo; I updated commit sha1. New commits:
|
|
comment:490
needs a rebase |
This was referenced Aug 23, 2020
|
Branch has merge conflicts |
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We modify
src/setup.pyto support editable installs via pipenv (pipenv install -e ., which invokespip install --editable). Since distribution continues using build/pkgs/sagelib/src/setup.py, the changes to src/setup.py made in this ticket do not affect the existing build process of the Sage library.With this ticket, the following code works:
Instead of the first three commands, one can of course also use
tox -e local-sudo-standard build-local.It generates a virtual env in
src/.venvthat contains all necessary Python packages, and an editable install of sage.Moreover, the installation of some packages might fail (depending on your system configuration). In this case one can manually build it using sage and then install the sage-built wheel in the pipenv. For example,
The output of the added GH workflow is available here: https://github.com/sagemath/sagetrac-mirror/actions?query=workflow%3A%22Build+%26+Test+using+pipenv%22+branch%3Apublic%2Fbuild%2Finplace
Most tests are working, but a few are failing. Only a few tests are failing:
TODO (as followup tickets):
python -m venv,virtualenv(20.x), orpipenv. This is Extend documentation on how to set up the virtual environment #31342.pipenvpatterns and antipatterns for python library project pypa/pipenv#1911 (comment). However, normally you want to put the lock file under version control for the reasons described in https://trac.sagemath.org/timeline?from=2021-02-03T13%3A15%3A22Z. The official statement is "always include in version control", see git - Should Pipfile.lock be committed to version control? pypa/pipenv#598.sage-env.Previous tickets on in-place builds, editable installs:
Depends on #30779
Depends on #30672
Depends on #30673
Depends on #30709
Depends on #30706
Depends on #30901
Depends on #31080
Depends on #29850
Depends on #29314
Depends on #30937
Depends on #30935
Depends on #30731
Depends on #30859
Depends on #30923
Depends on #30910
Depends on #30944
Depends on #30770
Depends on #31216
Depends on #31029
Depends on #30912
Depends on #31357
Depends on #31362
Depends on #31365
Depends on #31377
CC: @tobiasdiez @kiwifb @jhpalmieri @isuruf
Component: build
Keywords: sd111
Author: Tobias Diez
Branch/Commit: public/build/inplace @
c736e0cIssue created by migration from https://trac.sagemath.org/ticket/30371
The text was updated successfully, but these errors were encountered: