Optimize and reorganize GitHub-hosted dependencies #30719
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kdmccormick
force-pushed
the
kdmccormick/github-in-reorganize
branch
from
cc6ef59 to
5126a44
Compare
kdmccormick
force-pushed
the
kdmccormick/github-in-reorganize
branch
from
3c4198a to
05b8885
Compare
kdmccormick
changed the title
kdmccormick
force-pushed
the
kdmccormick/github-in-reorganize
branch
from
05b8885 to
8f07577
Compare
kdmccormick
changed the title
kdmccormick
force-pushed
the
kdmccormick/github-in-reorganize
branch
from
8f07577 to
318fcc0
Compare
They have fallen out of date since we switched from Jenkins- run upgrades to GitHub Actions -run upgrades.
They give the impression that, for example, third-party XBlocks belong in github.in. In reality, GitHub-hosted requirements should be avoided in all circumstances. Third-party XBlocks are best added to base.in as a PyPI-hosted dependency. Furthermore, the existing section headers are not even being followed.
We update github.in to use the proper git-based depencency
format specified in the file comment. This format installs
a package as a pre-built wheel:
git+https://github.com/...
instead of a development-mode editable requirement:
-e https://github.com/...
Installing packages in editable mode increases the amount of time
it takes to install edx-platform dependencies, increases the
resulting virtual environment's size, and installs packages in a
way that has several subtle differences compared to the way
wheels are installed:
https://setuptools.pypa.io/en/latest/userguide/development_mode.html#limitations
kdmccormick
force-pushed
the
kdmccormick/github-in-reorganize
branch
from
318fcc0 to
02bf6ea
Compare
kdmccormick
changed the title
kdmccormick
changed the title
I would expect the post-pip-compile steps in `make upgrade` to have taken care of chaninging `-e file://...` into `-e .`, but it didn't for some reason. Normally I would debug this, but #30890 is going to merge in a week or two and it will remove `-e .` from the requirement pins entirely, so I'm just going to fix it manually for now.
feanil
approved these changes
Sep 19, 2022
|
EdX Release Notice: This PR has been deployed to the staging environment in preparation for a release to production. |
|
EdX Release Notice: This PR has been deployed to the production environment. |
1 similar comment
|
EdX Release Notice: This PR has been deployed to the production environment. |
|
EdX Release Notice: This PR has been rolled back from the production environment. |
|
Rerverted in #31021 |
|
EdX Release Notice: This PR has been deployed to the production environment. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The TL;DR is that this will decrease the amount of time & bandwidth it takes to run
pip install -r requirements/edx/base.txtand therefore reduce the size and build time of Tutor's openedx image. It will do so by optimizing the pins in github.in and encouraging developers to only add PyPI-hosted dependencies instead of GitHub-hosted ones.For a longer description, see the commits.
Supporting information
None
Testing instructions
If we can run
make upgradefrom this branch, and the result builds, then we're good.Deadline
None
Other information
Follow-up:
Post-merge: Open a new GH issue for turning all these pins into PyPI requirements, and potentially deleting github.in altogether.