feat: upgrade to Palm #827
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
regisb
force-pushed
the
palm
branch
3 times, most recently
from
6c4f344
to
9e7929b
Compare
ghassanmas
reviewed
Apr 17, 2023
regisb
force-pushed
the
palm
branch
5 times, most recently
from
c19bee7
to
83f21ab
Compare
regisb
force-pushed
the
palm
branch
8 times, most recently
from
5452c4c
to
32f2129
Compare
ghassanmas
reviewed
May 6, 2023
regisb
force-pushed
the
palm
branch
4 times, most recently
from
0edeec7
to
5c8e0db
Compare
regisb
force-pushed
the
palm
branch
4 times, most recently
from
dcd155e
to
0ca679d
Compare
Automatically pull Docker build cache from remote registry. This considerably improves build performances, as discovered here: https://github.com/overhangio/test-docker-build/
Among other changes: ORA2 file uploads were stored in a folder named "SET-ME-PLEASE (ex. bucket-name)" (sigh). With this change, the folder should be automatically renamed to "openedxuploads". This issue has been occuring since June 2019... (sigh²) Close #707
This allows us to run:
tutor local do importdemocourse --repo=https://github.com/openedx/openedx-test-course --version=open-release/palm.master --repo-dir=test-course/course
Registering a user was causing a 400 error because the LMS expected the TOS checkbox to be checked, but it's not displayed in the frontend. So we just disable it. Close openedx/wg-build-test-release#262
Instead, the compose plugin must be installed. We deprecate docker-compose because v1 is not supported starting from the end of June 2023. See "evolution of compose": https://docs.docker.com/compose/compose-v2/
These values are by default anyway.
These tutor settings are mostly useless and make templates much more difficult to work with.
It was useless to create a *-permissions job for every application. Instead, we create a single "permissions" service. It can be extended via the "docker-compose-permissions-command" patch.
This paves the way for persisting bind-mounts across runs, since this setting will be a list.
We make use of the Docker build cache to install python and nodejs requirements faster in the case of repeated builds. This feature is only possible for users of BuildKit, so we detect whether `docker buildx` is available at runtime. We do not make use of `COPY --link` because the `--link` option is incompatible with `--chown=app:app`: docker/buildx#1408 For reference, see: https://www.docker.com/blog/dockerfiles-now-support-multiple-build-contexts/ https://docs.docker.com/engine/reference/commandline/buildx_build/#build-context
This is an important change, where we get remove the previous `--mount`
option, and instead opt for persistent bind-mounts.
Persistent bind mounts have several advantages:
- They make it easier to remember which folders need to be bind-mounted.
- Code is *much* less clunky, as we no longer need to generate temporary
docker-compose files.
- They allow us to bind-mount host directories *at build time* using the
buildx `--build-context` option.
- The transition from development to production becomes much easier, as
images will automatically be built using the host repo.
The only drawback is that persistent bind-mounts are slightly less
portable: when a config.yml file is moved to a different folder, many
things will break if the repo is not checked out in the same path.
For instance, this is how to start working on a local fork of
edx-platform:
tutor config save --append MOUNTS=/path/to/edx-platform
And that's all there is to it. No, this fork will be used whenever we
run:
tutor images build openedx
tutor local start
tutor dev start
This change is made possible by huge improvements in the build time
performance. These improvements make it convenient to re-build Docker
images often.
Related issues:
openedx-unsupported/wg-developer-experience#71
openedx-unsupported/wg-developer-experience#66
https://github.com/openedx/wg-developer-experience/issues/166
Templated hooks we almost completely useless, so we get rid of them. This allows us to get rid entirely of hook names and hook indexes, which makes the whole implementation much simpler. Hook removal (with `clear_all`) is achieved thanks to weak references.
We no longer run `docker-compose up --build`. Instead, users are encouraged to build the "openedx-dev" Docker image.
To achieve that, we introduce a new IMAGES_BUILD_REQUIRED filter.
Manual configuration via the `MOUNTS` setting was inconvenient. We (re)introduce a new(ish) `tutor mounts` command. Old timers will perhaps remember that we used to have a `tutor bindmount` command. Well, it's back! But better and different.
This is for openedx/wg-build-test-release#288 Note that we also upgrade mongodb from 4.0 to 4.2, because somehow this hasn't been done in olive.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TODO:
--skip-buildoption inlocal/dev startand don't ever add a build context to the docker-compose files. (and don't build on start)