Skip to content
This repository has been archived by the owner on May 2, 2024. It is now read-only.

Tutor Adoption 4: Devstack Parity: Improve Dev Workflows #146

Closed
kdmccormick opened this issue Jan 5, 2023 · 13 comments
Closed

Tutor Adoption 4: Devstack Parity: Improve Dev Workflows #146

kdmccormick opened this issue Jan 5, 2023 · 13 comments
Assignees
Labels
epic Large unit of work, consisting of multiple tasks

Comments

@kdmccormick
Copy link
Collaborator

kdmccormick commented Jan 5, 2023

Background

#144 describes three options for improving tutor dev workflows around locally-changed code and requirements. This is one option.

### Tasks
- [x] https://github.com/openedx/wg-developer-experience/issues/152
- [x] https://github.com/overhangio/tutor/pull/813
- [ ] https://github.com/openedx/edx-platform/issues/31798
- [ ] https://github.com/openedx/wg-developer-experience/issues/150
- [ ] https://github.com/openedx/wg-devops/issues/22
- [ ] https://github.com/openedx/wg-developer-experience/issues/162
- [ ] https://github.com/openedx/wg-devops/issues/21
- [ ] https://github.com/kdmccormick/tutor/pull/32
- [ ] https://github.com/kdmccormick/tutor/pull/33
- [ ] https://github.com/openedx/wg-developer-experience/issues/177
- [ ] https://github.com/openedx/wg-developer-experience/issues/159

Notes

These tasks are based on this comment from the quickdev TEP:

I too want to make sure we are solving the right problems rather than applying workarounds to things that are broken on a deeper level. Let me break down the problems I see with tutor dev as a it stands.

node_modules

I’m no npm expert, but from my research it seems that npm really wants node_modules to be stored in the root of the repository itself. The ways around this I can think of are:

  1. Install all packages in “global” mode (npm -g) so that they are installed somewhere in the container outside of the repository. In a prototype I was able to hackily achieve this by shadowing the npm binary with a script that forwarded all arguments to npm -g. This would also require further unraveling of the paver-based assets pipeline, which is a good thing anyway, and is something I think we could get 2U’s help on.
  2. In the dev Docker image, move /openedx/edx-platform/node_modules to /openedx/node_modules and replace the former with a link to the latter. Then, do the same thing for mounted repositories using an ENTRYPOINT script. I had this idea working in an earlier version of quickdev, but dropped it because it seemed too hacky.
  3. For micro-frontends, stop trying to use Docker containers in development mode. For backend services, just accept that we will have this problem until all legacy frontends are removed and our backends are purely JSON API servers.

python virtualenv

Thankfully, since the venv is stored at /openedx/venv, which is outside of edx-platform, running pip install -r requirements/dev.txt isn’t required after mounting a repository. However, people often want to modify Python requirements and/or install local versions of packages without too much of a hassle.
Outside of turning the venv into a named volume, I think the best improvement we could make would be to speed up the dev image build as much as possible. That way, it is much faster for developers to update edx-platform’s requirements lists and re-build the image. A few ideas I have:

static assets

I’m not very knowledgable here but I’m sure the situation could be improved upstream. There’s discussion about the asset pipeline on this github issue. Even without overhauling the whole edx-platform asset pipeline, I have to hope that’s there’s something we could do to move the generated assets out of edx-platform…

jobs

This one’s already in progress but it’s worth mentioning anyway: having the pluggable jobs framework merged should help us condense any remaining “mounting repository preparation” commands down into single command, e.g. tutor dev do prepare-mounted-platform --mount=edx-platform.

egg-info

This one’s fairly minor, but as far as I can tell, edx-platform will always need Open_edX.egg-info file to be present in the repository. If we don’t use a named volume, this means that pip install -e . is non-optional. I think the jobs framework, mentioned above, could handle this nicely.

@kdmccormick kdmccormick self-assigned this Jan 5, 2023
@kdmccormick kdmccormick moved this from 📥 New to 🛠️ In Progress in Developer Experience Working Group Jan 5, 2023
@kdmccormick kdmccormick changed the title Apply upstream fixes for improved tutor dev workflows Option 3: Apply upstream fixes for improved tutor dev workflows Jan 5, 2023
@kdmccormick kdmccormick changed the title Option 3: Apply upstream fixes for improved tutor dev workflows Option 3: Apply edx-platform fixes for improved tutor dev workflows Jan 5, 2023
@kdmccormick kdmccormick changed the title Option 3: Apply edx-platform fixes for improved tutor dev workflows Option 3: Apply upstream fixes for improved tutor dev workflows Jan 5, 2023
@kdmccormick kdmccormick moved this from 🛠️ In Progress to ⚠️ Blocked in Developer Experience Working Group Jan 5, 2023
@regisb
Copy link

regisb commented Jan 6, 2023

I think it's great that you are attempting to resolve the underlying issues by attacking edx-platform directly. It's certainly the most difficult path forward, but also the one with the most potential. Let me know how if I can help.

@bradenmacdonald
Copy link

python virtualenv

For me one of the recurring problems is that the edx-platform dependencies change all the time...

$ tutor dev start
... gives some error ...
$ tutor quickdev pip-restore
$ tutor dev start
... now it works

But later I need a custom edx-platform:

tutor dev start -m ~/edx-platform
... gives some error depending on what's changed, like: ImportError: cannot import name 'VerticalBlockRenderCompleted' from 'openedx_filters.learning.filters'
$ tutor dev run -m ~/edx-platform lms pip install -r requirements/edx/development.txt
$ tutor dev run -m ~/edx-platform lms ./manage.py lms migrate 
... now it works

What I think we need is some way that we can automatically detect when there's a difference between the requirements specified in the current requirements/*.txt file(s) and the actual venv in use, and then automatically update the venv to match before launching the dev environment. If this were possible, it would solve the primary annoyance I have using Tutor as a devstack, which is that it's very unclear what pieces are working together and which command will "reset" things so that the right tutor version is used with the right edx-platform version is used with the right venv is updated to the right requirements versions so that everything works.

Or another way of phrasing that is, can things like #153 and openedx/edx-platform#31505 take us far enough that we run pip install automatically every time we start tutor dev, so that the "right" requirements are always installed in the venv ?

@kdmccormick kdmccormick added the epic Large unit of work, consisting of multiple tasks label Mar 6, 2023
kdmccormick added a commit to kdmccormick/tutor that referenced this issue Mar 13, 2023
Before this commit, setting up an edx-platform development environment
took multiple steps:

  tutor dev launch
  tutor dev run --mount=/path/to/edx-platform lms bash
  >> pip install -e .
  >> npm clean-install
  >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

  tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick added a commit to kdmccormick/tutor that referenced this issue Mar 13, 2023
Before this commit, setting up an edx-platform development environment
took multiple steps:

  tutor dev launch
  tutor dev run --mount=/path/to/edx-platform lms bash
  >> pip install -e .
  >> npm clean-install
  >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

  tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick added a commit to kdmccormick/tutor that referenced this issue Mar 13, 2023
Before this commit, setting up an edx-platform development environment
took multiple steps:

  tutor dev launch
  tutor dev run --mount=/path/to/edx-platform lms bash
  >> pip install -e .
  >> npm clean-install
  >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

  tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick added a commit to kdmccormick/tutor that referenced this issue Mar 13, 2023
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick added a commit to kdmccormick/tutor that referenced this issue Mar 13, 2023
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick added a commit to kdmccormick/tutor that referenced this issue Mar 13, 2023
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick added a commit to kdmccormick/tutor that referenced this issue Mar 13, 2023
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick added a commit to kdmccormick/tutor that referenced this issue Mar 14, 2023
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
regisb pushed a commit to overhangio/tutor that referenced this issue Mar 15, 2023
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

These bind-mounts:

* ../build/openedx/themes:/openedx/themes
* ../build/openedx/requirements:/openedx/requirements

existed in the dev lms and cms containers, but they did
not exist in the lms-job and cms-job containers.

This means that themes and requirements that were *built into the
image* would exist in the job containers, but live updates to the
themes and requirements would not apply.

To resolve this, we set ``volumes:`` on the lms-job and cms-job
services so that they match the volumes for the normal lms and
cms services.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
@kdmccormick
Copy link
Collaborator Author

Sorry Braden, I read your comment last month and totally forgot to respond. The workflow you bring up is a very common one and it's something I've been thinking about a lot!

Something that makes this issue more challenging/interesting is that there are a bunch of containers all running edx-platform: lms, cms, their -worker variants, potentially their -job variants, and potentially any number of ephemeral run containers. Each container has its own /openedx/venv, and the changes to the venv are destroyed when the container is destroyed. So, in order to ensure updated requirements at runtime we'd need to run pip install within each container upon startup. Out of curiosity, I tried this, and it went about as poorly as you'd expect :P

Quickdev addresses this by using a shared, persisted volume for /openedx/venv across all containers. In that world, I bet it'd be possible to run pip install once whenever the platform is started. I believe fixes like #153 and openedx/edx-platform#31505 would bring a no-op pip install down to the realm of 10-30 seconds, making this strategy fairly reasonable.

Back to Tutor proper, though: without a shared /openedx/venv, updating requirements at runtime just does not seem viable. Instead, I think we'd need to make use of the Docker build, which is designed for the use case "when something changes, rebuild the affected parts (and only the affected parts)". Specifically, I can see two way of leveraging the Docker build:

  1. When edx-platform is mounted, copy requirements/edx/development.txt (and package-lock.json, while we're at it) into the build context. Then, in the Dockerfile, use those requirements lists as input to the openedx-dev image. That would trigger a requirements rebuild whenever starting or rebooting a platform with modified requirements.
  2. Have edx-platform developers work within the build context. That is: ask developers to move their repo to $(tutor config printroot)/env/build/openedx/edx-platform. Then, in the Dockerfile, we could use the repo as input to the openedx-dev image. Seems crazy at first, but it might get us a lot: all this stuff around requirement management, static asset building, and running setup.py could be handled at the Dockerfile level instead of a hand-rolled initialization system.

Sorry, I know that's lot, but I would love to hear your thoughts when you have the chance @bradenmacdonald and @regisb . At the moment, requirements management for tutor dev is noticably harder than it is in Devstack. With some smart refactoring, though, I think we could make it noticably easier and more reliable than it is in Devstack.

@bradenmacdonald
Copy link

@kdmccormick Thanks! I'm not knowledgeable enough to predict how well your two ideas would work, but they're great ideas and I'd be very interested in seeing what happens if we try them out.

@kdmccormick kdmccormick changed the title Option 3: Apply upstream fixes for improved tutor dev workflows Option 3: Upstream & core fixes for improved tutor dev workflows Mar 19, 2023
@kdmccormick kdmccormick changed the title Option 3: Upstream & core fixes for improved tutor dev workflows Tutor Adoption: Devstack Parity: Dev Workflows: Upstream & Core fixes Mar 19, 2023
@kdmccormick kdmccormick changed the title Tutor Adoption: Devstack Parity: Dev Workflows: Upstream & Core fixes Tutor Adoption: Devstack Parity: Dev Workflows: Fix Upstream & Core Mar 19, 2023
@kdmccormick kdmccormick changed the title Tutor Adoption: Devstack Parity: Dev Workflows: Fix Upstream & Core Tutor Adoption: Devstack Parity: Dev Workflows: 1. Upstream & Core Mar 19, 2023
@kdmccormick kdmccormick changed the title Tutor Adoption: Devstack Parity: Dev Workflows: 1. Upstream & Core Tutor Adoption: Devstack Parity: Dev Workflows: Option 3 (Upstream) Mar 19, 2023
@kdmccormick kdmccormick changed the title Tutor Adoption: Devstack Parity: Dev Workflows: Option 3 (Upstream) Tutor Adoption 4: Devstack Parity: Dev Workflows Option 3: Upstream Mar 20, 2023
@regisb
Copy link

regisb commented Mar 20, 2023

Option 2 seems quite inconvenient. In my experience, developers are not keen on moving their repos. But I might be wrong.

I need to brainstorm this a little. FTR improving the dev workflow is also a top priority for me, and I agree that it's just a little too hard to hack on edx-platform as it is.

Would it make our lives easier if we moved /openedx/venv to /openedx/edx-platform/.venv? That way, an edx-platform's venv would be automatically bind-mounted every time we bind-mount edx-platform. I realize it would be a sort of step backward, as we are trying to move stuff away from edx-platform... Still, it's probably worth considering.

@kdmccormick
Copy link
Collaborator Author

Would it make our lives easier if we moved /openedx/venv to /openedx/edx-platform/.venv? That way, an edx-platform's venv would be automatically bind-mounted every time we bind-mount edx-platform. I realize it would be a sort of step backward, as we are trying to move stuff away from edx-platform... Still, it's probably worth considering.

Agreed, it is worth considering. It makes me realize that bind-mounted tutor dev is currently in an awkward middle ground. One very important thing lives in the image:

  • the Python venv

whereas the rest of it lives in the bind-mount:

  • the Python requirements list
  • package-lock.json
  • node_modules
  • compiled assets

Perhaps we should pick a side. I am interesting in prototyping the first one. If you prototype the second one, we could compare results.

Install/compile everything into the image

This is the direction I've been heading. Firstly, I would do either (1) or (2) from my previous comment.

Then, I would implement #150 and openedx/wg-devops#22, or some variation of those issues.

Install/compile everything into the bind-mount

We would move /openedx/venv to /openedx/edx-platform/.venv. We could pip-install Python requirements again when initializating a bind-mounted platform. This would probably work well enough for Linux.

Unfortunately, writing to bind-mounts on macOS is catastrophically slow, so asking those devs to run pip install (or, for that matter, npm install) into the bind-mount is just cruel. However, we could copy the node_modules and .venv directories down from the image to the host.

The current implementation of the copyfrom command, unfortunately, will not work well on macOS, because it's implemented using a bind-mount. I think it could be reimplemented using docker cp, though.

@bradenmacdonald
Copy link

Unfortunately, writing to bind-mounts on macOS is catastrophically slow, so asking those devs to run pip install (or, for that matter, npm install) into the bind-mount is just cruel. However, we could copy the node_modules and .venv directories down from the image to the host.

FWIW I measured this just now; once the pip cache is populated, pip install -r requirements/edx/base.txt takes 1m5s using a Docker volume but 4m15s using a bind mount (virtiofs). I imagine the platform would also run slower if python has to read all of the .pyc files out of a bind mount instead of just some.

@regisb
Copy link

regisb commented Mar 22, 2023

Let's try to consider this issue from the perspective of a macOS user, as it appears that it's the environment with the most constraints. What is the "right" way to run a containerized development environment in this context? Is there any hope at all, outside of running a native ./manage.py lms runserver on the host?

@kdmccormick
Copy link
Collaborator Author

The major macOS constraint I've become aware of is don't write anything significant to bind-mounts. The two solutions that leads me to are:

  • Use named volumes instead of bindmounts (ie, quickdev). This works well, but it introduces spooky background state into everyone's development environment.
  • Build as much as possible into the image (options 1 and 2 above). I haven't tried this yet, but I'm eager to.

FWIW, I do have a Mac to test on now.

@bradenmacdonald
Copy link

Yeah, using a virtual environment in the container is totally fine, as long as it's not bind-mounted. The problem is just that python doesn't have a simple/fast way to keep that venv in sync with the requirements files. Ideally there would be some readonly venv in the base image, and a writeable venv layer over top of it, and something that automatically rebuilds the writable venv delta whenever it no longer matches the requirements files. This could be as simple as a hacky script to make a hash of the requirements files and write that to the venv folder after each pip install, plus repeating that automatically when the hash doesn't match.

@regisb
Copy link

regisb commented Apr 7, 2023

I am tempted to say that optimizing bind-mounts for macOS should not be our job, but then it seems that there are no practical solutions proposed upstream: docker/for-mac#3677

So we don't really have a choice, right? I am still not in favour of named volumes, so the only option left is to build as much as possible into the image, such that the bind-mounted volumes remain as small as possible.

The good news is that I think we can achieve this without burdening the developers too much. I have a working proof of concept that works as follows:

  • At build-time, bind-mount the source repo in the image using a build-context. See example 3 from this article and this section of the Docker docs. Thanks to the --build-context option we should be able to implement a tutor images build --bind-mount=/path/to/edx-platform option.
  • Aggressively cache pip install and npm clean-install commands using RUN --mount=type=cache,... commands. In Olive my proof of concept is still slowed-down by edx-platform editable requirements but this will get better in Palm thanks to Kyle's efforts.
  • Persistent bind-mounts which allow developers to run tutor dev bind-mount /path/to/edx-platform once instead of adding the -m /path/to/edx-platform to every command. I haven't implemented this command yet, but it should not be too hard.
  • Move /openedx/edx-platform/node_modules to /openedx/node_modules and symlink to it (Kyle this is your option 2 above). I have just attempted to do this and it seems to be working as expected.

So I think we can attempt to resolve this issue using this approach.

Ideally there would be some readonly venv in the base image, and a writeable venv layer over top of it

This is a really interesting idea. I understand we could achieve that goal with a quick-and-dirty script, but I wonder if there is any "official" way to run a combination of multiple virtualenv?

@bradenmacdonald
Copy link

This is a really interesting idea. I understand we could achieve that goal with a quick-and-dirty script, but I wonder if there is any "official" way to run a combination of multiple virtualenv?

Doesn't docker do this for us already though, if the base venv layer is established during the build, then it becomes readonly and any changes made it the container will be done in a copy-on-write overlay of the venv, as long as it's not bind mounted.

@kdmccormick kdmccormick changed the title Tutor Adoption 4: Devstack Parity: Dev Workflows Option 3: Upstream Tutor Adoption 4: Devstack Parity: Improve Dev Workflows Sep 7, 2023
@kdmccormick
Copy link
Collaborator Author

We've made a lot of progress here, so I'm closing this, with two remaining follow-up issues:

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
epic Large unit of work, consisting of multiple tasks
Projects
None yet
Development

No branches or pull requests

3 participants