[IMPROVEMENT] Dev containers for PGM

[mgovers]

Describe the problem

Currently, it is quite difficult to set up the development environment for PGM. This creates an artificial barrier for new contributors. A solution could be to use development containers, which contain the PGM dependencies and build dependencies (e.g., compiler, CMake, Ninja, Python, ...)

Improvement proposal

• Step 1: Investigate what the optimal way to do things is. Potential starting points
  • Dev containers (please also investigate to what extend cross-IDE support is possible)
  • Raw docker containers (see e.g. https://github.com/mgovers/docker-test-repo for a test repo with the PGM build and development dependencies)
  • ... ?
• Step 2: Create a proposal and/or proof-of-concept based on the investigation. Discuss with PGM maintainers and choose a path going forward
• Step 3: Implement the chosen way

[mgovers]

c.c. @emabre this issue was created based on your input. Do you already have previous working experience with this? If so, can you please share things that may be useful?

The PGM maintainers will also do their own research (so that we at least make an informed decision), but any input will be useful.

The implementation part of this ticket is a potential good-first-issue.

[emabre]

Hi @mgovers thanks for opening this issue, it would be really cool to have container for developing PGM.
I am not an expert of Docker but I have some experience with it, and I previously managed to build PGM into a docker container.
I am going to sort out some notes of mine and share them here below. I'll try to do that this evening so you'll have the notes by tomorrow morning.

[TonyXiang8787]

Hi @mgovers, @emabre,

Thanks for bringing this up. A dev container can definitely help some developers to get onboard. I do have one comment: this should be an opt-in configuration. If any developer open IDE (like vscode) with clean cloned PGM repo, the default configuration should always be the native development. The IDE can optionally ask the developer to switch to the dev container. But dev container should not start as default.

[figueroa1395]

Maybe it is worth mentioning that since Docker is licensed, and is also the default backend for a dev container, it may be worth investigating something like Podman, that can also be used as a dev container backend and is fully compatible with Docker, albeit license free. Just brainstorming for whoever picks this up.

[emabre]

Some info that might be useful to consider when preparing the container setup:

• I see that here (https://github.com/mgovers/docker-test-repo) you have have setup a sample container based on homebrew. In my case, I did it from the image Ubuntu 24.04 image. I guess in your case the image is much lighter, if it works as expected it's certainly preferrable.
• the ability to run git commands like "commit" within the continer is obvioulsy necessary because that allows to run the git hooks where are all the dependencies are installed
• the ability to run git commands like "git pull" or "git push" from within the container is very desirable but not strictly necessary, in my opinion. These commands require the ssh machinery to be accessible within the container. In my case I did not do that setup, I just pulled/pushed from outside the container. In this setup, the permissions have to be managed carefully because both the host OS user and the docker user must be able to write to the directory where the PGM code is saved.
• Unfortunately, I did not write a Dockerfile when working on my container, so I have nothing well ordered to share that could be used as starting point. I just followed the quick and dirty approach consisting of shelling into the container, editing it with some commands and committing the existing container (docker commit) to save a new local image. I took some notes of the commands that I ran. But since they are a bit chaotic, I am not posting them here not to generate confusion (anyways, I did not run much stuff besides the basic instructions that are listed in the build guide). Nevertheless, I can share them upon request.
• I mounted the folder containing the PGM repo as a volume into the container by using the -v option (my command was docker run -it -v ~/myprogr/power-grid-model-dock:/home/power-grid-model ema/docker-builds-pgm /bin/bash)
• I don't know much of how a dev container differs from a raw container. What I can say is that in my case I used to keep my container up (-it flag) to build, run pytests and run the git hooks multiple times. So it was definitely not a throwaway container.
• In my case I also decided to limit the RAM that the container can use (using the option --memory=3000m, so the resulting command became docker run --memory=3000m -it -v ~/myprogr/power-grid-model-dock:/home/power-grid-model ema/docker-builds-pgm /bin/bash). I decided to do that after my PC froze while I was building PGM. I'm not totally sure that the issue came from the resources used by the container, but it's worth taking it into account in case anybody else experiences the same. I have 8Gb of RAM in total and the CPU is Intel i5-8265U, not a high performance environment.
• I think it could be a good idea of leveraging docker-compose to orchestrate the container, that would make it easier to specify and customize where to mount the volume with the PGM repo (if you choose to use a volume). It might also simplify some other setups, like the definition of env variables (like CC and CXX?) that the user could easily customize just by editing the compose file. If the container and the setup has to be compatible with other containerizaion systems, then it's worth investigationg to what extent Podman, Rancher and other docker alternative support the docker-compose functionality, I never tried any container runtime other than Docker.
• it is a good idea not to forget to install some basic file editing tooling (like vim) inside the container, so that the user would have a last resort resource to instpect and fix issues that might occur (I did it in my container).

In case you have any further questions, feel free to ask :-)

[figueroa1395]

This issue was completed by #1161, #1191, and #1206.

• Dev Environment: setup and the documentation for using the container #1161 set the foundations for the dev container.
• Dev environment: add support and documentation for Docker #1191 added docker support for the dev container.
• Enable UV in build and CI #1206 includes the use of uv for the dev container setup.

The documentation for setting up a dev container can be found in https://power-grid-model.readthedocs.io/en/stable/working_with_containers/container-setup.html.

C.C. @emabre I thought you may want to give it a try given your input in this topic.