Share code between features

[AlexanderLanin]

Motivation

While devcontainers features aim to enable modern software development, the feature code itself is stuck in a lot of code duplication.
For example we can find # Determine the appropriate non-root user 17 times within the features repo. It's the same for me for detecting python version etc etc.

One of the main principles in software development is DRY: Don't Repeat Yourself. That is currently not possible for features!

Proposal

Variant: generic pre-build

Add support for pre-build.sh (in src/_pre_build.sh, src/_pre_build/pre_build.sh, src/_special_cli_stuff/pre_build.sh) which would be executed upon build and feature test calls with an appropriate set of parameters. This way the script can do arbitrary things like copy common code. But it could do even more like downloading common code from yet another source. It could merge code into single files. etc.

Variant: common code

Add support for src/_common which could contain code that is used throughout my features. Build and feature test calls will package that code as well as the install.sh script of the feature. Within install.sh one can simply source/call anything from _common as it's part of the package.

P.S. I wasn't sure whether to put this to cli or spec, please move if incorrect.

[chrmarti]

Or: The devcontainer-feature.json could list relative paths of files and folders that should be copied to the tar file during packaging.

It could be nice if the file structure in the tar file was the same as in the source repository, but currently the devcontainer-feature.json has to be top-level in the tar. Maybe something to think about.

[Chuxel]

I like this suggestion since I think even in devcontainers/features there are common utilities we could have in a utils.sh file that is sourced in install.sh. Right now I think we'd need to setup a CI job to copy files around given how much code is copied between them right now.

FYI on non-root user, #91 is in the CLI and in flight getting released which would allow you to just use a _REMOTE_USER env var. But, there's many other utility functions beyond this one in that repository.

[AlexanderLanin]

19 days have passed and I have a new view on that. That's how fast the software world changes ;-)

Current setup unintentionally allows a powerful reuse of those scripts we have, for example:
curl -s https://raw.githubusercontent.com/devcontainers/features/main/src/python/install.sh | sudo VERSION=3.7 bash

Of course there are a bunch of such install scripts, but generally speaking those that are used for devcontainers are well tested due to ease of testing in different environments.

[joshspicer]

+1 to this. Below is just some spitballing, but an idea i've been tossing around is the concept of library Features. They cannot be referenced directly in a devcontainer.json, but rather a Feature will declare that it depends on a library.

{
   "id": "ruby":,
   "version": "1.2.3", 
    "libraries": [
      "ghcr.io/devcontainers/features/lib:1"
    ]
}

The CLI would then automatically fetch ghcr.io/devcontainers/features/lib:1 and source the contents into the executing install.sh script. These libs would be cached as any other OCI artifact, but allows an author to quickly update "helper functions" for their entire suite of Features.

For the devcontainers maintainers team, we would publish and maintain a ghcr.io/devcontainers/features/lib library Feature that anyone could use. For our own Features in devcontainers/features, it would reduce the size of each install.sh significantly and reduce the copy/paste errors and out-of-sync errors we occasionally see.

[jcbhmr]

Also discussion in https://github.com/orgs/devcontainers/discussions/10 and https://github.com/devcontainers-contrib/features/discussions/83

[phorcys420]

Current setup unintentionally allows a powerful reuse of those scripts we have, for example: curl -s https://raw.githubusercontent.com/devcontainers/features/main/src/python/install.sh | sudo VERSION=3.7 bash

Of course there are a bunch of such install scripts, but generally speaking those that are used for devcontainers are well tested due to ease of testing in different environments.

The issue with this approach is that you're loading an unversioned script file in a versioned feature.
Let's say you write your feature depending on some utility and that the function you're using is removed from said utility, then it'll just break unexpectedly.

I think a way you can counter this is by pinning the version using a commit hash, but that also makes it a hassle for maintainers.

[phorcys420]

+1 to this. Below is just some spitballing, but an idea i've been tossing around is the concept of library Features. They cannot be referenced directly in a devcontainer.json, but rather a Feature will declare that it depends on a library.

{
   "id": "ruby":,
   "version": "1.2.3", 
    "libraries": [
      "ghcr.io/devcontainers/features/lib:1"
    ]
}

The CLI would then automatically fetch ghcr.io/devcontainers/features/lib:1 and source the contents into the executing install.sh script. These libs would be cached as any other OCI artifact, but allows an author to quickly update "helper functions" for their entire suite of Features.

For the devcontainers maintainers team, we would publish and maintain a ghcr.io/devcontainers/features/lib library Feature that anyone could use. For our own Features in devcontainers/features, it would reduce the size of each install.sh significantly and reduce the copy/paste errors and out-of-sync errors we occasionally see.

I really like this idea and I'm currently experimenting with this using the dependsOn property as a workaround while this is not available OOTB and loading a feature that only serves purpose as a library (it just copies files to /usr/share/phorcys420-devcontainer-features/).

The issue I'm currently facing is the exact same one I mentioned in my other comment: how do we make sure we allow different library versions to coexist without breaking stuff ?

---

I've come up with the following way to avoid this problem :

• Copy features over to /usr/share/devcontainer-library/v1.0.1 (where v1.0.1 is the current version number)
• Symlink /usr/share/devcontainer-library/v1.0.1 to
  • /usr/share/devcontainer-library/v1 (where v1 is the major version)
  • /usr/share/devcontainer-library/current

That way you can simply depend on a major version but if you need to depend on a specific minor one then the different minor versions won't interfere with eachother (i.e if they used the same folder, they could overwrite eachother).

The issue I currently have though is that the library feature should know it's own version from within the install.sh script and that doesn't seem to be the case.
I'd have to hardcode the version number a second time in the script in the devcontainer-feature.json file, but then that's pretty error-prone.

---

EDIT: So whilst the environment variables available in the environment don't tell you what version is currently being installed, the devcontainer-feature.json file is accessible from within the install script, so I've found the following workaround:

VERSION=$(jq -r ".version" devcontainer-feature.json)

It's a bit hacky and does require an external dependency (i need jq in all my features at the moment anyways) but at least you don't need to repeat yourself.