Add support for alternate definition configs under .devcontainer

[2percentsilk]

As part of the monorepo support for Codespaces, we want users to be able to choose a devcontainer specific for their scenario from a list of devcontainer definitions contained with the .devcontainer folder.
For example, they could have .devcontainer/actions/devcontainer.json as well as .devcontainer/vscode/devcontainer.json and providing either path would work and successfully create the dev container specified at that path.

As an example, we'd expect for any of the follow paths to devcontainer definitions to be acceptable:

repository_root
 | .devcontainer.json
 |
 | .devcontainer
   |
   +-- foo
   |  |  
   |  +-- devcontainer.json
   |  \-- Dockerfile
   | 
   +-- foo
   |  |  
   |  +-- devcontainer.json 

As a stretch, it would be great if the definition could be 2+ folders down such that .devcontainer/actions/allisb/devcontainer.json would also work but one subdirectory is sufficient for this first stage.

[2percentsilk]

@bamurtaugh @Chuxel for thoughts/FYI :)

[Chuxel]

@chrmarti @bamurtaugh @joshspicer Yeah this fits pretty well with other requests we've seen to allow people to select in Remote - Containers. If detected, we prompt for which one to use there and it would be simple for unique configurations like you'd see in a monorepo.

This is also a separate and distinct problem from multi-container attach, which needs a different model. This can help with scenarios like microsoft/vscode-remote-release#6188 too (via a common docker-compose.yml).

The idea of overrides / per-user config also could fit here given we've also talked about a possible parent property to allow for small variations in otherwise common config. One devcontainer.json could point to another in this same nested structure. That said, interestingly, using a local "feature" could allow you to share common content across the different devcontainer.json might be enough to get things going with just nested devcontainer.json files.

So riffing of the example above:

repository_root
 | .devcontainer
   |
   +-- common-features
   |  |  
   |  +-- features.json
   |  \-- install.sh   
   |
   +-- foo1
   |  |  
   |  +-- devcontainer.json
   | 
   +-- foo2
   |  |  
   |  +-- devcontainer.json 

The devcontainer.json file could then contain:

"image": "foo1",
"features": { "../common-features#feature": "latest" }

In fact, this could help some definitions. We've seen PRs both that involve connecting to a device (like Flutter), and using a GPU (like Anaconda or CUDA) that can take advantage of a native device in some cases, but in others needs a different config to stay emulated or on the CPU. These definitions and repos can then have mostly the same config with a slight variation and a local path to a feature.

[bamurtaugh]

What both of you describe makes a lot of sense.

The devcontainer.json file could then contain:

"image": "foo1",
"features": { "../common-features#feature": "latest" }

In the example above @Chuxel, is the goal that either of the devcontainer.json's in foo1 or foo2 could contain this snippet, and users would want to create a set of features common to all /several of their definitions in the common-features folder?

[Chuxel]

@bamurtaugh Yep. You can do this with install scripts to some extent with a custom Dockerfile and a different context, but its clunky, and doesn't allow for extension install, etc. Features would resolve that.

[Chuxel]

Related: microsoft/vscode-remote-release#1165, microsoft/vscode-remote-release#2413

[chrmarti]

I wonder: Editors and IDEs tend to share project settings using a file or folder in the project folder (.vimrc, .sublime-project, .idea, .vscode). Aren't monorepos using one such settings file/folder per sub-project? Would it make sense to align with this and use one .devcontainer folder per sub-project in the monorepo? E.g., <root>/foo/.devcontainer/devcontainer.json and <root>/bar/.devcontainer/devcontainer.json for sub-projects foo and bar.

[bamurtaugh]

@chrmarti Would that look like any (or multiple) of the following?

Option one (supporting .devcontainer.json and .devcontainer in the root, just requiring .devcontainer for sub-projects. This would lead to nested .devcontainer folders):

repository_root
 | .devcontainer.json
 |
 | .devcontainer
   |
   +-- foo
   |  |  
   | .devcontainer
      |  +-- devcontainer.json
      |  \-- Dockerfile
   | 
   +-- foo
   |  |  
   | .devcontainer
      |  +-- devcontainer.json 

Two (support .devcontainer in the root, but it wouldn't have nested projects / devcontainer folders and instead just a root-level devcontainer.json and Dockerfile):

repository_root
 | .devcontainer.json
 |
 | .devcontainer
    |  +-- devcontainer.json
    |  \-- Dockerfile
 |
 +-- foo
 |  |  
 | .devcontainer
    |  +-- devcontainer.json
    |  \-- Dockerfile
 | 
 +-- foo
 |  |  
 | .devcontainer
    |  +-- devcontainer.json 

Three (don't use .devcontainer at the root, instead requiring each sub-project to provide .devcontainer.json, and still support a root .devcontainer.json):

repository_root
 | .devcontainer.json
   |
   +-- foo
   |  |  
   | .devcontainer
      |  +-- devcontainer.json
      |  \-- Dockerfile
   | 
   +-- foo
   |  |  
   | .devcontainer
      |  +-- devcontainer.json 

Four (no dev container files/folders supported at the root):

repository_root
   |
   +-- foo
   |  |  
   | .devcontainer
      |  +-- devcontainer.json
      |  \-- Dockerfile
   | 
   +-- foo
   |  |  
   | .devcontainer
      |  +-- devcontainer.json 

I think my main questions are:

• Would we require a .devcontainer folder in each sub-project, and remove .devcontainer from the root?
• Does this affect what the user can include in the root? i.e. still allow .devcontainer.json, still allow or require .devcontainer which could include:
  • a root-level devcontainer.json and Dockerfile, or
  • Nested .devcontainer folders

[Chuxel]

I wonder: Editors and IDEs tend to share project settings using a file or folder in the project folder (.vimrc, .sublime-project, .idea, .vscode). Aren't monorepos using one such settings file/folder per sub-project? Would it make sense to align with this and use one .devcontainer folder per sub-project in the monorepo? E.g., <root>/foo/.devcontainer/devcontainer.json and <root>/bar/.devcontainer/devcontainer.json for sub-projects foo and bar.

@chrmarti @bamurtaugh Yes and no. Sometimes that's the case - but a monorepo can also be for a large single "application". Definitely heard feedback about role specific config in these cases - e.g. a front end developer verses back end, core services verses web app, etc. In these cases, it's the same abstract "project", but it involves shared libraries or services in the same repo, and different people working on it with different focuses. At scale, you end up needing multiple "views" on it - e.g. there's multiple .code-workspace files / solutions, etc on the same file tree.

In this case, each person's local machine is setup differently based on the role they play in developing the application. You want that same thing to happen when opening something in a dev container - its setup should be role specific for the same application. Right now, there's no good way to do this with devcontainer.json (without ugly hacks).

If the monorepo is the giant kind that actually crosses multiple applications, then yes, tool specific config would be under a sub-folder, but that's not what this is trying to solve. You can already have sub-folders with separate devcontainer.json folders - nothing in the spec prevents that.

The ask here is to deal with the scenario that is not well supported today that the Codespaces team has been hearing about frequently.

[bamurtaugh]

From my understanding, the main new features here is: VS Code searches for dev container definitions in root and sub-folders. Once VS Code detects all of the sub-definitions, it allows users to pick one when opening their project in a dev container.

Apologies if I'm missing this, but would this ask be just part of / one type of monorepo support? Here users will nest their sub-container definitions in one main .devcontainer folder at the root since this type of monorepo involves folks all on the same large repo with different roles. If users have dev containers spread into their sub-projects, rather than all in one central .devcontainer, that'd be what @chrmarti was suggesting and @Chuxel described as:

If the monorepo is the giant kind that actually crosses multiple applications, then yes, tool specific config would be under a sub-folder, but that's not what this is trying to solve

[Chuxel]

Apologies if I'm missing this, but would this ask be just part of / one type of monorepo support?

@bamurtaugh Yep, there's two types, one of which we already have covered in the spec even if just partly covered by the products.

1. Supported now by spec: Several independent "projects" in a single large repository. This is common with source control systems like Subversion, and in some cases orgs follow this practices in Git as well, but less commonly. Per the spec, you could run a "dev container CLI" in a sub-folder in the repository that itself has a .devcontainer folder in it. In this case, we still automatically mount the git repository into the container for image/Dockerfile, and users can control the mount in the Docker Compose case per the spec. So in concept, it works. (That said, Codespaces has product gaps here, and so does Remote - Containers for opening a repo in a container volume, but that's not a spec problem.)

2. Gap in spec: Several "views" on the same repository contents based on project/developer role or focus, rather than a single sub-folder. In this case, you likely need multiple sub-folders in the monorepo to build and develop the application. However, different roles have different focuses, extensions, settings, runtimes, etc. There's no single sub-folder to anchor to in determining what to include/not include. The request is to allow multiple variations under the root .devcontainer folder. Users that open the root folder can then make a choice as to which one they want to use, but still have access to the entire source tree as-needed. VS Code Workspace files and other tool's equivalents then allow you to filter down if desired. This particular style is pretty common in more recent cloud native/digital native application designs.

The "workspaceMount" property requires an absolute path for the "source", so you can't easily identify the source tree as being up a level, and even if you could its klunky. Of the two types, the Codespaces team is hearing more about the second case so far - which is why they raised the ask.

[Chuxel]

@chrmarti @bamurtaugh Given this has been picked up for implementation in what will become the reference implementation, I moved this one to "finalization." LMK if you disagree.

[Chuxel]

Please stop discussing this here - it is off topic for the specific issue. This sounds like the remote extension's attach functionality which is not related to this issue and will continue to exist. There's no pod or any other infrastructure definition in these configuration files. With the open sourcing of a CLI (#9), other infrastructure definitions can be referenced should communities want to contribute. What you describe is how things work already - the local filesystem does not have to be used and you can connect to a SSH host and work with containers from there, or you can just keep using attach. Both are fine. Other places to discuss could be #20 or #10 since I think things are already headed in the direction you describe.

[Chuxel]

Codespaces has gone ahead and implemented this capability (https://github.blog/2022-04-20-codespaces-multi-repository-monorepo-scenarios/). So, it's effectively part of the spec, but #9 will include it once released and Remote - Containers has not yet added support for making the selection via UX. I think we can close this out once #9 is released. @bamurtaugh @edgonmsft I know you folks are working on docs - I don't think this support is in the current PRs, correct?

[bamurtaugh]

Thanks for the tag, Chuck!

I don't believe so - is there a spot we're envisioning for it to be highlighted in spec materials? Perhaps in https://github.com/devcontainers/spec (@edgonmsft is working on other info for this repo)? I can add it to Remote-Containers docs too whenever it may be supported via the UI.

[dragon512]

I just wanted to chime in. This feature is what I think I have been looking for. I work on many different open-source projects for my job and it would be of great value to define different image configurations to load easily. Behavior for tools I work on has different behaviors for various reasons that are subtle when running in let's say alma8 vs ubuntu, vs Fedora. It would be of great value for the community to be able to define a set of supported operating environments in the source project so anyone in the community working on that project can easily test the difference. The current limitation of only one environment is difficult to work with.

[chrmarti]

This is part of the spec, implemented in the reference implementation and supported by GitHub Codespaces and the Dev Containers extension for VS Code.