Auto-generate configuration for Visual Studio Code's Remote-Containers extension

[lbssousa]

Context

Following discussions in microsoft/vscode-remote-release#3345, I would like to suggest that toolbox binary receives a feature to auto-generate all the configuration needed for VSCode's Remote-Containers extension to allow attaching to a given Toolbox container, which I'll name ${TOOLBOX_CONTAINER_NAME} from now on.

Instructions

The needed steps, based on my personal use case, could be the following (I'll suppose here that toolbox would support VSCode as installed from official RPM/DEB repository, but I'll also give alternatives for an eventual support for VSCode as installed from Flathub):

1. Create a folder ${HOME}/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/nameConfigs, if it doesn't exist. I'll refer to it as ${VSCODE_ATTACHED_CONFIG_DIR} from now on.
   • If VSCode was installed from Flathub, the correct path above is ${HOME}/.var/app/com.visualstudio.code/config/Code/User/globalStorage/ms-vscode-remote.remote-containers/nameConfigs
2. If VSCode was installed from Flathub, it's strongly recommended to inject the following variables to its environment:

flatpak --user --override --env HOST_DISPLAY="$DISPLAY" --env HOST_SHELL="$SHELL" --env HOST_SSH_AUTH_SOCK="$SSH_AUTH_SOCK" com.visualsudio.code

3. Create/Update file ${VSCODE_ATTACHED_CONFIG_DIR}/${TOOLBOX_CONTAINER_NAME}.json with the following contents:

{
  // Support requested in https://github.com/microsoft/vscode-remote-release/issues/4053.
  // Replace ${TOOLBOX_CONTAINER_NAME} with any other name
  // you want to be displayed in VSCode's status bar
  // "name": "${TOOLBOX_CONTAINER_NAME}",
  "remoteUser": "${localEnv:USER}",
  "settings": {
    "remote.containers.copyGitConfig": false,
    "remote.containers.gitCredentialHelperConfigLocation": "none",
    "terminal.integrated.profiles.linux": {
      "capsh": {
        "path": "/usr/sbin/capsh",
        "args": [
          "--caps=",
          "--",
          "-c",
          "exec \"$@\"",
          "/bin/sh",
          "${localEnv:SHELL}",
          "-l"
        ]
      }
    },
    "terminal.integrated.defaultProfile.linux": "capsh"
  },
  "remoteEnv": {
    "COLORTERM": "${localEnv:COLORTERM}",
    "DBUS_SESSION_BUS_ADDRESS": "${localEnv:DBUS_SESSION_BUS_ADDRESS}",
    "DESKTOP_SESSION": "${localEnv:DESKTOP_SESSION}",
    "DISPLAY": "${localEnv:DISPLAY}", // or "${localEnv:HOST_DISPLAY}", If using VSCode from Flathub (see step 2 above)
    "LANG": "${localEnv:LANG}",
    "SHELL": "${localEnv:SHELL}", // or "${localEnv:HOST_SHELL}", If using VSCode from Flathub (see step 2 above)
    "SSH_AUTH_SOCK": "${localEnv:SSH_AUTH_SOCK}", // or "${localEnv:HOST_SSH_AUTH_SOCK}", If using VSCode from Flathub (see step 2 above)
    "TERM": "${localEnv:TERM}",
    "VTE_VERSION": "${localEnv:VTE_VERSION}",
    "XDG_CURRENT_DESKTOP": "${localEnv:XDG_CURRENT_DESKTOP}",
    "XDG_DATA_DIRS": "${localEnv:XDG_DATA_DIRS}",
    "XDG_MENU_PREFIX": "${localEnv:XDG_MENU_PREFIX}",
    "XDG_RUNTIME_DIR": "${localEnv:XDG_RUNTIME_DIR}",
    "XDG_SESSION_DESKTOP": "${localEnv:XDG_SESSION_DESKTOP}",
    "XDG_SESSION_TYPE": "${localEnv:XDG_SESSION_TYPE}"
  }
}

3. Give read/write permissions to /root folder within the container for the current user (or, at least, give read permissions to /root and create a folder /root/.vscode-server with read/write permissions).

4. For containers created with Toolbox prior to version 0.0.97, run the following command before starting VSCode:

toolbox -c ${TOOLBOX_CONTAINER_NAME} run true

5. Additional steps are needed to make Flatpak'ed VSCode spawn podman from host system.

Additional notes

• Properties terminal.integrated.shell.linux and terminal.integrated.shellArgs.linux may need to be reviewed to match toolbox enter ${TOOLBOX_CONTAINER_NAME} behaviour.

• There are still some discussion in Support Toolbox (a.k.a. Fedora Toolbox) microsoft/vscode-remote-release#3345 about the fact that .vscode-server is being installed under /root folder within the container, rather than remoteUser's home folder. I'm personally OK with the current behaviour, because I can keep .vscode-server folder within the container, but it may not be the desired behaviour for the others.

Supersedes: #610

[naipotato]

3. Give read/write permissions to /root folder within the container for the current user (or, at least, give read permissions to /root and create a folder /root/.vscode-server with read/write permissions).

How? I'm failing to do that

[lbssousa]

You can do either

sudo chmod 777 /root

or

sudo chmod 755 /root
sudo mkdir -m0777 /root/.vscode-server

[naipotato]

@lbssousa Uhm, that works... I wonder then what I've misinterpreted 🤔️

Thanks

[spikhoff]

@lbssousa I found your instructions to be very useful.

I'm wondering how could we get rid of this root permission issue, but this probably needs to be solved in microsoft/vscode-remote-release side.

Also I think toolbox should not generate vscode specific configuration, instead maybe it would be better to have vscode extension for that specific use-case (at least until vscode-remote starts supporting toolbox natively).

[lbssousa]

@spikhoff I personally don't believe Microsoft guys will make a Remote Development extension, or just extend Remote Containers one, just to support Toolbox out of box (and Remote Development extensions aren't open source, so we hardly could write our own extension for Toolbox). If attachment to a Toolbox container works without need of touching Remote Containers extension, there's nothing to to on their side (except maybe microsoft/vscode-remote-release#4013 or microsoft/vscode-remote-release#4584). Remember that they only added support to Podman because its CLI supports the same arguments as Docker CLI.

[spikhoff]

When I restart my system I get permission error, vscode-server tries to install under /root.
"Attach to Container" starts working again if I create a new toolbox.

toolbox rm --force fedora-toolbox-33 && toolbox enter

Edit: Also restarting pod triggers this bug.

podman restart fedora-toolbox-33 

[lbssousa]

@spikhoff please put here the output of the following commands:

podman stop fedora-toolbox-33
toolbox enter
sudo chmod 777 /root
ls -l /

Then, restart your system and put the output of the following commands:

toolbox enter
ls -l /

[spikhoff]

@lbssousa, Thanks, that indeed works!
I was just curious and tried to understand why it fails...

[lbssousa]

Aren't you trying to change your host system's /root folder permissions, rather than the one within the container?

[spikhoff]

No, but I was bit worried what happens when I change permissions inside container, would they propagate to hosts filesystem or not. So I tested and they don't.

I did not know that :)

[owtaylor]

I was just experimenting with VSCode Remote Containers + Toolbox some, and came up with a similar configuration to that here. (Should have done some searching first...) Initial impressions with that, and with trying these exact instructions for a different toolbox are that it works quite well. Some imperfections I notice

• We should see if it's possible for the VSCode Flatpak to ship /app/bin/podman /app/bin/docker scripts - I don't think there's anything that you'd want those to do in the sandbox except for flatpak-spawn --host podman "$@"
• The step of starting up a VSCode and using "attach to running container" would need documentation.
• The recent list retains the combination of the workspace and the container used to run it, which is nice, but not fully as good as being able to bind the container to the workspace
  • The container name is not always displayed in the list - often it’s just [Container]
  • If you don’t use the recent list, it would be easy to open the workspace in the wrong container
• The titlebar shows up as, e.g, manage.py - fm-orchestrator [Container registry.fedoraproject.org/f34/fedora-toolbox:34 (F34)] - Visual Studio Code - would ideally be manager.py - fm-orchestrator [Toolbox F34] - Visual Studio Code

---

Would this be better done as some sort of "plugin" where you could drop a script into a directory to be run at container creation time, and avoiding hardcoding details of closed-source software into the toolbox core?. But on the other hand, that would require you to have that installed before you created your toolbox, or delete and recreate your toolboxes. Or a drop-in 'toolbox code' subcommand so that 'toolbox code .' sets things up and launches VSCode on the current folder in the current directory, in the style of 'code .' in WSL.

---

I'm wondering how could we get rid of this root permission issue, but this probably needs to be solved in microsoft/vscode-remote-release side.

If you follow through the comments on microsoft/vscode-remote-release#3345, the /root location is coming from HOME 'podman inspect '. Since toolbox is happy with that being /root, it would presumably be happy with that being any other location, including the users $HOME or a directory under that.

That being said, I agree with @lbssousa 's conclusion that writing things in /root/.vscode-server is possibly a feature, not a bug. the vscode remote extensionsexpects to set up a vscode server individually in each container, and I'm not sure what would happen if multiple containers shared the same ,vscode-server directory - probably nothing good.

---

It would be worthwhile to compare the Remote-WSL experience to the best we can achieve with Remote-Containers + Toolbox today and keep an issue open for https://github.com/microsoft/vscode-remote-release for what is needed to get to feature-parity. Without actually trying out Remote-WSL, it seems like it's mostly cosmetic UI issues:

• Suppress the image name from the titlebar and recent projects list
• Have "New Toolbox window" "Reopen in Toolbox" commands

There's likely also optimizations that could be done using the shared-$HOME nature of Toolbox, but that's a bigger ask, and may simply not matter.

[lbssousa]

• The titlebar shows up as, e.g, manage.py - fm-orchestrator [Container registry.fedoraproject.org/f34/fedora-toolbox:34 (F34)] - Visual Studio Code - would ideally be manager.py - fm-orchestrator [Toolbox F34] - Visual Studio Code

There's already an issue open for this: microsoft/vscode-remote-release#4053

[lbssousa]

After experimenting VSCode unofficial Flathub package with Toolbox for a few months, I've observed that the following environment variables may need to be set explictly, because they're not inherited from system host, but are overriden by Flatpak sandbox instead:

• DISPLAY
• SHELL
• SSH_AUTH_SOCK

UPDATE 2021-03-16: As a workaround, we can inject these values as extra variables to Flatpak's environment:

flatpak --user --override --env HOST_DISPLAY="$DISPLAY" --env HOST_SHELL="$SHELL" --env HOST_SSH_AUTH_SOCK="$SSH_AUTH_SOCK" com.visualsudio.code

and then use expansions ${localEnv:HOST_DISPLAY}, ${localEnv:HOST_SHELL}, and ${localEnv:HOST_SSH_AUTH_SOCK} in attached-container configuration file.

[chergert]

We already handle all this natively in GNOME Builder, so I guess the question is if you think it makes sense for us to switch to using this too?

[lbssousa]

We already handle all this natively in GNOME Builder, so I guess the question is if you think it makes sense for us to switch to using this too?

Not sure if it fits your case. We're discussing here about what's needed to make Visual Studio Code (installed via RPM or Flatpak package) able to attach to Toolbox containers through Remote-Containers extension + podman container runtime.

[owtaylor]

I created a repository: owtaylor/toolbox-vscode - what this contains currently is an implementation of a code command line script that a) sets up configuration for integration b) launches Visual Studio Code with an appropriate --remote parameter. It seems to work pretty well for me, but it is complex and feedback would be appreciated.

Currently, this can be dropped somewhere on your path like ~/.local/bin/code - I could imagine that toolbox could grow a facility where (say) /usr/share/toolbox/bin was in the $PATH for all toolboxes.

The upside of it is that it looks simple if you know to start from the command line. The disadvantage is that you have to start that way - there's no reasonable path through the UI to configure a new toolbox container. (If you are already connected to a container, opening a new folder in it works fine.)

[lbssousa]

@owtaylor , I've updated the OP of this issue with additional notes to simplify attached-container configuration when VSCode is installed from Flathub. You may want to apply it to your project.

[owtaylor]

@owtaylor , I've updated the OP of this issue with additional notes to simplify attached-container configuration when VSCode is installed from Flathub. You may want to apply it to your project.

If I understand, you are referring to using flapak override --env HOST_DISPLAY="$DISPLAY, and so forth. There seems to be two possibilities:

• We hardcode values in the attached-container configuration file
• We hardcode values in the Flatpak override settings

What do you see as the advantages of the second approach?

(An approach to getting actual dynamic values occurs to me: we could manipulate things in the run-podman-on-host wrapper script so that ${containerEnv:HOST_DISPLAY} has the right value. I'm not sure if that involves munging the return value of 'podman inspect' or adding --env options to podman-exec. Would be hacky either way.)

[owtaylor]

Tested to work if I entirely omit DISPLAY/SHELL/SSH_AUTH_SOCKET from remoteEnv and then use this podman-host wrapper script:

#!/bin/bash

if [ $1 == "exec" ] ; then
    shift
    env_args=()
    while read env ; do
        env_args+=("-e" "$env")
    done < <(flatpak-spawn --host sh -c 'cat /proc/`pgrep -u $UID -f ^/usr/libexec/flatpak-session-helper`/environ' | \
                 grep -z -E '^(DISPLAY|SHELL|SSH_AUTH_SOCK)=' | tr '\0' '\n')
    exec flatpak-spawn --host podman exec "${env_args[@]}" "$@"
else
    exec flatpak-spawn --host podman "$@"
fi

Have to think if that's a good idea or not ... :-/ [also affects all podman/docker invocations not just ones targeting a toolbox]

[owtaylor]

Much simplified version of the above:

#!/bin/bash                                                                                                                                                                                                        

if [ $1 == "exec" ] ; then
    shift
    exec flatpak-spawn --host sh -c \
         'exec podman exec -e DISPLAY="$DISPLAY" -e SHELL="$SHELL" -e SSH_AUTH_SOCK="$SSH_AUTH_SOCK" "$@"' \
         - "$@"
else
    exec flatpak-spawn --host podman "$@"
fi

[lbssousa]

@owtaylor I've tested your latest podman-host wrapper script (the simpler one) and it worked as expected for me, provided I remove DISPLAY, SHELL, and SSH_AUTH_SOCK from "remoteEnv" section in attached-container configuration file. The only drawback is that "terminal.integrated.shellArgs.linux" arguments will still replace "{localEnv:SHELL}" with /bin/sh, but it's not a big problem. Maybe your code-wrapper script can handle this.

[owtaylor]

The only drawback is that "terminal.integrated.shellArgs.linux" arguments will still replace "{localEnv:SHELL}" with /bin/sh

Good catch! I had that modified locally and didn't notice. What if you change "terminal.integrated.shellArgs.linux" to:

["--caps=", "--", "-c", "exec \"$SHELL\" -l"]

That seems to work for me. We are already running an intermediate bash process via capsh and have $SHELL set correctly, so why not use it?

[owtaylor]

That being said, I agree with @lbssousa 's conclusion that writing things in /root/.vscode-server is possibly a feature, not a bug. the vscode remote extensions_expects_ to set up a vscode server individually in each container, and I'm not sure what would happen if multiple containers shared the same ,vscode-server directory - probably nothing good.

Unfortunately, it seems there is instability here, and depending on the version of podman used to create the container, the container environment might end up with HOME as /root, /var/home/<user>, or /. So we'll have to do some tricky dancing (and maybe make toolbox force whatever value we think is best.)

[lbssousa]

Hmmm. I've just noticed that, too. Running podman inspect on both and old container (built with Toolbox 0.0.97 and Podman 2.x) and a new (built with version 0.0.99.1 and Podman 3.0.1), I've noticed that HOME variable in new container's environment now points to my home folder rather than /root folder. Not sure if this behaviour change is due to new Podman or new Toolbox version. @debarshiray , any thoughts?

[owtaylor]

I put an attempt at dealing with the diversity in: owtaylor/toolbox-vscode#2 - @lbssousa if you want to take a look and see if it makes sense to you, that would be appreciated!