espressif/idf dockerfile writes to host filesystem as wrong owner (IDFGH-13794)

[tobiashagge]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

5.3.1

Operating System used.

Windows

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

None

What is the expected behavior?

Commands in example project containers derived from espressif/idf should write files as the 'esp' user.

What is the actual behavior?

Files are written as 'ubuntu' user, even though whoami shows the current user as 'esp'. idf.py throws many permission denied errors.

Steps to reproduce.

Install WSL, VSCcode, C++ build tools, and ESP-IDF as per the directions. Create a copy of hello_world example project (also tested with blink) using the VS code interface. Launch the dev container.

whoami
touch thisisatest
ls -l

Build or installation Logs.

The issue occurs prior to build.

More Information.

The cause of the issue is explained at this link.; as of 24.04 the ubuntu image sets up a default user. Possible solutions are to downgrade the ubuntu base image, reconfigure the image to run as the ubuntu user, or delete the ubuntu user (as described in the link) in the child image. See this issue; the workaround there shows that downgrading to ubuntu 22.04 would resolve the issue.

[fhrbata]

Hello @tobiashagge , thank you for bringing this to our attention. Isn't it possible to fix/adjust this on the VSC devcontainer side? I believe, if you delete the ubuntu user there, it should work. IIUC the VSC extension is replacing the docker user UID in the /etc/passwd file with the host user UID after the container is created, so the mounted workspace is available without permissions problems to the docker user. This can be seen in the devcontainer build log.

Step 8/10 : RUN eval $(sed -n "s/${REMOTE_USER}:[^:]*:\([^:]*\):\([^:]*\):[^:]*:\([^:]*\).*/OLD_UID=\1;OLD_GID=\2;HOME_FOLDER=\3/p" /etc/passwd);       eval $(sed -n "s/\([^:]*\):[^:]*:${NEW_UID}:.*/EXISTING_USER=\1/p" /etc/passwd);       eval $(sed -n "s/\([^:]*\):[^:]*:${NEW_GID}:.*/EXISTING_GROUP=\1/p" /etc/group);       if [ -z "$OLD_UID" ]; then              echo "Remote user not found in /etc/passwd ($REMOTE_USER).";    elif [ "$OLD_UID" = "$NEW_UID" -a "$OLD_GID" = "$NEW_GID" ]; then              echo "UIDs and GIDs are the same ($NEW_UID:$NEW_GID).";         elif [ "$OLD_UID" != "$NEW_UID" -a -n "$EXISTING_USER" ]; then echo "User with UID exists ($EXISTING_USER=$NEW_UID).";         else            if [ "$OLD_GID" != "$NEW_GID" -a -n "$EXISTING_GROUP" ]; then                 echo "Group with GID exists ($EXISTING_GROUP=$NEW_GID).";                        NEW_GID="$OLD_GID";             fi;             echo "Updating UID:GID from $OLD_UID:$OLD_GID to $NEW_UID:$NEW_GID.";          sed -i -e "s/\(${REMOTE_USER}:[^:]*:\)[^:]*:[^:]*/\1${NEW_UID}:${NEW_GID}/" /etc/passwd;                if [ "$OLD_GID" != "$NEW_GID" ]; then                  sed -i -e "s/\([^:]*:[^:]*:\)${OLD_GID}:/\1${NEW_GID}:/" /etc/group;            fi;             chown -R $NEW_UID:$NEW_GID $HOME_FOLDER;       fi;

The problem you are facing is that the docker user, in this case ubuntu, with the same host user UID is already added and the code above does nothing in that case. This can be also seen in the devcontainer log.

User with UID exists (ubuntu=1000)

When the ubuntu docker user is removed the code changes the esp UID to the host user UID in the /etc/passwd.

Updating UID:GID from 1050:1050 to 1000:965.

The esp-idf container does not add any users and I believe that the right place to handle this is in the VSC devcontainer.

What are your thoughts on this?

Thank you

[tobiashagge]

Thank you for your detailed reply.

I don't see the same log messages anywhere in "Dev Containers Developer: Show All Logs..." (maybe a version issue), but I think your take is correct, and I appreciate the additional context.

I'll move this back to vscode-esp-idf-extension. It looks like the issue can be resolved by adding one line to a template file.

[fhrbata]

@tobiashagge , thank you for looking into this again. I'm no VSC user and this was the first time I was looking into the devcontainers and VSC in general, so I for sure could have missed something. It seems that the extension is using already cached images, so I modified the Docker file and rebuild it . Here are my steps.

1. cp -r ~/esp/master/esp-idf/examples/get-started/hello_world .
2. cd hello_world
3. F1 -> esp-idf: add docker container configuration
4. Add e.g. RUN echo "CHANGE" into the .devcontainer/Dockerfile
5. >< open a remote window -> reopen in container
6. >< Dev Container: ESP-IDF QEMU open a remote windows -> rebuild container
7. F1 -> dev containers: show container log

Here you should hopefully see it.

with cached image

Step 8/10 : RUN eval $(sed -n "s/${REMOTE_USER}:[^:]*:\([^:]*\):\([^:]*\):[^:]*:\([^:]*\).*/OLD_UID=\1;OLD_GID=\2;HOME_FOLDER=\3/p" /etc/passwd);       eval $(sed -n "s/\([^:]*\):[^:]*:${NEW_UID}:.*/EXISTING_USER=\1/p" /etc/passwd);       eval $(sed -n "s/\([^:]*\):[^:]*:${NEW_GID}:.*/EXISTING_GROUP=\1/p" /etc/group);       if [ -z "$OLD_UID" ]; then              echo "Remote user not found in /etc/passwd ($REMOTE_USER).";    elif [ "$OLD_UID" = "$NEW_UID" -a "$OLD_GID" = "$NEW_GID" ]; then              echo "UIDs and GIDs are the same ($NEW_UID:$NEW_GID).";         elif [ "$OLD_UID" != "$NEW_UID" -a -n "$EXISTING_USER" ]; then echo "User with UID exists ($EXISTING_USER=$NEW_UID).";         else            if [ "$OLD_GID" != "$NEW_GID" -a -n "$EXISTING_GROUP" ]; then                 echo "Group with GID exists ($EXISTING_GROUP=$NEW_GID).";                        NEW_GID="$OLD_GID";             fi;             echo "Updating UID:GID from $OLD_UID:$OLD_GID to $NEW_UID:$NEW_GID.";          sed -i -e "s/\(${REMOTE_USER}:[^:]*:\)[^:]*:[^:]*/\1${NEW_UID}:${NEW_GID}/" /etc/passwd;                if [ "$OLD_GID" != "$NEW_GID" ]; then                  sed -i -e "s/\([^:]*:[^:]*:\)${OLD_GID}:/\1${NEW_GID}:/" /etc/group;            fi;             chown -R $NEW_UID:$NEW_GID $HOME_FOLDER;       fi;
 ---> Using cache
 ---> a74e6ed41905

without cached image

Step 8/10 : RUN eval $(sed -n "s/${REMOTE_USER}:[^:]*:\([^:]*\):\([^:]*\):[^:]*:\([^:]*\).*/OLD_UID=\1;OLD_GID=\2;HOME_FOLDER=\3/p" /etc/passwd);       eval $(sed -n "s/\([^:]*\):[^:]*:${NEW_UID}:.*/EXISTING_USER=\1/p" /etc/passwd);    eval $(sed -n "s/\([^:]*\):[^:]*:${NEW_GID}:.*/EXISTING_GROUP=\1/p" /etc/group);        if [ -z "$OLD_UID" ]; then              echo "Remote user not found in /etc/passwd ($REMOTE_USER).";        elif [ "$OLD_UID" = "$NEW_UID" -a "$OLD_GID" = "$NEW_GID" ]; then            echo "UIDs and GIDs are the same ($NEW_UID:$NEW_GID).";     elif [ "$OLD_UID" != "$NEW_UID" -a -n "$EXISTING_USER" ]; then          echo "User with UID exists ($EXISTING_USER=$NEW_UID).";         else            if [ "$OLD_GID" != "$NEW_GID" -a -n "$EXISTING_GROUP" ]; then                       echo "Group with GID exists ($EXISTING_GROUP=$NEW_GID).";                       NEW_GID="$OLD_GID";             fi;             echo "Updating UID:GID from $OLD_UID:$OLD_GID to $NEW_UID:$NEW_GID.";               sed -i -e "s/\(${REMOTE_USER}:[^:]*:\)[^:]*:[^:]*/\1${NEW_UID}:${NEW_GID}/" /etc/passwd;                if [ "$OLD_GID" != "$NEW_GID" ]; then                sed -i -e "s/\([^:]*:[^:]*:\)${OLD_GID}:/\1${NEW_GID}:/" /etc/group;            fi;             chown -R $NEW_UID:$NEW_GID $HOME_FOLDER;    fi;
 ---> Running in 83273e578689
User with UID exists (ubuntu=1000).
Removing intermediate container 83273e578689

[dobairoland]

I'm closing this with the assumption that this will be addressed by espressif/vscode-esp-idf-extension#1320. We will re-open if this changes.