Files aren't visible to commands run in the SandBox in WSL in docker mode

[FlipEngineering]

Am I missing something?
workspace not visible in sandbox, and therefore can't execute python scripts in sandbox.

I'm on Windows 10 and I'm running WSL2 Ubuntu image, with Docker Desktop (integration enabled).
Connected to WSL in VS Code, and there in the shell I'm running these commands as described in the README, with light adaptation, maybe it's some of the changes I made?

These two lines of the docker command are "none standard":

    -e LLM_BASE_URL="https://api.groq.com/openai/v1/chat/completions" \
    -e SANDBOX_TYPE=exec \

the SANDBOX_TYPE = exec is because I wasn't able to get rid of the sshd error that most of you should be familiar with these days.

export LLM_API_KEY="gsk_XXX"
export WORKSPACE_BASE=$(pwd)/workspace

docker run \
    -e LLM_API_KEY \
    -e LLM_BASE_URL="https://api.groq.com/openai/v1/chat/completions" \
    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
    -e SANDBOX_TYPE=exec \
    -v $WORKSPACE_BASE:/opt/workspace_base \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal=host-gateway \
    ghcr.io/opendevin/opendevin:main 

After that I

I also tested to make some files in the working directory of the shell:

docker exec -it 4e0410591257 /bin/bash

root@docker-desktop:/workspace# touch test.txt

when getting devin to run ls it does show that file:

also me trying to trick open devin into going into /opt/workspace_base didn't work as you can clearly see

But Devin as well successfully writes files in the specified workspace folder in the opendevin docker, and that shows as well in vscode, just the sandbox docker doesn't seem to be linking the correct folder, if any at all(?)

I've removed all WSL dockers multiple times, same within, have tried the 0.3.1 version of openDevin and the main. with multiple iterations of cleaning the dockers out and downloading the new ones.

I feel like if there would be more people with this basic issue there would be some issues here with this, but I don't seem to find them.

Any idea what I'm doing wrong?

[rbren]

I think there might be a bug in the exec box--I've noticed something similar.

[rbren]

What's strange is that if I run

poetry run python opendevin/sandbox/docker/exec_box.py

everything is fine. I can run ls and see everything in my workspace

And I exec into the sandbox created by the agent controller, everything looks fine too.

But for some reason, if I ask the agent to run ls, it returns empty...

[rbren]

Actually--I just pulled main and this seems to be working. @FlipEngineering can you see if main is working for you? Here's what I get:

❯ docker run --pull=always \
-e LLM_API_KEY  \
-e SANDBOX_TYPE=exec \
-e WORKSPACE_MOUNT_PATH=$WORKSPACE_DIR     \
-v $WORKSPACE_DIR:/opt/workspace_base     \
-v /var/run/docker.sock:/var/run/docker.sock     \
-p 2000:3000     \
--add-host host.docker.internal=host-gateway     \
ghcr.io/opendevin/opendevin:main  python opendevin/main.py -t "run ls" -m claude-3-opus-20240229 -c MonologueAgent

main: Pulling from opendevin/opendevin
Digest: sha256:33abf220a562d8549725795cbeb57fac817bab5ae0836196fef13914dfc20368
Status: Image is up to date for ghcr.io/opendevin/opendevin:main
11:22:33 - opendevin:INFO: llm.py:52 - Initializing LLM with model: claude-3-opus-20240229
11:22:34 - opendevin:INFO: exec_box.py:221 - Container stopped
11:22:34 - opendevin:INFO: exec_box.py:239 - Container started


==============
STEP 0

11:22:35 - PLAN
run ls
11:22:40 - ACTION
CmdRunAction(command='ls', background=False, action='run')
11:22:40 - OBSERVATION
README.md
app
beep_script.sh
calculate_area.py
calculate_circle_area.py

[zhonggegege]

I often encounter this problem, mostly because the session is in progress and the task is still executing.

[qiviut]

I used to have the same problem, but it seems to be working thanks to the latest commits and switching from ExecBox back to the default.

[SmartManoj]

Seems mounting inside a docker is not working in wsl.

Here I gave a nonexistent directory in the host.

Relevant GH issue abiosoft/colima#503 from slack

Relevant issues:
https://forums.docker.com/t/docker-in-docker-share-storage-layers-with-host/22041

[SmartManoj]

Title: Workspace is not mounted for Sandobx started by Docker

[galakfryar]

I've worked around the problem with WSL by running this instead, using the volume feature from docker instead

docker run -d --name opendevin
--pull=always
--add-host host.docker.internal:host-gateway
-e SANDBOX_USER_ID="$(id -u)"
-e LLM_API_KEY="ollama"
-e LLM_BASE_URL="http://host.docker.internal:11434"
-e WORKSPACE_MOUNT_PATH=opendevin
-v opendevin:/opt/workspace_base
-v /var/run/docker.sock:/var/run/docker.sock
-p 3000:3000
ghcr.io/opendevin/opendevin:main

this works for me :D, I can see the volume mounted on the opendevin container and the sandbox container.

[SmartManoj]

Docker desktop shows as mounted on sandbox but it is not. Did you test it?

[rbren]

CC @williambarberjr

[rbren]

tl;dr: here's how you can help us debug this:

• start the app, and start a task
• run docker ps to find
  • the app container ID (image is ghcr.io/opendevin/opendevin)
  • the sandbox container ID (image is ghcr.io/opendevin/sandbox)
• using those IDs, run:

docker exec -it $APP_CONTAINER_ID ls -lah /opt/workspace_base

docker exec -it $SANDBOX_CONTAINER_ID ls -lah /workspace

---

Here's how the filesystems are supposed to work:

• You mount your workspace to the app container when starting it. That's where

export WORKSPACE_BASE=$(pwd)/workspace;
# ...
    -v $WORKSPACE_BASE:/opt/workspace_base \

comes from. So first good thing to check is that if you docker exec -it $APP_CONTAINER_ID /bin/bash you can see your files in /opt/workspace_base.

Next, when the sandbox starts, it mounts $WORKSPACE_MOUNT_PATH to /workspace inside the sandbox container. That's where

# ...
    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
# ...

comes from. So next thing to check: if you docker exec -it $SANDBOX_ID /bin/bash, are the files in there?

Lastly, we have some API endpoints that return a list of files in the workspace. If the above two work fine, the problem is likely there.

[SmartManoj]

Seems it affects only WSL..

Actual workspace

Seems the sandbox workspace is mounted to a different folder (inside the first container?). It's a persistent folder.

Mount path

Code

#The directory you want OpenDevin to modify. MUST be an absolute path!
export WORKSPACE_BASE=$(pwd)/workspace;

docker run \
    -it \
    --pull=always \
    -e SANDBOX_USER_ID=$(id -u) \
    -e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE \
    -v $WORKSPACE_BASE:/opt/workspace_base \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    ghcr.io/opendevin/opendevin:0.5

APP CONTAINER

SANDOBX CONTAINER

[SmartManoj]

Like this, it may be a docker problem affecting Windows.
Workaround: Run natively.

[SmartManoj]

Actually--I just pulled main and this seems to be working.

@rbren Your OS?

[BradKML]

Is the best solution now is just use Ubuntu (VM or otherwise) instead of Ubuntu in WSL?

[SmartManoj]

Running natively would be best in terms of storage and memory.

[tobitege]

Is the best solution now is just use Ubuntu (VM or otherwise) instead of Ubuntu in WSL?

I'm tending to the same conclusion and might try a VM instead of WSL. Though I'm not sure if the issues I experience are more an issue with Docker and mounted volumes.

[BradKML]

@galakfryar could you test the commands again (CodeAct with Ollama on WSL) following this conversation? Not sure if "file in workspace" issue still lingers or not if it is just a volume flag issue.
@tobitege which VM software would you recommend for Windows home desktops?

[tobitege]

@galakfryar could you test the commands again (CodeAct with Ollama on WSL) following this conversation? Not sure if "file in workspace" issue still lingers or not if it is just a volume flag issue. @tobitege which VM software would you recommend for Windows home desktops?

Not sure I can give a recommendation. Just from personal use, I have VirtualBox installed. There are plenty of images available for download, for pretty much every Linux distro, and it's free.
I haven't yet started using it with OpenDevin, yet, though.

[SmartManoj]

Works in Docker Desktop v4.31.1 & Docker version 26.1.4

[tobitege]

The issue isn't reproducable right now. Closing for now.