[Bug]: Runtime Directory Hash Mismatch

[xingyaoww]

Is there an existing issue for the same bug?

• I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
• I have checked the existing issues.

Describe the bug

The directory hash calculated for Runtime API does not match when calculate in different environments.

For example, In this CI: https://github.com/All-Hands-AI/OpenHands/actions/runs/10739603421/job/29785986651

When building the images (Build Image (nikolaik/python-nodejs:python3.11-nodejs22, nikolaik)), we have 4859922960d366c29664226abe370db3 as hash:

Run poetry run python3 openhands/runtime/utils/runtime_build.py --base_image nikolaik/python-nodejs:python3.11-nodejs22 --build_folder containers/runtime --force_rebuild
13:59:42 - openhands:INFO: runtime_build.py:399 - Copying the source code and generating the Dockerfile in the build folder: containers/runtime
13:59:42 - openhands:INFO: runtime_build.py:406 - Runtime image repo: ghcr.io/all-hands-ai/runtime and runtime image tag: oh_v0.9.2_image_nikolaik_s_python-nodejs_tag_python3.[11](https://github.com/All-Hands-AI/OpenHands/actions/runs/10739603421/job/29785986651#step:10:12)-nodejs22
13:59:42 - openhands:INFO: runtime_build.py:43 - Using project root: /home/runner/work/OpenHands/OpenHands
13:59:43 - openhands:INFO: runtime_build.py:60 - * Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - poetry-core
* Getting build dependencies for sdist...
* Building sdist...
Successfully built openhands_ai-0.9.2.tar.gz

[13](https://github.com/All-Hands-AI/OpenHands/actions/runs/10739603421/job/29785986651#step:10:14):59:43 - openhands:INFO: runtime_build.py:72 - Source distribution created at /tmp/tmp0d0kf3fg/openhands_ai-0.9.2.tar.gz
13:59:43 - openhands:INFO: runtime_build.py:83 - Unpacked source code directory: /tmp/tmp0d0kf3fg/code
13:59:43 - openhands:INFO: runtime_build.py:[15](https://github.com/All-Hands-AI/OpenHands/actions/runs/10739603421/job/29785986651#step:10:16)4 - Input base image: nikolaik/python-nodejs:python3.11-nodejs22
Skip init: False
Extra deps: None
Hash for docker build directory [/tmp/tmp0d0kf3fg] (contents: ['Dockerfile', 'code']): 4859922960d366c29664226abe370db3

13:59:43 - openhands:INFO: runtime_build.py:309 - Force re-build: Will try to re-build image [ghcr.io/all-hands-ai/runtime:oh_v0.9.2_image_nikolaik_s_python-nodejs_tag_python3.11-nodejs[22](https://github.com/All-Hands-AI/OpenHands/actions/runs/10739603421/job/29785986651#step:10:23)] from scratch.

13:59:43 - openhands:INFO: runtime_build.py:43 - Using project root: /home/runner/work/OpenHands/OpenHands
13:59:45 - openhands:INFO: runtime_build.py:60 - * Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - poetry-core
* Getting build dependencies for sdist...
* Building sdist...
Successfully built openhands_ai-0.9.2.tar.gz

13:59:45 - openhands:INFO: runtime_build.py:72 - Source distribution created at /tmp/tmpp7sfio7p/openhands_ai-0.9.2.tar.gz
13:59:45 - openhands:INFO: runtime_build.py:83 - Unpacked source code directory: /tmp/tmpp7sfio7p/code
13:59:45 - openhands:INFO: runtime_build.py:154 - Input base image: nikolaik/python-nodejs:python3.11-nodejs22
Skip init: False
Extra deps: None
Hash for docker build directory [/tmp/tmpp7sfio7p] (contents: ['Dockerfile', 'code']): 4859922960d366c296642[26](https://github.com/All-Hands-AI/OpenHands/actions/runs/10739603421/job/29785986651#step:10:27)abe370db3

13:59:45 - openhands:INFO: runtime_build.py:334 - Dry run: Skipping image build for [ghcr.io/all-hands-ai/runtime:oh_v0.9.2_image_nikolaik_s_python-nodejs_tag_python3.11-nodejs22]
13:59:45 - openhands:INFO: runtime_build.py:4[27](https://github.com/All-Hands-AI/OpenHands/actions/runs/10739603421/job/29785986651#step:10:28) - Build folder [containers/runtime] is ready: ['Dockerfile', 'code', 'config.sh', 'README.md']
13:59:45 - openhands:INFO: runtime_build.py:441 - `config.sh` is updated with the image repo[ghcr.io/all-hands-ai/runtime] and tags [oh_v0.9.2_image_nikolaik_s_python-nodejs_tag_python3.11-nodejs22, 485992[29](https://github.com/All-Hands-AI/OpenHands/actions/runs/10739603421/job/29785986651#step:10:30)60d366c29664226abe370db3]
13:59:45 - openhands:INFO: runtime_build.py:444 - Dockerfile, source code and config.sh are ready in containers/runtime

But in the same exact code base but in different environments Test Runtime (nikolaik), we got b0f3fa5e3c2a114044e2e3a74827aa99 as new hash:

14:04:53 - openhands:INFO: runtime_build.py:72 - Source distribution created at /tmp/tmp0m0t3e15/openhands_ai-0.9.2.tar.gz
14:04:53 - openhands:INFO: runtime_build.py:83 - Unpacked source code directory: /tmp/tmp0m0t3e15/code
14:04:53 - openhands:INFO: runtime_build.py:154 - Input base image: ghcr.io/all-hands-ai/runtime:0f118df9106c1c6fe728552eec1bff1ae486a7db-nikolaik
Skip init: False
Extra deps: None
Hash for docker build directory [/tmp/tmp0m0t3e15] (contents: ['Dockerfile', 'code']): 147610fda58a9e2cdca3efa0942f29f6

14:04:53 - openhands:INFO: runtime_build.py:174 - The provided image [ghcr.io/all-hands-ai/runtime:0f118df9106c1c6fe728552eec1bff1ae486a7db-nikolaik] is already a valid runtime image.
Will try to reuse it as is.
14:04:53 - openhands:INFO: docker.py:72 - Checking, if image ghcr.io/all-hands-ai/runtime:147610fda58a9e2cdca3efa0942f29f6 exists locally.
14:04:53 - openhands:INFO: docker.py:78 - Image not found locally. Trying to pull it, please wait...
14:04:53 - openhands:INFO: docker.py:88 - Could not pull image directly.
14:04:53 - openhands:INFO: docker.py:72 - Checking, if image ghcr.io/all-hands-ai/runtime:0f118df9106c1c6fe728552eec1bff1ae486a7db-nikolaik exists locally.
14:04:53 - openhands:INFO: docker.py:78 - Image not found locally. Trying to pull it, please wait...
14:05:55 - openhands:INFO: docker.py:82 - Image ghcr.io/all-hands-ai/runtime:0f118df9106c1c6fe728552eec1bff1ae486a7db-nikolaik pulled successfully.
14:05:55 - openhands:INFO: runtime_build.py:265 - Could not find docker image [ghcr.io/all-hands-ai/runtime:147610fda58a9e2cdca3efa0942f29f6]
Will try to re-build it from latest [ghcr.io/all-hands-ai/runtime:0f118df9106c1c6fe728552eec1bff1ae486a7db-nikolaik] image to potentially save time for dependencies installation.

14:05:55 - openhands:INFO: runtime_build.py:43 - Using project root: /home/runner/work/OpenHands/OpenHands
14:05:56 - openhands:INFO: runtime_build.py:60 - * Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - poetry-core
* Getting build dependencies for sdist...
* Building sdist...
Successfully built openhands_ai-0.9.2.tar.gz

14:05:56 - openhands:INFO: runtime_build.py:72 - Source distribution created at /tmp/tmplos96htc/openhands_ai-0.9.2.tar.gz
14:05:57 - openhands:INFO: runtime_build.py:83 - Unpacked source code directory: /tmp/tmplos96htc/code
14:05:57 - openhands:INFO: runtime_build.py:154 - Input base image: ghcr.io/all-hands-ai/runtime:0f118df9106c1c6fe728552eec1bff1ae486a7db-nikolaik
Skip init: True
Extra deps: None
Hash for docker build directory [/tmp/tmplos96htc] (contents: ['Dockerfile', 'code']): b0f3fa5e3c2a11[40](https://github.com/All-Hands-AI/OpenHands/actions/runs/10739603421/job/29786240563#step:9:41)44e2e3a74827aa99

The mismatch of this hash will cause issues: if the user is launching OpenHands with docker run, it will likely generate a different dir hash, making the user have to re-build the runtime image the first time they launch via docker run.

Current OpenHands version

main

Installation and Configuration

N/A

Model and Agent

No response

Operating System

No response

Reproduction Steps

No response

Logs, Errors, Screenshots, and Additional Context

No response

[tobitege]

The "fix" here could be as simple as using the same-named temp directory (to place the sdist in) before creating hashes as apparently any different path also changes the hash.

[xingyaoww]

@tobitege i think the issue was a little bit trickier than i initially thought - we copy a different subset of folders into app docker and changed their permission. I suspect because of that, the hash calculated within the app docker is different than the one calculated for the development mode.

[tobitege]

eww that complicates things even more 😔