-
Notifications
You must be signed in to change notification settings - Fork 799
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add docker-compose
support for easier and more portable environment set-up
#519
Open
apockill
wants to merge
26
commits into
huggingface:main
Choose a base branch
from
apockill:feature/docker-compose
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 4 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
4f245d8
Add first revision of the `docker-compose.yaml`
apockill 64e368f
Add first revision of the `Dockerfile`
apockill 0b32337
Add docker instructions
apockill f7e3c3c
Clean up docs
apockill 80e75ad
Add `tty: true`
apockill 634d15a
Improve handling of poetry extras
apockill 69a3b9d
Improve handling of poetry extras
apockill a0940e0
Add required ports for visualization scripts
apockill 6631247
Add libsm6
apockill 59c1a78
Remove the need to do `xhost` commands
apockill 26dcaff
Use `poetry run` as the entrypoint
apockill c6d9ef2
Use consistent strings in docker-compose
apockill bca43d1
Bump poetry lockfile
apockill 9314f87
Successfully built pyav, and was able to use `cv2.imshow`!
apockill 7514f59
Add `python3-pip` to build
apockill 7d9d05d
Don't isolate python dependencies in container (this plays better wit…
apockill 3a35fd6
Remove spd-say so it doesn't crash the system
apockill e473a3d
Use a pre-built ffmpeg (this seems to also work with pyav!)
apockill 23aa4c3
Merge in `main`
apockill e901cc2
Simplify `mv` command to not reference directories too specifically
apockill 3e3eaa6
Fix pyav issues after the `Dataset 2.0` merge
apockill 016d9a0
Add credentials management so `wandb` and `huggingface` work properly
apockill e4073f7
Merge branch 'main' into feature/docker-compose
apockill d0729c5
Merge branch 'main' into feature/docker-compose
apockill 3b5748a
Add torch cache to compose file
apockill 23ffaff
Merge remote-tracking branch 'origin/feature/docker-compose' into fea…
apockill File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
name: lerobot | ||
|
||
services: | ||
lerobot: | ||
image: lerobot-env | ||
build: | ||
context: . | ||
dockerfile: docker/lerobot-compose/Dockerfile | ||
volumes: | ||
# Necessary for display passthrough | ||
- "/tmp/.X11-unix:/tmp/.X11-unix:rw" | ||
# Mount the current directory so users are not confused about data saving locations | ||
- ".:/lerobot" | ||
environment: | ||
# Necessary for display passthrough | ||
DISPLAY: $DISPLAY | ||
# Give the container as much access as possible to the host machine. | ||
# Folks using this library might be less familiar with how docker isolates devices | ||
# such as USB or networks, so removing that layer is helpful for the purposes of | ||
# development on this library. | ||
# | ||
# For production, remove the following line and add the necessary devices and networks | ||
privileged: true | ||
network_mode: host | ||
cap_add: | ||
- ALL |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
FROM nvidia/cuda:12.2.2-devel-ubuntu22.04 | ||
apockill marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
# Configure image | ||
ARG PYTHON_VERSION=3.10 | ||
ARG DEBIAN_FRONTEND=noninteractive | ||
ARG POETRY_EXTRAS="" | ||
ENV PIP_CACHE=/root/.cache/pip | ||
|
||
# Install apt dependencies | ||
RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
build-essential cmake \ | ||
git git-lfs openssh-client \ | ||
nano vim less util-linux tree \ | ||
htop atop nvtop \ | ||
sed gawk grep curl wget zip unzip \ | ||
tcpdump sysstat screen tmux \ | ||
libglib2.0-0 libgl1-mesa-glx libegl1-mesa \ | ||
speech-dispatcher \ | ||
# python*-dev is required for building certain python dependencies | ||
python${PYTHON_VERSION} python${PYTHON_VERSION}-venv python${PYTHON_VERSION}-dev\ | ||
&& apt-get clean && rm -rf /var/lib/apt/lists/* | ||
|
||
# Install ffmpeg build dependencies. See: | ||
# https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu | ||
# TODO(aliberts): create image to build dependencies from source instead | ||
RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
autoconf automake yasm \ | ||
libass-dev \ | ||
libfreetype6-dev \ | ||
libgnutls28-dev \ | ||
libunistring-dev \ | ||
libmp3lame-dev \ | ||
libtool \ | ||
libvorbis-dev \ | ||
meson \ | ||
ninja-build \ | ||
pkg-config \ | ||
texinfo \ | ||
yasm \ | ||
zlib1g-dev \ | ||
nasm \ | ||
libx264-dev \ | ||
libx265-dev libnuma-dev \ | ||
libvpx-dev \ | ||
libfdk-aac-dev \ | ||
libopus-dev \ | ||
libsvtav1-dev libsvtav1enc-dev libsvtav1dec-dev \ | ||
libdav1d-dev | ||
|
||
# Install gh cli tool | ||
RUN (type -p wget >/dev/null || (apt update && apt-get install wget -y)) \ | ||
&& mkdir -p -m 755 /etc/apt/keyrings \ | ||
&& wget -qO- https://cli.github.com/packages/githubcli-archive-keyring.gpg | tee /etc/apt/keyrings/githubcli-archive-keyring.gpg > /dev/null \ | ||
&& chmod go+r /etc/apt/keyrings/githubcli-archive-keyring.gpg \ | ||
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | tee /etc/apt/sources.list.d/github-cli.list > /dev/null \ | ||
&& apt update \ | ||
&& apt install gh -y \ | ||
&& apt clean && rm -rf /var/lib/apt/lists/* | ||
|
||
# Setup `python` | ||
RUN ln -s /usr/bin/python3 /usr/bin/python | ||
|
||
# Install poetry | ||
RUN curl -sSL https://install.python-poetry.org | python - | ||
ENV PATH="/root/.local/bin:$PATH" | ||
RUN echo 'if [ "$HOME" != "/root" ]; then ln -sf /root/.local/bin/poetry $HOME/.local/bin/poetry; fi' >> /root/.bashrc | ||
RUN poetry config virtualenvs.create false | ||
RUN poetry config virtualenvs.in-project true | ||
|
||
# Copy in pyproject | ||
WORKDIR /lerobot | ||
COPY pyproject.toml poetry.lock ./ | ||
RUN apt-get update && apt-get install -y python3-dev | ||
# Install dependencies, while using docker builkit cache to speed up future builds | ||
RUN --mount=type=cache,target="${PIP_CACHE}" \ | ||
poetry install --extras ${POETRY_EXTRAS} | ||
|
||
# Set EGL as the rendering backend for MuJoCo | ||
ENV MUJOCO_GL="egl" |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In future PR's we could split up this service to be
lerobot-x86
andlerobot-arm
to support multiple platforms (like jetson).For now, I think let's see how people use this, improve the experience, then tackle multi-arch support in the future.