Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes isaaclab scripts to deal with Isaac Sim pip installation #631

Merged
merged 28 commits into from
Jul 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
a63d97e
adds initial fix
Mayankm96 Jul 3, 2024
c739249
working setup without env file
Mayankm96 Jul 3, 2024
d09fff0
works without any other code
Mayankm96 Jul 3, 2024
9d92b49
runs formatter
Mayankm96 Jul 3, 2024
f568db5
runs formatter
Mayankm96 Jul 3, 2024
0327c3a
fix scripts to use isaacsim pip paths
Mayankm96 Jul 3, 2024
acbda9a
gets path correclty
Mayankm96 Jul 3, 2024
1234de4
checks steps for bash
Mayankm96 Jul 3, 2024
08a57cf
adds dockerfile for pip installation
Mayankm96 Jul 3, 2024
43682be
adds python in pip dockerfile
Mayankm96 Jul 3, 2024
ce363cc
fixes docs
Mayankm96 Jul 3, 2024
c51cce3
fixes vscode setup on launch
Mayankm96 Jul 3, 2024
c85f119
runs formatter
Mayankm96 Jul 3, 2024
682882b
fixes based on kelly
Mayankm96 Jul 4, 2024
df24b93
adds throwing error for wrong python version
Mayankm96 Jul 4, 2024
ea93778
Merge branch 'main' into fix/isaaclab-script
Mayankm96 Jul 4, 2024
2cc68fe
Merge branch 'main' into fix/isaaclab-script
Mayankm96 Jul 4, 2024
cc26f0a
fix backslash in paths
kellyguo11 Jul 4, 2024
d7881ce
Merge branch 'fix/isaaclab-script' of github.com:isaac-sim/IsaacLab i…
kellyguo11 Jul 4, 2024
bdd772e
makes yq installation not mandatory
Mayankm96 Jul 4, 2024
1cc8e21
simplify logic in windows
Mayankm96 Jul 4, 2024
64d928b
deals with docker container cases
Mayankm96 Jul 4, 2024
0be5e7e
fixes for windows script
kellyguo11 Jul 5, 2024
5d050c6
deals with corner cases
Mayankm96 Jul 5, 2024
467c272
runs formatter
Mayankm96 Jul 5, 2024
bff56ae
adds back vscode setting
Mayankm96 Jul 5, 2024
07a6ea4
runs formatter
Mayankm96 Jul 5, 2024
a496c29
better warning
Mayankm96 Jul 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 45 additions & 38 deletions .vscode/tools/setup_vscode.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@
# check if the isaac-sim directory exists
if not os.path.exists(isaacsim_dir):
raise FileNotFoundError(
f"Could not find the isaac-sim directory: {isaacsim_dir}. There are two possible reasons for this:\n"
"\t1. The Isaac Sim directory does not exist as a symlink in the Isaac Lab directory.\n"
"\t2. The script could import the 'isaacsim' package. This could be due to the 'isaacsim' package not being "
f"Could not find the isaac-sim directory: {isaacsim_dir}. There are two possible reasons for this:"
f"\n\t1. The Isaac Sim directory does not exist as a symlink at: {os.path.join(ISAACLAB_DIR, '_isaac_sim')}"
"\n\t2. The script could import the 'isaacsim' package. This could be due to the 'isaacsim' package not being "
"installed in the Python environment.\n"
"Please make sure that the Isaac Sim directory exists or that the 'isaacsim' package is installed."
"\nPlease make sure that the Isaac Sim directory exists or that the 'isaacsim' package is installed."
)

ISAACSIM_DIR = isaacsim_dir
Expand All @@ -48,39 +48,49 @@ def overwrite_python_analysis_extra_paths(isaaclab_settings: str) -> str:
The extraPaths are replaced with the path names from the isaac-sim settings file that exists in the
"{ISAACSIM_DIR}/.vscode/settings.json" file.

If the isaac-sim settings file does not exist, the extraPaths are not overwritten.

Args:
isaaclab_settings: The settings string to use as template.

Returns:
The settings string with overwritten python analysis extra paths.

Raises:
FileNotFoundError: If the isaac-sim settings file does not exist.
"""
# isaac-sim settings
isaacsim_vscode_filename = os.path.join(ISAACSIM_DIR, ".vscode", "settings.json")
# make sure the isaac-sim settings file exists
if not os.path.exists(isaacsim_vscode_filename):
raise FileNotFoundError(f"Could not find the isaac-sim settings file: {isaacsim_vscode_filename}")

# read the path names from the isaac-sim settings file
with open(isaacsim_vscode_filename) as f:
vscode_settings = f.read()
# extract the path names
# search for the python.analysis.extraPaths section and extract the contents
settings = re.search(r"\"python.analysis.extraPaths\": \[.*?\]", vscode_settings, flags=re.MULTILINE | re.DOTALL)
settings = settings.group(0)
settings = settings.split('"python.analysis.extraPaths": [')[-1]
settings = settings.split("]")[0]

# read the path names from the isaac-sim settings file
path_names = settings.split(",")
path_names = [path_name.strip().strip('"') for path_name in path_names]
path_names = [path_name for path_name in path_names if len(path_name) > 0]

# change the path names to be relative to the Isaac Lab directory
rel_path = os.path.relpath(ISAACSIM_DIR, ISAACLAB_DIR)
path_names = ['"${workspaceFolder}/' + rel_path + "/" + path_name + '"' for path_name in path_names]

# we use the isaac-sim settings file to get the python.analysis.extraPaths for kit extensions
# if this file does not exist, we will not add any extra paths
if os.path.exists(isaacsim_vscode_filename):
# read the path names from the isaac-sim settings file
with open(isaacsim_vscode_filename) as f:
vscode_settings = f.read()
# extract the path names
# search for the python.analysis.extraPaths section and extract the contents
settings = re.search(
r"\"python.analysis.extraPaths\": \[.*?\]", vscode_settings, flags=re.MULTILINE | re.DOTALL
)
settings = settings.group(0)
settings = settings.split('"python.analysis.extraPaths": [')[-1]
settings = settings.split("]")[0]

# read the path names from the isaac-sim settings file
path_names = settings.split(",")
path_names = [path_name.strip().strip('"') for path_name in path_names]
path_names = [path_name for path_name in path_names if len(path_name) > 0]

# change the path names to be relative to the Isaac Lab directory
rel_path = os.path.relpath(ISAACSIM_DIR, ISAACLAB_DIR)
path_names = ['"${workspaceFolder}/' + rel_path + "/" + path_name + '"' for path_name in path_names]
else:
path_names = []
print(
f"[WARN] Could not find Isaac Sim VSCode settings: {isaacsim_vscode_filename}."
"\n\tThis will result in missing 'python.analysis.extraPaths' in the VSCode"
"\n\tsettings, which limits the functionality of the Python language server."
"\n\tHowever, it does not affect the functionality of the Isaac Lab project."
"\n\tWe are working on a fix for this issue with the Isaac Sim team."
)

# add the path names that are in the Isaac Lab extensions directory
isaaclab_extensions = os.listdir(os.path.join(ISAACLAB_DIR, "source", "extensions"))
Expand All @@ -89,9 +99,9 @@ def overwrite_python_analysis_extra_paths(isaaclab_settings: str) -> str:
# combine them into a single string
path_names = ",\n\t\t".expandtabs(4).join(path_names)
# deal with the path separator being different on Windows and Unix
path_names = path_names.replace("/", os.sep)
path_names = path_names.replace("\\", "/")

# replace the path names in the Isaac Lab settings file with the path names from the isaac-sim settings file
# replace the path names in the Isaac Lab settings file with the path names parsed
isaaclab_settings = re.sub(
r"\"python.analysis.extraPaths\": \[.*?\]",
'"python.analysis.extraPaths": [\n\t\t'.expandtabs(4) + path_names + "\n\t]".expandtabs(4),
Expand All @@ -116,10 +126,7 @@ def overwrite_default_python_interpreter(isaaclab_settings: str) -> str:
The settings string with overwritten default python interpreter.
"""
# read executable name
python_exe = sys.executable
# if python interpreter is from conda, use that. Otherwise, use the template.
if "conda" not in python_exe:
return isaaclab_settings
python_exe = sys.executable.replace("\\", "/")
# replace the default python interpreter in the Isaac Lab settings file with the path to the
# python interpreter in the Isaac Lab directory
isaaclab_settings = re.sub(
Expand All @@ -146,9 +153,9 @@ def main():

# overwrite the python.analysis.extraPaths in the Isaac Lab settings file with the path names
isaaclab_settings = overwrite_python_analysis_extra_paths(isaaclab_template_settings)
# overwrite the default python interpreter in the Isaac Lab settings file
# NOTE: thisis disabled since we don't need it. The default interpreter should always be the one from isaac-sim
# isaaclab_settings = overwrite_default_python_interpreter(isaaclab_settings)
# overwrite the default python interpreter in the Isaac Lab settings file with the path to the
# python interpreter used to call this script
isaaclab_settings = overwrite_default_python_interpreter(isaaclab_settings)

# add template notice to the top of the file
header_message = (
Expand Down
100 changes: 100 additions & 0 deletions docker/Dockerfile.pip
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Copyright (c) 2022-2024, The Isaac Lab Project Developers.
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause

# This Dockerfile is used to build a Docker image for the Isaac Lab framework.
#
# It uses the pip package manager to install Isaac Sim and the Isaac Lab framework.
#
# To build the Docker image and run the Docker container, follow the steps below:
#
# 1. Build the Docker image:
# docker build -t isaac-lab:latest -f docker/Dockerfile.pip .
# 2. Run the Docker container:
# docker run -it --gpus all --rm --network=host --name isaac-lab -v $(pwd):/root/isaaclab isaac-lab:latest

# Base image: Ubuntu 22.04
FROM ubuntu:22.04 AS base

# Set default RUN shell to bash
SHELL ["/bin/bash", "-c"]

# Adds labels to the Dockerfile
LABEL version="1.0"
LABEL description="Dockerfile for building and running the Isaac Lab framework in Ubuntu 22.04 container image."

# Arguments
# Path to the Isaac Lab directory
ENV ISAACLAB_PATH=/root/isaaclab
# Home dir of docker user, typically '/root'
ENV DOCKER_USER_HOME=/root

# Set environment variables
ENV LANG=C.UTF-8
ENV DEBIAN_FRONTEND=noninteractive

USER root

# Install dependencies and remove cache
RUN --mount=type=cache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
build-essential \
cmake \
git \
libglib2.0-0 \
ncurses-term && \
apt -y autoremove && apt clean autoclean && \
rm -rf /var/lib/apt/lists/*

# for singularity usage, have to create the directories that will binded
RUN mkdir -p ${DOCKER_USER_HOME}/.cache/ov && \
mkdir -p ${DOCKER_USER_HOME}/.cache/pip && \
mkdir -p ${DOCKER_USER_HOME}/.cache/nvidia/GLCache && \
mkdir -p ${DOCKER_USER_HOME}/.nv/ComputeCache && \
mkdir -p ${DOCKER_USER_HOME}/.nvidia-omniverse/logs && \
mkdir -p ${DOCKER_USER_HOME}/.local/share/ov/data && \
mkdir -p ${DOCKER_USER_HOME}/Documents

# for singularity usage, create NVIDIA binary placeholders
RUN touch /bin/nvidia-smi && \
touch /bin/nvidia-debugdump && \
touch /bin/nvidia-persistenced && \
touch /bin/nvidia-cuda-mps-control && \
touch /bin/nvidia-cuda-mps-server && \
touch /etc/localtime && \
mkdir -p /var/run/nvidia-persistenced && \
touch /var/run/nvidia-persistenced/socket

# Install python3 and pip
RUN apt-get update && apt-get install -y --no-install-recommends \
python3 \
python3-pip && \
apt -y autoremove && apt clean autoclean && \
rm -rf /var/lib/apt/lists/*
# Maintain python3 as the default python version
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 1
RUN update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 1

# Install python packages
RUN pip3 install --no-cache-dir --upgrade pip && \
pip install torch==2.2.2 --index-url https://download.pytorch.org/whl/cu118

RUN pip install isaacsim-rl isaacsim-replicator isaacsim-extscache-physics isaacsim-extscache-kit-sdk isaacsim-extscache-kit isaacsim-app --extra-index-url https://pypi.nvidia.com

# Mount the Isaac Lab directory (files to exclude are defined in .dockerignore)
COPY ../ ${ISAACLAB_PATH}

# installing Isaac Lab dependencies
# use pip caching to avoid reinstalling large packages
RUN --mount=type=cache,target=${DOCKER_USER_HOME}/.cache/pip \
${ISAACLAB_PATH}/isaaclab.sh --install

# aliasing isaaclab.sh and python for convenience
RUN echo "export ISAACLAB_PATH=${ISAACLAB_PATH}" >> ${HOME}/.bashrc && \
echo "alias isaaclab=${ISAACLAB_PATH}/isaaclab.sh" >> ${HOME}/.bashrc && \
echo "export TZ=$(date +%Z)" >> ${HOME}/.bashrc

# make working directory as the Isaac Lab directory
# this is the default directory when the container is run
WORKDIR ${ISAACLAB_PATH}
27 changes: 18 additions & 9 deletions docker/container.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ fi

# print the usage description
print_help () {
echo -e "\nusage: $(basename "$0") [-h] [run] [start] [stop] -- Utility for handling docker in Isaac Lab."
echo -e "\nusage: $(basename "$0") [-h] [...] -- Utility for handling Docker in Isaac Lab."
echo -e "\noptional arguments:"
echo -e "\t-h, --help Display the help content."
echo -e "\tstart [profile] Build the docker image and create the container in detached mode."
Expand All @@ -36,8 +36,9 @@ print_help () {
echo -e "\tjob [profile] [job_args] Submit a job to the cluster."
echo -e "\tconfig [profile] Parse, resolve and render compose file in canonical format."
echo -e "\n"
echo -e "[profile] is the optional container profile specification and [job_args] optional arguments specific"
echo -e "to the executed script"
echo -e "where: "
echo -e "\t[profile] is the optional container profile specification. Example: 'isaaclab', 'base', 'ros2'."
echo -e "\t[job_args] are optional arguments specific to the executed script."
echo -e "\n" >&2
}

Expand All @@ -56,8 +57,8 @@ install_apptainer() {

install_yq() {
# Installing yq to handle file parsing
# Installation procedure from here: https://github.com/mikefarah/yq
read -p "[INFO] Required 'yq' package could not be found. Would you like to install it via wget? (y/N)" yq_answer
# Installation procedure from here: https://github.com/mikefarah/yq?tab=readme-ov-file#linux-via-snap
read -p "[INFO] Required 'yq' package could not be found. Would you like to install it via snap? (y/N)" yq_answer
if [ "$yq_answer" != "${yq_answer#[Yy]}" ]; then
sudo snap install yq
else
Expand All @@ -67,17 +68,29 @@ install_yq() {
}

set_statefile_variable() {
# Check if yq is installed
if ! command -v yq &> /dev/null; then
install_yq
fi
# Stores key $1 with value $2 in yaml $STATEFILE
yq -i '.["'"$1"'"] = "'"$2"'"' $STATEFILE
}

load_statefile_variable() {
# Check if yq is installed
if ! command -v yq &> /dev/null; then
install_yq
fi
# Loads key $1 from yaml $STATEFILE as an envvar
# If key does not exist, the loaded var will equal "null"
eval $1="$(yq ".$1" $STATEFILE)"
}

delete_statefile_variable() {
# Check if yq is installed
if ! command -v yq &> /dev/null; then
install_yq
fi
# Deletes key $1 from yaml $STATEFILE
yq -i "del(.$1)" $STATEFILE
}
Expand Down Expand Up @@ -270,10 +283,6 @@ if ! command -v docker &> /dev/null; then
exit 1
fi

if ! command -v yq &> /dev/null; then
install_yq
fi

# parse arguments
mode="$1"
profile_arg="$2" # Capture the second argument as the potential profile argument
Expand Down
3 changes: 3 additions & 0 deletions docs/source/setup/installation/binaries_installation.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
.. _isaacsim-binaries-installation:


Installation using Isaac Sim Binaries
=====================================

Expand Down
16 changes: 9 additions & 7 deletions docs/source/setup/installation/pip_installation.rst
Original file line number Diff line number Diff line change
@@ -1,22 +1,24 @@
.. _isaacsim-pip-installation:

Installation using Isaac Sim pip
================================


Installing Isaac Sim
--------------------

.. note::

Installing Isaac Sim from pip is currently an experimental feature.
If errors occur, please report them to the
`Isaac Sim Forums <https://docs.omniverse.nvidia.com/isaacsim/latest/common/feedback.html>`_
and install Isaac Sim from pre-built binaries.
From Isaac Sim 4.0 release, it is possible to install Isaac Sim using pip. This approach is experimental and may have
compatibility issues with some Linux distributions. If you encounter any issues, please report them to the
`Isaac Sim Forums <https://docs.omniverse.nvidia.com/isaacsim/latest/common/feedback.html>`_.

.. note::
.. attention::

Installing Isaac Sim with pip requires GLIBC 2.34+ version compatibility.
To check the GLIBC version on your system, use command ``ldd --version``.

This may pose compatibility issues with some Linux distributions. For instance, Ubuntu 20.04 LTS has GLIBC 2.31
by default. If you encounter compatibility issues, we recommend following the
:ref:`Isaac Sim Binaries Installation <isaacsim-binaries-installation>` approach.

- To use the pip installation approach for Isaac Sim, we recommend first creating a virtual environment.
Ensure that the python version of the virtual environment is **Python 3.10**.
Expand Down
Loading
Loading