Skip to content

Latest commit

 

History

History
111 lines (89 loc) · 4.19 KB

8_use_stretch.md

File metadata and controls

111 lines (89 loc) · 4.19 KB
Raw

This tutorial explains how to use Stretch 3 with LeRobot.

Setup

Familiarize yourself with Stretch by following its tutorials (recommended).

To use LeRobot on Stretch, 3 options are available:

  • tethered setup
  • untethered setup
  • ssh directly into Stretch (you will first need to install and configure openssh-server on stretch using one of the two above setups)

Install LeRobot

On Stretch's CLI, follow these steps:

  1. Install Miniconda:
mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm ~/miniconda3/miniconda.sh
~/miniconda3/bin/conda init bash
  1. Comment out these lines in ~/.profile (this can mess up paths used by conda and ~/.local/bin should already be in your PATH)
# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi

  1. Restart shell or source ~/.bashrc

  2. Create and activate a fresh conda environment for lerobot

conda create -y -n lerobot python=3.10 && conda activate lerobot
  1. Clone LeRobot:
git clone https://github.com/huggingface/lerobot.git ~/lerobot
  1. Install LeRobot
cd ~/lerobot && pip install -e ".[stretch]"

conda install -y -c conda-forge ffmpeg
pip uninstall -y opencv-python
conda install -y -c conda-forge "opencv>=4.10.0"

Teleoperate, record a dataset and run a policy

Note: As indicated in Stretch's doc, you may need to free the "robot process" after booting Stretch by running stretch_free_robot_process.py

Before operating Stretch, you need to home it first. In the scripts used below, if the robot is not already homed it will be automatically homed first. Be mindful about giving Stretch some space as this procedure will move the robot's arm and gripper. If you want to "manually" home Stretch first, you can simply run this command:

python lerobot/scripts/control_robot.py calibrate \
    --robot-path lerobot/configs/robot/stretch.yaml

This is equivalent to running stretch_robot_home.py

Try out teleoperation (you can learn about the controls in Stretch's documentation):

python lerobot/scripts/control_robot.py teleoperate \
    --robot-path lerobot/configs/robot/stretch.yaml

This is essentially the same as running stretch_gamepad_teleop.py

Store your Hugging Face repository name in a variable to run these commands:

HF_USER=$(huggingface-cli whoami | head -n 1)
echo $HF_USER

Once you're familiar with the gamepad controls and after a bit of practice, try to record your first dataset with Stretch. Store your Hugging Face repository name in a variable to run these commands:

HF_USER=$(huggingface-cli whoami | head -n 1)
echo $HF_USER

Record one episode:

python lerobot/scripts/control_robot.py record \
    --robot-path lerobot/configs/robot/stretch.yaml \
    --fps 30 \
    --root data \
    --repo-id ${HF_USER}/stretch_test \
    --tags stretch tutorial \
    --warmup-time-s 3 \
    --episode-time-s 40 \
    --reset-time-s 10 \
    --num-episodes 1 \
    --push-to-hub 0

Note that if you're using ssh to connect to Stretch and run this script, you won't be able to visualize its cameras feed (though they'll still be recording).

Now try to replay this episode (make sure the robot's initial position is the same):

python lerobot/scripts/control_robot.py replay \
    --robot-path lerobot/configs/robot/stretch.yaml \
    --fps 30 \
    --root data \
    --repo-id ${HF_USER}/stretch_test \
    --episode 0