Skip to content

Commit

Permalink
Update VR info
Browse files Browse the repository at this point in the history
  • Loading branch information
glannuzel committed Apr 24, 2024
1 parent e226dfb commit f6adfae
Show file tree
Hide file tree
Showing 24 changed files with 175 additions and 100 deletions.
Binary file added assets/images/vr/use-teleop/emergency-stop.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/images/vr/use-teleop/help-panel.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed assets/images/vr/use-teleop/htc-vive-mapping.png
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/vr/use-teleop/mirror-info.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file removed assets/images/vr/use-teleop/valve-index-mapping.png
Binary file not shown.
20 changes: 19 additions & 1 deletion config/_default/menus/menus.en.toml
Original file line number Diff line number Diff line change
Expand Up @@ -428,11 +428,29 @@
url = "vr/use-teleop/start"

[[vr]]
name = "Teleoperation messages"
name = "Control the mobile base"
weight = 440
parent = "vr-use"
url = "vr/use-teleop/mobile-base"

[[vr]]
name = "Reduce motion sickness"
weight = 440
parent = "vr-use"
url = "vr/use-teleop/motion-sickness"

[[vr]]
name = "Teleoperation messages"
weight = 450
parent = "vr-use"
url = "vr/use-teleop/messages"

[[vr]]
name = "Emergency stop"
weight = 460
parent = "vr-use"
url = "vr/use-teleop/emergency-stop"

##################
## Problem section
##################
Expand Down
2 changes: 2 additions & 0 deletions content/vr/introduction/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,5 @@ The Virtual Reality (VR) teleoperation application enables you to **control the
By connecting to your robot, the teleoperation application gives you the ability to **move Reachy's arm** with the tracking of the VR controllers, to **rotate Reachy's head** following your own head movements and to **see through Reachy's cameras**.

You can also **manipulate objects** remotely controlling Reachy's grippers with your controllers' triggers.

{{< youtube vVIBlbS2zJs >}}
28 changes: 13 additions & 15 deletions content/vr/problem/debug.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,26 +7,33 @@ draft: false
images: []
type: docs
toc: true
weight: "120"
weight: "150"
---

## Check the info on the app!
Connect to the robot to get more information on the connection status and the status of the robot. Open the "info" menu on the left of the mirror.

{{< img "images/vr/problem/mirror-info.png" 600x "Mirror Info">}}

### Connection status
The connection status give you information about the communication with the robot. Existing connection status are the following:
* **Connected to a remote Reachy** *(green)*: everything seems to be working fine
* **Connected to a remote Reachy. No restart available** *(yellow)*: you do not have access to the restart service, but can use normally teleoperation
* **Connected to a remote Reachy. Mobile base unavailable** *(purple)*: the configuration of your robot declares a mobile base, but no mobility service is available. You can still teleoperate the robot with no mobility.
* **Trying to connect** *(blue)*: the app is looking for the connection with the robot
* **Robot connection failed** *(orange)*: you are connected to a remote robot, but either the camera feed or the data stream failed. Teleoperation is not possible
* **Unable to connect to remote server** *(red)*: no robot or service is detected after trying to connect

### Network connection quality
The status of the network connection can also help you:
* **Good network connection** indicates the application manage to have fast responses from the robot
* **Unable to reach robot** indicates the application doesn't manage to get any answer from the given IP address on the network

### Services availability

You can also check which services are available:
* **Camera**: camera service from the cameras. ***Mandatory for teleoperation (except for single arm configuration)***
* **Joints data**: joints services for sending and receiving data from the robot's joints. ***Mandatory for teleoperation***
* **Mobility**: services to control the mobile base, available only on robots equipped with a mobile base
* **Camera**: camera service from the cameras. ***Mandatory for teleoperation***
* **Audio**: service to get sound from the robot to the operator
* **Microphone**: service to send sound of the operator through the robot
* **Motors**: joints services for sending and receiving data from the robot's motors ***Mandatory for teleoperation***

## The app doesn't connect to the robot

Expand Down Expand Up @@ -61,12 +68,3 @@ A warning message may also be displayed during teleoperation indicated the netwo
If a motor is overheating, it may have **stopped working**, which can lead in movements looking very different than yours. In reality, the arm is still trying to move according to yours, but the unmoving joints make the configuration of the arm hard to understand.
In most of the cases, an **error message** should be displayed in the teleoperation, telling that at least 1 motor is in critical error.
Nevertheless it may happen that no error message is displayed, if the motor stopped working before having time to send the information to the teleoperation app: in that case, you received a warning message telling at least 1 motor was heating up previously during teleoperation. Check the **temperature of the motors** in the **Info panel** of the transition room.

## The mobile base doesn't move
Several elements can make the mobile base unreactive to your inputs.
* If you are too close to a wall or object, the LIDAR anti-collision safety unables the mobile base to go closer to the obstacle. The mobile base will therefore not move in this direction, but you can still go in other directions. [More information on the anti-collision safety](https://docs.pollen-robotics.com/sdk/mobile-base/safety/)
* The mobility button has been disabled: to check the status of the mobility button, go in the help panel in the menu (welcome page). Set the mobility to ON.
{{< img "images/vr/problem/mirror-settings.png" 600x "Mirror Setttings">}}
* The mobility services are unavailable: check the status of the service in the help panel of the menu. The status of the mobility services is displayed in the info menu.
* The configuration of your robot does not declare a mobile base, therefore the teleoperation application does not provide any mobility service. Check if a mobile base is expected in the Robot detected configuration section.

2 changes: 1 addition & 1 deletion content/vr/problem/support-vr.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ draft: false
images: []
type: docs
toc: true
weight: "130"
weight: "160"
---

## Discord
Expand Down
64 changes: 15 additions & 49 deletions content/vr/use-teleop/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,43 +11,24 @@ hidden: true
weight: "90"
---

## Oculus Quest
> A reminder of the controller inputs mapping is available in the **help** section of the *transition room* in the VR teleoperation application:
{{< img "images/vr/use-teleop/help-panel.png" 600x "Help panel in VR transition room">}}
{{< img "images/vr/use-teleop/oculus-quest-mapping.png" 600x "Oculus Quest controller mapping">}}

|Name|Feature description |
|----|--------------------|
|**A**|**At robot teleoperation start:** Start robot teleoperation|
| |**During teleoperation:** Return to menu|
|**B**|**During teleoperation:** Mobile base boost|
|**X**|**During teleoperation:** Open emotion menu (keep pressed) / Activate selected emotion (release)|
| |**When leaving teleoperation (A pressed):** Lock robot position|
|**Left Thumbstick**|**During teleoperation:** Control mobile base translation|
|**Right Thumbstick**|**During teleoperation:** Control mobile base rotation|
| |*If emotion menu is open:* Select emotion|
|**Left Index Trigger**|**In menu:** Select button|
| |**During teleoperation:** Control left gripper|
|**Right Index Trigger**|**In menu:** Select button|
| |**During teleoperation:** Control right gripper|
|**Left Controller position / orientation**|**During teleoperation:** Reachy's left arm end effector position / orientation|
|**Right Controller position / orientation**|**During teleoperation:** Reachy's right arm end effector position / orientation|
|**Headset orientation**|**During teleoperation:** Reachy's head orientation|
## Meta Quest

### Standard inputs

## Valve Index
{{< img "images/vr/use-teleop/meta-quest-mapping.png" 600x "Meta Quest controller mapping">}}

{{< img "images/vr/use-teleop/valve-index-mapping.png" 600x "Valve Index controller mapping">}}

|Name|Feature description |
|Input|Feature description |
|----|--------------------|
|**A** ***right***|**At robot teleoperation start:** Start robot teleoperation|
| |**During teleoperation:** Return to menu|
|**A** ***left***|**During teleoperation:** Open emotion menu (keep pressed) / Activate selected emotion (release)|
| |**When leaving teleoperation (right A pressed):** Lock robot position|
|**B** ***right***|**During teleoperation:** Mobile base boost|
|**A**|**At robot teleoperation start:** Start robot teleoperation|
| |**During teleoperation:** Return to menu|
|**B**|**During teleoperation:** Mobile base boost|
|**X**|**When leaving teleoperation (A pressed):** Lock robot position|
|**Left Thumbstick**|**During teleoperation:** Control mobile base translation|
|**Right Thumbstick**|**During teleoperation:** Control mobile base rotation|
| |*If emotion menu is open:* Select emotion|
|**Left Index Trigger**|**In menu:** Select button|
| |**During teleoperation:** Control left gripper|
|**Right Index Trigger**|**In menu:** Select button|
Expand All @@ -56,26 +37,11 @@ weight: "90"
|**Right Controller position / orientation**|**During teleoperation:** Reachy's right arm end effector position / orientation|
|**Headset orientation**|**During teleoperation:** Reachy's head orientation|

## HTC Vive
### Emergency stop combination

{{< img "images/vr/use-teleop/htc-vive-mapping.png" 600x "Valve Index controller mapping">}}
{{< img "images/vr/use-teleop/meta-quest-emergency-stop.png" 600x "Meta Quest controller emergency stop">}}

|Name|Feature description |
|Input|Feature description |
|----|--------------------|
|**Sandwich menu** ***right***|**At robot teleoperation start:** Start robot teleoperation|
| |**During teleoperation:** Return to menu|
|**Sandwich menu** ***left***|**During teleoperation:** Open emotion menu (keep pressed) / Activate selected emotion (release)|
| |**When leaving teleoperation (right menu pressed):** Lock robot position|
|**Left Trackpad**|**During teleoperation:** Control mobile base translation|
|**Right Trackpad**|**During teleoperation:** Control mobile base rotation|
| |*Center click:* Mobile base boost|
| |*If emotion menu is open:* Select emotion|
|**Left Index Trigger**|**In menu:** Select button|
| |**During teleoperation:** Control left gripper|
|**Right Index Trigger**|**In menu:** Select button|
| |**During teleoperation:** Control right gripper|
|**Left Controller position / orientation**|**During teleoperation:** Reachy's left arm end effector position / orientation|
|**Right Controller right position / orientation**|**During teleoperation:** Reachy's right arm end effector position / orientation|
|**Headset orientation**|**During teleoperation:** Reachy's head orientation|


|**A + right index trigger + right middle finger trigger**|**During teleoperation:** Emergency stop|
|**X + left index trigger + left middle finger trigger**|**During teleoperation:** Emergency stop|
29 changes: 29 additions & 0 deletions content/vr/use-teleop/emergency-stop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
title: "Emergency stop"
description: "Stop immediately sending commands to the robot"
lead: "Stop immediately sending commands to the robot"
date: 2023-07-26T09:01:49+02:00
lastmod: 2023-07-26T09:01:49+02:00
draft: false
images: []
type: docs
toc: true
weight: "140"
---

In case you feel like something unexpected is happening with the robot while you are teleoperating it in VR, you can stop immediately teleoperation rather than using the standard exit menu, which requires to wait for a few seconds.

> The VR application emergency stop does not replace the physical emergency stop button of the robot.
While pressing either:
- (A) + right index trigger + right middle finger trigger
*or*
- (X) + left index trigger + left middle finger trigger
you will activate the VR emergency stop.
{{< img "images/vr/use-teleop/meta-quest-emergency-stop.png" 400x "Meta Quest controller emergency stop">}}

The teleoperation application will immediately **stop sending commands** to the robot, **reduce the torque** values of the arms for a few seconds, then turn the robot **compliant**.

{{< img "images/vr/use-teleop/emergency-stop.png" 600x "Help panel in VR transition room">}} </br>

You will remain in the teleoperation view until you press (A) as usually to go back to the transition room.
4 changes: 2 additions & 2 deletions content/vr/use-teleop/messages.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ draft: false
images: []
type: docs
toc: true
weight: "110"
weight: "130"
---

During Reachy teleoperation, several messages can show up in front of view.
Expand All @@ -22,4 +22,4 @@ When possible, please consider acting to prevent these warnings from becoming er

Other messages may signal **errors**, which will lead to a fast dysfunction of the teleoperation. These messages are to take into account quickly, as you may not be controlling the robot properly anymore when they appear. These messages are displayed on a **red background**.

{{< warning icon="👉🏾" text="When error messages appear, <b>stop teleoperation</b> and act conquently to protect your robot. " >}}
{{< warning icon="👉🏾" text="When error messages appear, <b>stop teleoperation</b> and act appropriately depending on the error type. " >}}
39 changes: 39 additions & 0 deletions content/vr/use-teleop/mobile-base.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
title: "Control the mobile base"
description: "Use the mobile in the VR teleoperation application"
lead: "How to use the mobile base in the VR teleoperation application"
date: 2023-07-26T09:01:49+02:00
lastmod: 2023-07-26T09:01:49+02:00
draft: false
images: []
type: docs
toc: true
weight: "110"
---

## Control the mobile base
Use the **thumbstick/trackpad** to control the mobile base!
The **left controller controls the translation** of the mobile base, while the **right one controls the rotation**.

**Is there any security to prevent collision with objects?**

**Yes!** If you are too close to a wall or object, the LIDAR anti-collision safety unables the mobile base to go closer to the obstacle. The mobile base will therefore not move in this direction, but you can still go in other directions. You will get a warning message when the anti-collision safety is triggered.
[More information on the anti-collision safety](https://docs.pollen-robotics.com/sdk/mobile-base/safety/)

Nevertheless, this security is for the mobile base and won't prevent the robot's arms to collide with external objects, so be aware while teleoperating the robot.

*Please note very small objects won't be detected by the LIDAR sensor.*

**What is the forward direction of Reachy?**

The forward direction is aligned with the **forward direction of the mobile base**, meaning that giving a forward instruction to the robot will always lead the robot to go physically forward, no matter the direction you are looking to.

Check the actual direction of your commands using the **indicator** at the bottom: the white arrow shows you the direction command relative to your actual head orientation. If your head is correctly aligned with the mobile base forward direction, this arrow will point forward if giving a forward command with your left controller.
{{< img "images/vr/use-teleop/straight_forward.png" 600x "Forward direction looking straight">}}
{{< img "images/vr/use-teleop/head_on_side_forward.png" 600x "Forward direction looking on the left">}}

In the above images, the same forward command is sent from the left controller.
On the first image, the user is looking straight (the black arrow is located in the target view), so the white mobility arrow is pointing front.
On the second image, the user is looking on the left side (the target view is on the left of the black arrow), so the forward direction is pointing right, as it is the direction aligned with the mobile base forward direction.

*Note that these images are only for example, mobility is not available on virtual Reachy.*
50 changes: 50 additions & 0 deletions content/vr/use-teleop/motion-sickness.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
title: "Motion sickness options"
description: "Add visual effects to reduce motion sickness while using the teleoperation application"
lead: "Add visual effects to reduce motion sickness while using the teleoperation application"
date: 2023-07-26T09:01:49+02:00
lastmod: 2023-07-26T09:01:49+02:00
draft: false
images: []
type: docs
toc: true
weight: "120"
---

Motion sickness may occur using the VR teleoperation application.

We added a few options you can activate to reduce motion sickness, giving you a fix point in your field of view, or avoid peripheral movements when moving with the robot.

You have access to these options in the ***transition room***.

On the left of the mirror, open the **Settings** tab, and configure the motion sickness options before starting teleoperating the robot.

{{< img-center "images/vr/getting-started/vr-settings.png" 600x "" >}}


### Reticle
Display a reticle to give a fixed point in the field of view. By default, the reticle only appears when the mobile base is moving.

*Option:*
- **Always display reticle**: always display the reticle, even if the mobile base is not moving.

{{< img-center "images/vr/getting-started/reticle.png" 300x "" >}}


### Navigation effects

- **No effect**
- **Tunneling**: when moving the mobile base, a black tunneling will appear in your peripheral vision and reduce your field of view
- **Reduced screen**: when moving the mobile base, the size of the image will be reduced to let you see an artificial horizon behind it.

*Option:*
- **Activate effect on demande only**: during teleoperation, press one of the joysticks to activate/deactivate the occurence of the selected effect.

If used with *tunneling*, deactivate the effect will disable the tunneling when moving the mobile base, activate it will let it appear automatically.

If used with *reduced screen*, activate or deactivate the effect will let you manually reduce the size of the image.

|Tunneling effect|Reduced screen effect |
|----|--------------------|
|{{< img-center "images/vr/getting-started/tunneling.png" 300x "" >}}|{{< img-center "images/vr/getting-started/reduced-screen.png" 300x "" >}}
|
Loading

0 comments on commit f6adfae

Please sign in to comment.