Merge pull request #183 from ARISE-Initiative/update_docs

Update robosuite documentations

README.md

@@ -6,6 +6,8 @@
 
 -------
 ## Latest Updates
+[02/17/2021] **v1.2.0**: Added observable sensor models :eyes: and dynamics randomization :game_die:
+
 [12/17/2020] **v1.1.0**: Refactored infrastructure and standardized model classes for much easier environment prototyping :wrench:
 
 -------

docs/Makefile

@@ -12,6 +12,9 @@ BUILDDIR      = _build
 help:
 	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
 
+apidoc:
+	@sphinx-apidoc -T --force ../robosuite -o source
+
 .PHONY: help Makefile
 
 # Catch-all target: route all unknown targets to Sphinx using the new

docs/algorithms/demonstrations.md

@@ -2,7 +2,7 @@
 
 ## Collecting Human Demonstrations
 
-We provide teleoperation utilities that allow users to control the robots with input devices, such as the keyboard and the [SpaceMouse](https://www.3dconnexion.com/spacemouse_compact/en/). Such functionality allows us to collect a dataset of human demonstrations for learning. We provide an example script to illustrate how to collect demonstrations. Our [collect_human_demonstrations](robosuite/scripts/collect_human_demonstrations.py) script takes the following arguments:
+We provide teleoperation utilities that allow users to control the robots with input devices, such as the keyboard and the [SpaceMouse](https://www.3dconnexion.com/spacemouse_compact/en/). Such functionality allows us to collect a dataset of human demonstrations for learning. We provide an example script to illustrate how to collect demonstrations. Our [collect_human_demonstrations](https://github.com/ARISE-Initiative/robosuite/blob/master/robosuite/scripts/collect_human_demonstrations.py) script takes the following arguments:
 
 - `directory:` path to a folder for where to store the pickle file of collected demonstrations
 - `environment:` name of the environment you would like to collect the demonstrations for
@@ -38,14 +38,14 @@ Note that the rendering window must be active for these commands to work.
 
 ## Replaying Human Demonstrations
 
-We have included an example script that illustrates how demonstrations can be loaded and played back. Our [playback_demonstrations_from_hdf5](robosuite/scripts/playback_demonstrations_from_hdf5.py) script selects demonstration episodes at random from a demonstration pickle file and replays them.
+We have included an example script that illustrates how demonstrations can be loaded and played back. Our [playback_demonstrations_from_hdf5](https://github.com/ARISE-Initiative/robosuite/blob/master/robosuite/scripts/playback_demonstrations_from_hdf5.py) script selects demonstration episodes at random from a demonstration pickle file and replays them.
 
 
 ## Existing Datasets
 
 We have included some sample demonstrations for each task at `models/assets/demonstrations`.
 
-Our twin project [RoboTurk](http://roboturk.stanford.edu) has also collected pilot datasets of more than a thousand demonstrations for two tasks in our suite via crowdsourcing. You can find detailed information about the RoboTurk datasets [here](docs/demonstrations.md#roboturk-dataset). 
+Our twin project [RoboTurk](http://roboturk.stanford.edu) has also collected pilot datasets of more than a thousand demonstrations for two tasks in our suite via crowdsourcing. You can find detailed information about the RoboTurk datasets [here](roboturk).
 
 
 ## Structure of collected demonstrations
@@ -81,4 +81,9 @@ The reason for storing mujoco states instead of raw observations is to make it e
 
 ## Using Demonstrations for Learning
 
-[Several](https://arxiv.org/abs/1802.09564) [prior](https://arxiv.org/abs/1807.06919) [works](https://arxiv.org/abs/1804.02717) have demonstrated the effectiveness of altering the start state distribution of training episodes for learning RL policies. We provide a generic utility for setting various types of learning curriculums which dictate how to sample from demonstration episodes when doing an environment reset. For more information see the `DemoSamplerWrapper` class. We have provided an example of how to use this wrapper along with a demonstration pickle file in the [demo_learning_curriculum](robosuite/scripts/demo_learning_curriculum.py) script.
+[Several](https://arxiv.org/abs/1802.09564) [prior](https://arxiv.org/abs/1807.06919) [works](https://arxiv.org/abs/1804.02717) have demonstrated the effectiveness of altering the start state distribution of training episodes for learning RL policies. We provide a generic utility for setting various types of learning curriculums which dictate how to sample from demonstration episodes when doing an environment reset. For more information see the `DemoSamplerWrapper` class.
+
+## Warnings
+We have verified that deterministic action playback works specifically when playing back demonstrations on the *same machine* that the demonstrations were originally collected upon. However, this means that deterministic action playback is NOT guaranteed (in fact, very unlikely) to work across platforms or even across different machines using the same OS.
+
+While action playback trajectories are quite similar even if not completely identical to the original collected state trajectories, they do tend to drift over time, and should not be relied upon to accurately replicate demonstrations. Instead, we recommend directly setting states to reproduce the collected trajectories, as shown in [playback_demonstrations_from_hdf5](https://github.com/ARISE-Initiative/robosuite/blob/master/robosuite/scripts/playback_demonstrations_from_hdf5.py).

docs/algorithms/roboturk.md

@@ -2,9 +2,9 @@
 
 [RoboTurk](https://roboturk.stanford.edu/) is a crowdsourcing platform developed in order to enabled collecting large-scale manipulation datasets. Below, we describe RoboTurk datasets that are compatible with robosuite.
 
-## Updated Datasets compatible with v1.0
+## Updated Datasets compatible with v1.0+
 
-We are currently in the process of reformatting the demonstrations. These datasets will be made available soon.
+We are currently in the process of organizing a standardized dataset for our benchmarking tasks, which will be made available soon and compatible with v1.2.0+. In the meantime, we have provided a [small-scale dataset](https://drive.google.com/drive/folders/1LLkuFnRdqQ6xn1cYzkbJUs_DreaAvN7i?usp=sharing) of expert demonstrations on two of our tasks.
 
 ## Original Datasets compatible with v0.3
 

docs/algorithms/sim2real.md

@@ -0,0 +1,157 @@
+# Sim-to-Real Transfer
+This page covers the randomization techniques to narrow the reality gap of our robotics simulation. These techniques, which concerns about [visual observations](#visuals), [system dynamics](#dynamics), and [sensors](#sensors), are employed to improve the efficacy of transferring our simulation-trained models to the real world.
+
+
+## Visuals
+
+It is well shown that randomizing the visuals in simulation can play an important role in sim2real applications. **robosuite** provides various `Modder` classes to control different aspects of the visual environment. This includes:
+
+- `CameraModder`: Modder for controlling camera parameters, including FOV and pose
+- `TextureModder`: Modder for controlling visual objects' appearances, including texture and material properties
+- `LightingModder`: Modder for controlling lighting parameters, including light source properties and pose
+
+Each of these Modders can be used by the user to directly override default simulation settings, or to randomize their respective properties mid-sim. We provide [demo_domain_randomization.py](../demos.html#domain-randomization) to showcase all of these modders being applied to randomize an environment during every sim step.
+
+
+## Dynamics
+
+In order to achieve reasonable runtime speeds, many physics simulation platforms often must simplify the underlying physics model. Mujoco is no different, and as a result, many parameters such as friction, damping, and contact constraints do not fully capture real-world dynamics.
+
+To better compensate for this, **robosuite** provides the `DynamicsModder` class, which can control individual dynamics parameters for each model within an environment. Theses parameters are sorted by element group, and briefly described below (for more information, please see [Mujoco XML Reference](http://www.mujoco.org/book/XMLreference.html)):
+
+#### Opt (Global) Parameters
+- `density`: Density of the medium (i.e.: air)
+- `viscosity`: Viscosity of the medium (i.e.: air)
+
+#### Body Parameters
+- `position`: (x, y, z) Position of the body relative to its parent body
+- `quaternion`: (qw, qx, qy, qz) Quaternion of the body relative to its parent body
+- `inertia`: (ixx, iyy, izz) diagonal components of the inertia matrix associated with this body
+- `mass`: mass of the body
+
+#### Geom Parameters
+- `friction`: (sliding, torsional, rolling) friction values for this geom
+- `solref`: (timeconst, dampratio) contact solver values for this geom
+- `solimp`: (dmin, dmax, width, midpoint, power) contact solver impedance values for this geom
+
+#### Joint parameters
+- `stiffness`: Stiffness for this joint
+- `frictionloss`: Friction loss associated with this joint
+- `damping`: Damping value for this joint
+- `armature`: Gear inertia for this joint
+
+This `DynamicsModder` follows the same basic API as the other `Modder` classes, and allows per-parameter and per-group randomization enabling. Apart from randomization, this modder can also be instantiated to selectively modify values at runtime. A brief example is given below:
+
+```python
+import robosuite as suite
+from robosuite.utils.mjmod import DynamicsModder
+import numpy as np
+
+# Create environment and modder
+env = suite.make("Lift", robots="Panda")
+modder = DynamicsModder(sim=env.sim, random_state=np.random.RandomState(5))
+
+# Define function for easy printing
+cube_body_id = env.sim.model.body_name2id(env.cube.root_body)
+cube_geom_ids = [env.sim.model.geom_name2id(geom) for geom in env.cube.contact_geoms]
+
+def print_params():
+    print(f"cube mass: {env.sim.model.body_mass[cube_body_id]}")
+    print(f"cube frictions: {env.sim.model.geom_friction[cube_geom_ids]}")
+    print()
+
+# Print out initial parameter values
+print("INITIAL VALUES")
+print_params()
+
+# Modify the cube's properties
+modder.mod(env.cube.root_body, "mass", 5.0)                                # make the cube really heavy
+for geom_name in env.cube.contact_geoms:
+    modder.mod(geom_name, "friction", [2.0, 0.2, 0.04])           # greatly increase the friction
+modder.update()                                                   # make sure the changes propagate in sim
+
+# Print out modified parameter values
+print("MODIFIED VALUES")
+print_params()
+
+# We can also restore defaults (original values) at any time
+modder.restore_defaults()
+
+# Print out restored initial parameter values
+print("RESTORED VALUES")
+print_params()
+```
+
+Running [demo_domain_randomization.py](../demos.html#domain-randomization) is another method for demo'ing (albeit an extreme example of) this functionality.
+
+Note that the modder already has some sanity checks in place to prevent presumably undesired / non-sensical behavior, such as adding damping / frictionloss to a free joint or setting a non-zero stiffness value to a joint that is normally non-stiff to begin with.
+
+
+## Sensors
+
+By default, Mujoco sensors are deterministic and delay-free, which is often an unrealistic assumption to make in the real world. To better close this domain gap, **robosuite** provides a realistic, customizable interface via the [Observable](../source/robosuite.utils.html#module-robosuite.utils.observables) class API. Observables model realistic sensor sampling, in which ground truth data is sampled (`sensor`), passed through a corrupting function (`corrupter`), and then finally passed through a filtering function (`filter`). Moreover, each observable has its own `sampling_rate` and `delayer` function which simulates sensor delay. While default values are used to instantiate each observable during environment creation, each of these components can be modified by the user at runtime using `env.modify_observable(...)` . Moreover, each observable is assigned a modality, and are grouped together in the returned observation dictionary during the `env.step()` call. For example, if an environment consists of camera observations and a single robot's proprioceptive observations, the observation dict structure might look as follows:
+
+```python
+{
+    "frontview_image": np.array(...),    # this has modality "image"
+    "frontview_depth": np.array(...),    # this has modality "image"
+    "robot0_joint_pos": np.array(...),   # this has modality "robot0_proprio"
+    "robot0_gripper_pos": np.array(...), # this has modality "robot0_proprio"
+    "image-state": np.array(...),           # this is a concatenation of all image observations
+    "robot0_proprio-state": np.array(...),  # this is a concatenation of all robot0_proprio observations
+}
+```
+
+Note that for memory efficiency the `image-state` is not returned by default (this can be toggled in `robosuite/utils/macros.py`).
+
+We showcase how the `Observable` functionality can be used to model sensor corruption and delay via [demo_sensor_corruption.py](../demos.html#sensor-realism). We also highlight that each of the `sensor`, `corrupter`, and `filter` functions can be arbitrarily specified to suit the end-user's usage. For example, a common use case for these observables is to keep track of sampled values from a sensor operating at a higher frequency than the environment step (control) frequency. In this case, the `filter` function can be leveraged to keep track of the real-time sensor values as they're being sampled. We provide a minimal script showcasing this ability below:
+
+```python
+import robosuite as suite
+import numpy as np
+from robosuite.utils.buffers import RingBuffer
+
+# Create env instance
+control_freq = 10
+env = suite.make("Lift", robots="Panda", has_offscreen_renderer=False, use_camera_obs=False, control_freq=control_freq)
+
+# Define a ringbuffer to store joint position values
+buffer = RingBuffer(dim=env.robots[0].robot_model.dof, length=10)
+
+# Create a function that we'll use as the "filter" for the joint position Observable
+# This is a pass-through operation, but we record the value every time it gets called
+# As per the Observables API, this should take in an arbitrary numeric and return the same type / shape
+def filter_fcn(corrupted_value):
+    # Record the inputted value
+    buffer.push(corrupted_value)
+    # Return this value (no-op performed)
+    return corrupted_value
+
+# Now, let's enable the joint position Observable with this filter function
+env.modify_observable(
+    observable_name="robot0_joint_pos",
+    attribute="filter",
+    modifier=filter_fcn,
+)
+
+# Let's also increase the sampling rate to showcase the Observable's ability to update multiple times per env step
+obs_sampling_freq = control_freq * 4
+env.modify_observable(
+    observable_name="robot0_joint_pos",
+    attribute="sampling_rate",
+    modifier=obs_sampling_freq,
+)
+
+# Take a single environment step with positive joint velocity actions
+arm_action = np.ones(env.robots[0].robot_model.dof) * 1.0
+gripper_action = [1]
+action = np.concatenate([arm_action, gripper_action])
+env.step(action)
+
+# Now we can analyze what values were recorded
+np.set_printoptions(precision=2)
+print(f"\nPolicy Frequency: {control_freq}, Observable Sampling Frequency: {obs_sampling_freq}")
+print(f"Number of recorded samples after 1 policy step: {buffer._size}\n")
+for i in range(buffer._size):
+    print(f"Recorded value {i}: {buffer.buf[i]}")
+```

docs/demos.md

@@ -46,6 +46,11 @@ The `demo_domain_randomization.py` script showcases our domain randomization fun
 
 ![collection of grippers](images/domain_randomization.png)
 
+
+### Sensor Realism
+The `demo_sensor_corruption.py` script shows how the [Observable](modules/sensors) functionality can be used to model sensor corruption and delay. The [Observable](modules/sensors) class provides easy-to-use interface for simulating real-world sensor noises, variable sensor delays, and sampling frequencies of realistic sensors. These techniques are useful for improving the generalization and robustness of the policies trained on robosuite, facilitating sim2real transfer to the real hardware. For example, see Appendix B of [Zhu et al. RSS'2018](https://arxiv.org/abs/1802.09564) and Section V of [Tan et al. RSS'2018](https://arxiv.org/abs/1804.10332).
+
+
 ### Gripper Selection
 The `demo_gripper_selection.py` script shows you how to select gripper for an environment. This is controlled by `gripper_type` keyword argument. The set of all grippers is defined by the global variable `robosuite.ALL_GRIPPERS`.
 

docs/index.rst

@@ -26,6 +26,7 @@ Welcome to robosuite's documentation!
    modules/environments
    modules/sensors
    modules/devices
+   modules/renderers
 
 .. toctree::
    :maxdepth: 1
@@ -40,8 +41,8 @@ Welcome to robosuite's documentation!
    :maxdepth: 1
    :caption: Modeling API
 
+   modeling/mujoco_model
    modeling/robot_model
-   modeling/gripper_model
    modeling/object_model
    modeling/arena
    modeling/task
@@ -58,6 +59,7 @@ Welcome to robosuite's documentation!
 
    algorithms/benchmarking
    algorithms/demonstrations
+   algorithms/sim2real
    algorithms/roboturk
 
 .. toctree::

docs/modeling/arena.rst

@@ -1,11 +1,15 @@
 Arena
 =====
 
+The ``Arena`` class serves as a base model for building the simulation environment.
+By default, this includes a ground plane and visual walls, and child classes extend this
+to additionally include other objects, e.g., a table or bins.
+
 Base Arena
 ----------
 
 .. autoclass:: robosuite.models.arenas.arena.Arena
 
   .. automethod:: __init__
   .. automethod:: set_origin
-  .. automethod:: add_pos_indicator
+  .. automethod:: set_camera

docs/modeling/mujoco_model.rst

@@ -0,0 +1,38 @@
+Mujoco Model
+============
+
+The ``MujocoModel`` class is the foundational class from which all other model classes extend from in robosuite. This class represents a standardized API for all models used in simulation and is the core modeling component that other model classes build upon. The ``MujocoXMLModel`` is an extension of this class that represents models based on an XML file.
+
+Base Mujoco Model
+-----------------
+
+.. autoclass:: robosuite.models.base.MujocoModel
+
+  .. automethod:: correct_naming
+  .. automethod:: set_sites_visibility
+  .. automethod:: exclude_from_prefixing
+  .. autoproperty:: name
+  .. autoproperty:: naming_prefix
+  .. autoproperty:: root_body
+  .. autoproperty:: bodies
+  .. autoproperty:: joints
+  .. autoproperty:: actuators
+  .. autoproperty:: sites
+  .. autoproperty:: sensors
+  .. autoproperty:: contact_geoms
+  .. autoproperty:: visual_geoms
+  .. autoproperty:: important_geoms
+  .. autoproperty:: important_sites
+  .. autoproperty:: important_sensors
+  .. autoproperty:: bottom_offset
+  .. autoproperty:: top_offset
+  .. autoproperty:: horizontal_radius
+
+
+XML Mujoco Model
+----------------
+
+.. autoclass:: robosuite.models.base.MujocoXMLModel
+
+  .. autoproperty:: base_offset
+  .. autoproperty:: contact_geom_rgba