Multi object search (#363)

* Add zenodo DOI badge (#396) * Update README.md (#406) * Update README.md * Update projects/python/simulation/SMPL+D_human_models/README.md Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com> * Update README.md * Update README.md --------- Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com> * add prev. commits * Update src/opendr/control/multi_object_search/README.md Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com> * env default value * removed default argument --------- Co-authored-by: Nikolaos Passalis <passalis@users.noreply.github.com> Co-authored-by: charsyme <63857415+charsyme@users.noreply.github.com> Co-authored-by: Kostas Tsampazis <27914645+tsampazk@users.noreply.github.com> Co-authored-by: ad-daniel <44834743+ad-daniel@users.noreply.github.com>
opendr-eu · Feb 16, 2023 · eb7d6ed · eb7d6ed
1 parent 7871c00
commit eb7d6ed
Show file tree

Hide file tree

Showing 39 changed files with 4,864 additions and 2 deletions.
diff --git a/.github/workflows/test_packages.yml b/.github/workflows/test_packages.yml
@@ -57,6 +57,7 @@ jobs:
           #- control/mobile_manipulation
           #- control/single_demo_grasp
           #- planning/end_to_end_planning
+          - control/multi_object_search
     runs-on: ${{ matrix.os }}
     steps:
     - uses: actions/checkout@v3
@@ -116,6 +117,7 @@ jobs:
           - control/mobile_manipulation
           - control/single_demo_grasp
           - planning/end_to_end_planning
+          - control/multi_object_search
     runs-on: ${{ matrix.os }}
     steps:
     - name: Set up Python 3.8

diff --git a/.github/workflows/tests_suite.yml b/.github/workflows/tests_suite.yml
@@ -87,6 +87,7 @@ jobs:
           - control/mobile_manipulation
           - control/single_demo_grasp
           - planning/end_to_end_planning
+          - control/multi_object_search
     runs-on: ${{ matrix.os }}
     steps:
     - uses: actions/checkout@v2
@@ -192,6 +193,7 @@ jobs:
           # - perception/object_detection_3d # passes, but disabled due to free() crash
           - perception/facial_expression_recognition
           - simulation/human_model_generation
+          - control/multi_object_search
           #- control/mobile_manipulation
           #- control/single_demo_grasp
           #- planning/end_to_end_planning
@@ -277,6 +279,7 @@ jobs:
           # - perception/object_detection_3d # passes, but disabled due to free() crash
           - perception/facial_expression_recognition
           - simulation/human_model_generation
+          - control/multi_object_search
           #- control/mobile_manipulation
           #- control/single_demo_grasp
           #- planning/end_to_end_planning
@@ -383,6 +386,7 @@ jobs:
           - control/mobile_manipulation
           - control/single_demo_grasp
           - planning/end_to_end_planning
+          - control/multi_object_search
     runs-on: ubuntu-20.04
     steps:
     - name: Download artifact

diff --git a/.github/workflows/tests_suite_develop.yml b/.github/workflows/tests_suite_develop.yml
@@ -88,6 +88,7 @@ jobs:
           - control/mobile_manipulation
           - control/single_demo_grasp
           - planning/end_to_end_planning
+          - control/multi_object_search
     runs-on: ${{ matrix.os }}
     steps:
     - uses: actions/checkout@v2
@@ -199,6 +200,7 @@ jobs:
           #- control/mobile_manipulation
           #- control/single_demo_grasp
           #- planning/end_to_end_planning
+          - control/multi_object_search
     runs-on: ubuntu-20.04
     steps:
     - uses: actions/checkout@v3
@@ -288,6 +290,7 @@ jobs:
           #- control/mobile_manipulation
           #- control/single_demo_grasp
           #- planning/end_to_end_planning
+          - control/multi_object_search
     runs-on: ubuntu-20.04
     steps:
     - uses: actions/checkout@v3
@@ -395,6 +398,7 @@ jobs:
           - control/mobile_manipulation
           - control/single_demo_grasp
           - planning/end_to_end_planning
+          - control/multi_object_search
     runs-on: ubuntu-20.04
     steps:
     - name: Download artifact

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -13,6 +13,7 @@ Released on XX, XX, 2023.
     - Added C API implementations of forward pass of X3D 2D Activity Recognition tool ([#383](https://github.com/opendr-eu/opendr/pull/383)).
     - Added C API implementations of forward pass of Progressive Spatiotemporal GCN Skeleton-based Action Recognition tool ([#383](https://github.com/opendr-eu/opendr/pull/383)).
     - Added Binary High Resolution Analysis tool ([#402](https://github.com/opendr-eu/opendr/pull/402)).
+    - Added Multi-Object-Search tool ([#363](https://github.com/opendr-eu/opendr/pull/363))
   - Enhancements:
     - Added support in C API for detection target structure and vector of detections ([#352](https://github.com/opendr-eu/opendr/pull/352))
     - Added support in C API for tensor structure and vector of tensors ([#383](https://github.com/opendr-eu/opendr/pull/383))

diff --git a/README.md b/README.md
@@ -22,6 +22,8 @@ ______________________________________________________________________
 [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 [![Test Suite (master)](https://github.com/opendr-eu/opendr/actions/workflows/tests_suite.yml/badge.svg)](https://github.com/opendr-eu/opendr/actions/workflows/tests_suite.yml)
 [![Test Packages](https://github.com/opendr-eu/opendr/actions/workflows/test_packages.yml/badge.svg)](https://github.com/opendr-eu/opendr/actions/workflows/test_packages.yml)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7540781.svg)](https://doi.org/10.5281/zenodo.7540781)
+
 </div>
 
 ## About

diff --git a/docs/reference/multi_object_search.md b/docs/reference/multi_object_search.md
@@ -0,0 +1,254 @@
+# multi_object_search module
+
+The *multi_object_search* module contains the *MultiObjectSearchRLLearner* class, which inherits from the abstract class *LearnerRL*.
+
+### Class MultiObjectSearchRLLearner
+Bases: `engine.learners.LearnerRL`
+
+The *MultiObjectSearchRLLearner* class is an RL agent that can be used to train wheeled robots for combining short-horizon control with long horizon reasoning into a single policy.
+Originally published in [[1]](#multi_object_search), Learning Long-Horizon Robot Exploration Strategies for Multi-Object Search in Continuous Action Spaces,(https://arxiv.org/abs/2205.11384).
+
+The [MultiObjectSearchRLLearner](/src/opendr/control/multi_object_search/multi_object_search_learner.py) class has the following public methods:
+
+#### `MultiObjectSearchRLLearner` constructor
+MultiObjectSearchRLLearner(self, env, lr, ent_coef, clip_range, gamma, n_steps, n_epochs, iters, batch_size, lr_schedule, backbone, checkpoint_after_iter, temp_path, device, seed, config_filename, nr_evaluations)
+
+Constructor parameters:
+
+- **env**: *gym.Env*\
+  Reinforcement learning environment to train or evaluate the agent on.
+- **lr**: *float, default=1e-5*\
+  Specifies the initial learning rate to be used during training.
+- **ent_coef**: *float, default=0.005*\
+  Specifies the entropy coefficient used as additional loss penalty during training.
+- **clip_range**: *float, default=0.1*\
+  Specifies the clipping parameter for PPO.
+- **gamma**: *float, default=0.99*\
+  Specifies the discount factor during training.
+- **n_steps**: *int, default=2048*\
+  Specifies the number of steps to run for each environment per update during training.
+- **n_epochs**: *int, default=4*\
+  Specifies the number of epochs when optimizing the surrogate loss during training.
+- **iters**: *int, default=6_000_000*\
+  Specifies the number of steps the training should run for.
+- **batch_size**: *int, default=64*\
+  Specifies the batch size during training.
+- **lr_schedule**: *{'', 'linear'}, default='linear'*\
+  Specifies the learning rate scheduler to use. Empty to use a constant rate.
+  Currently not implemented.
+- **backbone**: *{'MultiInputPolicy'}, default='MultiInputPolicy'*\
+  Specifies the architecture for the RL agent.
+- **checkpoint_after_iter**: *int, default=20_000*\
+  Specifies per how many training steps a checkpoint should be saved.
+  If it is set to 0 no checkpoints will be saved.
+- **temp_path**: *str, default=''*\
+  Specifies a path where the algorithm stores log files and saves checkpoints.
+- **device**: *{'cpu', 'cuda'}, default='cuda'*\
+  Specifies the device to be used.
+- **seed**: *int, default=None*\
+  Random seed for the agent.
+  If None a random seed will be used.
+- **config_filename**: *str, default=''*\
+  Specifies the configuration file with important settings for the Simulator and PPO.
+- **nr_evaluations**: *int, default=75*\
+  Number of episodes to evaluate over.
+
+#### `MultiObjectSearchRLLearner.fit`
+```python
+MultiObjectSearchRLLearner.fit(self, env, logging_path, silent, verbose)
+```
+
+Trains the agent on the environment.
+
+Parameters:
+
+- **env**: *gym.Env, default=None*\
+  If specified use this env to train.
+- **logging_path**: *str, default=''*\
+  Path for logging and checkpointing.
+
+
+#### `MultiObjectSearchRLLearner.eval`
+```python
+MultiObjectSearchRLLearner.eval(self, env, name_prefix, name_scene, nr_evaluations, deterministic_policy)
+```
+Evaluates the agent on the specified environment.
+
+Parameters:
+
+- **env**: *gym.Env*\
+  Environment to evaluate on.
+- **name_prefix**: *str, default=''*\
+  Name prefix for all logged variables.
+- **name_scene**: *str, default=''\
+  Name of the iGibson scene.
+- **nr_evaluations**: *int, default=75*\
+  Number of episodes to evaluate over.
+- **deterministic_policy**: *bool, default=False*\
+  Use deterministic or stochastic policy.
+
+
+#### `MultiObjectSearchRLLearner.save`
+```python
+MultiObjectSearchRLLearner.save(self, path)
+```
+Saves the model in the path provided.
+
+Parameters:
+
+- **path**: *str*\
+  Path to save the model, including the filename.
+
+
+#### `MultiObjectSearchRLLearner.load`
+```python
+MultiObjectSearchRLLearner.load(self, path)
+```
+Loads a model from the path provided.
+
+Parameters:
+
+- **path**: *str*\
+  Path of the model to be loaded.
+
+
+  #### `MultiObjectSearchRLLearner.infer`
+  ```python
+  MultiObjectSearchRLLearner.infer(self, batch, deterministic)
+  ```
+  Loads a model from the path provided.
+
+  Parameters:
+
+  - **batch**: *int*\
+    Number of samples to infer.
+  - **deterministic**: *bool, default=False*\
+    Use deterministic or stochastic policy.
+
+
+
+#### Simulation Setup
+The repository uses the iGibson Simulator as well as Stable-Baselines3 as external libraries.
+
+This means that for the training environment to run, it relies on using iGibson scenes.
+For that it is necessary to download the iGibson scenes.
+A script is provided in [multi_object_search]
+(/src/opendr/control/multi_object_search/requirements_installations.py)
+To download he iGibson and the inflated traversability maps, please execute the following script and accept the agreement.
+
+```sh
+python requirements_installations.py
+````
+
+The iGibson dataset requires a valid license, which needs to be added manually.
+The corresponding link can be found [here](https://docs.google.com/forms/d/e/1FAIpQLScPwhlUcHu_mwBqq5kQzT2VRIRwg_rJvF0IWYBk_LxEZiJIFg/viewform).
+In order to validate the iGibson dataset, copy the igibson.key file into the igibson/data/ folder.
+For more information please have a look on the official website: https://stanfordvl.github.io/iGibson/dataset.html
+
+##### Visualization
+To visualize the egocentric maps and their corresponding static map, add the flag `show_map=true` in`config.yaml`.
+
+
+#### Examples
+* **Training and evaluation in the iGibson environment on a Multi Object Task**
+As described above, follow the download instructions.
+  ```python
+    import torch
+    from typing import Callable
+    from opendr.control.multi_object_search import MultiObjectSearchRLLearner
+    from opendr.control.multi_object_search import MultiObjectEnv
+    from opendr.control.multi_object_search.algorithm.SB3.vec_env import VecEnvExt
+    from pathlib import Path
+    from igibson.utils.utils import parse_config
+
+
+    def main():
+        def make_env(rank: int, seed: int = 0, data_set=[]) -> Callable:
+
+            def _init() -> MultiObjectEnv:
+                env_ = MultiObjectEnv(
+                    config_file=CONFIG_FILE,
+                    scene_id=data_set[rank],
+                    mix_sample=mix_sample[data_set[rank]]
+                    )
+                env_.seed(seed + rank)
+                return env_
+            return _init
+
+        main_path = Path(__file__).parent
+        logpath = f"{main_path}/logs/demo_run"
+        CONFIG_FILE = str(f"{main_path}/best_defaults.yaml")
+
+        mix_sample = {'Merom_0_int': False}
+        train_set = ['Merom_0_int']
+
+        env = VecEnvExt([make_env(0, data_set=train_set)])
+        device = "cuda" if torch.cuda.is_available() else "cpu"
+
+        config = parse_config(CONFIG_FILE)
+
+        agent = MultiObjectSearchRLLearner(env, device=device, iters=config.get('train_iterations', 500),temp_path=logpath,config_filename=CONFIG_FILE)
+
+        # start training
+        agent.fit(env)
+
+        # evaluate on finding 6 objects on one test scene
+        scene = "Benevolence_1_int"
+        metrics = agent.eval(env,name_prefix='Multi_Object_Search', name_scene=scene, nr_evaluations= 75,deterministic_policy = False)
+
+        print(f"Success-rate for {scene} : {metrics['metrics']['success']} \nSPL for {scene} : {metrics['metrics']['spl']}")
+
+
+    if __name__ == '__main__':
+        main()
+  ```
+
+* **Evaluate a pretrained model**
+
+  ```python
+    import torch
+    from opendr.control.multi_object_search import MultiObjectSearchRLLearner
+    from opendr.control.multi_object_search import MultiObjectEnv
+    from pathlib import Path
+    from igibson.utils.utils import parse_config
+
+    def main():
+      main_path = Path(__file__).parent
+      logpath = f"{main_path}/logs/demo_run"
+      # best_defaults.yaml contains important settings. (see above)
+      CONFIG_FILE = str(f"{main_path}/best_defaults.yaml")
+
+      env = MultiObjectEnv(config_file=CONFIG_FILE, scene_id="Benevolence_1_int")
+
+      device = "cuda" if torch.cuda.is_available() else "cpu"
+
+      config = parse_config(CONFIG_FILE)
+
+      agent = MultiObjectSearchRLLearner(env, device=device, iters=config.get('train_iterations', 500),temp_path=logpath,config_filename=CONFIG_FILE)
+
+      # evaluate on finding 6 objects on all test scenes
+      eval_scenes = ['Benevolence_1_int', 'Pomaria_2_int', 'Benevolence_2_int', 'Wainscott_0_int', 'Beechwood_0_int',
+                'Pomaria_1_int', 'Merom_1_int']
+
+      agent.load("pretrained")
+
+      deterministic_policy = config.get('deterministic_policy', False)
+
+      for scene in eval_scenes:
+        metrics = agent.eval(env,name_prefix='Multi_Object_Search', name_scene=scene, nr_evaluations= 75,\
+                  deterministic_policy = deterministic_policy)
+
+        print(f"Success-rate for {scene} : {metrics['metrics']['success']} \nSPL for {scene} : {metrics['metrics']['spl']}")
+
+
+    if __name__ == '__main__':
+        main()
+  ```
+
+#### Notes
+
+The iGibson simulator might crash, when evaluating multiple environments while using the gui mode (in .yaml file).
+
+#### References
+<a name="multi-object-search" href="https://arxiv.org/abs/2205.11384">[1]</a> Learning Long-Horizon Robot Exploration Strategies for Multi-Object Search in Continuous Action Spaces,
+[arXiv](https://arxiv.org/abs/2205.11384).
diff --git a/projects/control/multi_object_search/README.md b/projects/control/multi_object_search/README.md
@@ -0,0 +1,28 @@
+# OpenDR multi object search demo
+<div align="left">
+  <a href="https://opensource.org/licenses/Apache-2.0">
+    <img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" height="20">
+  </a>
+</div>
+
+Live demo of multi object search using the [OpenDR toolkit](https://opendr.eu).
+
+
+## Set-up
+Follow the setup described for the [multi_object_search tool](/docs/reference/multi_object_search.md).
+
+## Running the example
+Multi Object Search task in the iGibson simulation can be run as follows:
+```bash
+python multi_object_search_demo.py
+```
+
+The demo either executes a training or evaluation instance. By default it will evaluate all evaluation-scenes for 75 episodes sequentially.
+The following will list the most important flags in
+```bash
+  best_defaults.yaml
+```
+Uncomment or comment (#) the desired robot by either using LoCoBot or Fetch. Training can be specified by `evaluation = false`(true by default).
+
+## Acknowledgement
+This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 871449 (OpenDR). This publication reflects the authors’ views only. The European Commission is not responsible for any use that may be made of the information it contains.