Release

.gitignore

@@ -0,0 +1,114 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+mjkey.txt
+MUJOCO_LOG.TXT
+
+data
+
+*~
+*.*~
+
+.DS_Store

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2019 OpenAI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,92 @@
+**Status:** Archive (code is provided as-is, no updates expected)
+
+# Safety Starter Agents
+
+A companion repo to the paper "Benchmarking Safe Exploration in Deep Reinforcement Learning," containing a variety of unconstrained and constrained RL algorithms.
+
+This repo contains the implementations of PPO, TRPO, PPO-Lagrangian, TRPO-Lagrangian, and CPO used to obtain the results in the "Benchmarking Safe Exploration" paper, as well as experimental implementations of SAC and SAC-Lagrangian not used in the paper.
+
+Note that the PPO implementations here follow the convention from [Spinning Up](https://spinningup.openai.com) rather than [Baselines](https://www.github.com/openai/baselines): they use the early stopping trick, omit observation and reward normalization, and do not use the clipped value loss, among other potential diffs. As a result, while it is easy to fairly compare this PPO to this TRPO, it is not the strongest PPO implementation (in the sense of sample efficiency) and can be improved on substantially.
+
+## Supported Platforms
+
+This package has been tested on Mac OS Mojave and Ubuntu 16.04 LTS, and is probably fine for most recent Mac and Linux operating systems. 
+
+Requires **Python 3.6 or greater.**  
+
+## Installation
+
+To install this package:
+
+```
+git clone https://github.com/openai/safety-starter-agents.git
+
+cd safety-starter-agents
+
+pip install -e .
+```
+
+**Warning:** Installing this package does **not** install Safety Gym. If you want to use the algorithms in this package to train agents on onstrained RL environments, make sure to install Safety Gym according to the instructions on the [Safety Gym repo](https://www.github.com/openai/safety-gym).
+
+
+## Getting Started
+
+**Example Script:** To run PPO-Lagrangian on the `Safexp-PointGoal1-v0` environment from Safety Gym, using neural networks of size (64,64):
+
+```
+from safe_rl import ppo_lagrangian
+import gym, safety_gym
+
+ppo_lagrangian(
+	env_fn = lambda : gym.make('Safexp-PointGoal1-v0'),
+	ac_kwargs = dict(hidden_sizes=(64,64))
+	)
+
+```
+
+
+**Reproduce Experiments from Paper:** To reproduce an experiment from the paper, run:
+
+```
+cd /path/to/safety-starter-agents/scripts
+python experiment.py --algo ALGO --task TASK --robot ROBOT --seed SEED 
+	--exp_name EXP_NAME --cpu CPU
+```
+
+where 
+
+* `ALGO` is in `['ppo', 'ppo_lagrangian', 'trpo', 'trpo_lagrangian', 'cpo']`.
+* `TASK` is in `['goal1', 'goal2', 'button1', 'button2', 'push1', 'push2']` .
+* `ROBOT` is in `['point', 'car', 'doggo']`.
+* `SEED` is an integer. In the paper experiments, we used seeds of 0, 10, and 20, but results may not reproduce perfectly deterministically across machines.
+* `CPU` is an integer for how many CPUs to parallelize across.
+
+`EXP_NAME` is an optional argument for the name of the folder where results will be saved. The save folder will be placed in `/path/to/safety-starter-agents/data`. 
+
+
+**Plot Results:** Plot results with:
+
+```
+cd /path/to/safety-starter-agents/scripts
+python plot.py data/path/to/experiment
+```
+
+**Watch Trained Policies:** Test policies with:
+
+```
+cd /path/to/safety-starter-agents/scripts
+python test_policy.py data/path/to/experiment
+```
+
+
+## Cite the Paper
+
+If you use Safety Starter Agents code in your paper, please cite:
+
+```
+@article{Ray2019,
+    author = {Ray, Alex and Achiam, Joshua and Amodei, Dario},
+    title = {{Benchmarking Safe Exploration in Deep Reinforcement Learning}},
+    year = {2019}
+}
+```

safe_rl/__init__.py

@@ -0,0 +1,5 @@
+from tensorflow.python.util import deprecation as deprecation
+deprecation._PRINT_DEPRECATION_WARNINGS = False
+
+from safe_rl.pg.algos import ppo, ppo_lagrangian, trpo, trpo_lagrangian, cpo
+from safe_rl.sac.sac import sac