Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Release 16] Release 16 Merge Back to Main #5255

Merged
merged 21 commits into from
Apr 14, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
7fae873
[🐛 🔨 ]Adding the ELO to the GlobalTrainingStatus (#5202)
vincentpierre Mar 31, 2021
2aaf326
[🐛 🔨 ] set_action_for_agent expects a ActionTuple with batch size 1. …
vincentpierre Mar 31, 2021
7b800d1
[bug-fix] When agent isn't training, don't clear update buffer (#5205)
Apr 1, 2021
f699d8d
[bug fix] Fix warning using demo recorder (#5216)
vincentpierre Apr 1, 2021
5d1c1e1
[Bug Fix] Issue 5204 : GymToUnityWrapper must call reset if done (#5207)
vincentpierre Mar 31, 2021
cf18e0e
Fixing the GymWrapper Logging issue (#5201)
vincentpierre Mar 31, 2021
deebc3d
Fix path to PushBlock demo (#5198)
andrewcoh Mar 30, 2021
efa8f34
[bug-fix] Fix POCA LSTM, pad sequences in the back (#5206)
Apr 5, 2021
bae46c2
[bug-fix] Use correct memories for LSTM SAC (#5228)
Apr 8, 2021
5e4f279
Load individual elements if state dict load fails (#5213)
andrewcoh Apr 6, 2021
ac26dc7
Add some entries to Release 16 changelog (#5237)
Apr 9, 2021
f8f2bfe
Add date to changelog
Apr 9, 2021
88f886b
better message for new model format (#5240)
Apr 10, 2021
420eb51
[Release 16] Update Python and release versions (#5234)
Apr 13, 2021
a391dcb
Update changelog date
Apr 13, 2021
e3b7fe3
[release_16] Update readme and readme table (#5252)
Apr 13, 2021
4088991
Merge branch 'main' into release_16_mm
Apr 14, 2021
894b435
[docs] Update main doc references (#5251)
Apr 14, 2021
879cc36
Merge branch 'release_16_branch' into release_16_mm
Apr 14, 2021
087251b
Revert logger
Apr 14, 2021
1d69bed
Revert barracuda model param loader
Apr 14, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# Unity ML-Agents Toolkit

[![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_15_docs/docs/)
[![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_16_docs/docs/)

[![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE)

Expand Down Expand Up @@ -47,8 +47,8 @@ descriptions of all these features.
## Releases & Documentation


**Our latest, stable release is `Release 15`. Click
[here](https://github.com/Unity-Technologies/ml-agents/tree/release_15_docs/docs/Readme.md)
**Our latest, stable release is `Release 16`. Click
[here](https://github.com/Unity-Technologies/ml-agents/tree/release_16_docs/docs/Readme.md)
to get started with the latest release of ML-Agents.**

The table below lists all our releases, including our `main` branch which is
Expand All @@ -68,16 +68,15 @@ under active development and may be unstable. A few helpful guidelines:
| **Version** | **Release Date** | **Source** | **Documentation** | **Download** | **Python Package** | **Unity Package** |
|:-------:|:------:|:-------------:|:-------:|:------------:|:------------:|:------------:|
| **main (unstable)** | -- | [source](https://github.com/Unity-Technologies/ml-agents/tree/main) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/main/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/main.zip) | -- | -- |
| **Release 15** | **March 17, 2021** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_15)** | **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_15_docs/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_15.zip)** | **[0.25.0](https://pypi.org/project/mlagents/0.25.0/)** | **[1.9.0](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.9/manual/index.html)** |
| **Release 16** | **April 13, 2021** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_16)** | **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_16_docs/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_16.zip)** | **[0.25.1](https://pypi.org/project/mlagents/0.25.1/)** | **[1.9.1](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.9/manual/index.html)** |
| **Release 15** | March 17, 2021 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_15) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_15_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_15.zip) | [0.25.0](https://pypi.org/project/mlagents/0.25.0/) | [1.9.0](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.9/manual/index.html) |
| **Verified Package 1.0.7** | **March 8, 2021** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/com.unity.ml-agents_1.0.7)** | **[docs](https://github.com/Unity-Technologies/ml-agents/blob/release_2_verified_docs/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/com.unity.ml-agents_1.0.7.zip)** | **[0.16.1](https://pypi.org/project/mlagents/0.16.1/)** | **[1.0.7](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.0/manual/index.html)** |
| **Release 14** | March 5, 2021 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_14) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_14_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_14.zip) | [0.24.1](https://pypi.org/project/mlagents/0.24.1/) | [1.8.1](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.8/manual/index.html) |
| **Release 13** | February 17, 2021 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_13) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_13_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_13.zip) | [0.24.0](https://pypi.org/project/mlagents/0.24.0/) | [1.8.0](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.8/manual/index.html) |
| **Release 12** | December 22, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_12) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_12_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_12.zip) | [0.23.0](https://pypi.org/project/mlagents/0.23.0/) | [1.7.2](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.7/manual/index.html) |
| **Release 11** | December 21, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_11) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_11_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_11.zip) | [0.23.0](https://pypi.org/project/mlagents/0.23.0/) | [1.7.0](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.7/manual/index.html) |
| **Release 10** | November 18, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_10) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_10.zip) | [0.22.0](https://pypi.org/project/mlagents/0.22.0/) | [1.6.0](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.6/manual/index.html) |
| **Verified Package 1.0.6** | November 16, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/com.unity.ml-agents_1.0.6) | [docs](https://github.com/Unity-Technologies/ml-agents/blob/release_2_verified_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/com.unity.ml-agents_1.0.6.zip) | [0.16.1](https://pypi.org/project/mlagents/0.16.1/) | [1.0.6](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.0/manual/index.html) |
| **Release 9** | November 4, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_9) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_9_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_9.zip) | [0.21.1](https://pypi.org/project/mlagents/0.21.1/) | [1.5.0](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.5/manual/index.html) |
| **Release 8** | October 14, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_8) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_8_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_8.zip) | [0.21.0](https://pypi.org/project/mlagents/0.21.0/) | [1.5.0](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.5/manual/index.html) |

If you are a researcher interested in a discussion of Unity as an AI platform,
see a pre-print of our
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ These limitations provided the motivation towards the development of the Grid Se

An image can be thought of as a matrix of a predefined width (W) and a height (H) and each pixel can be thought of as simply an array of length 3 (in the case of RGB), `[Red, Green, Blue]` holding the different channel information of the color (channel) intensities at that pixel location. Thus an image is just a 3 dimensional matrix of size WxHx3. A Grid Observation can be thought of as a generalization of this setup where in place of a pixel there is a "cell" which is an array of length N representing different channel intensities at that cell position. From a Convolutional Neural Network point of view, the introduction of multiple channels in an "image" isn't a new concept. One such example is using an RGB-Depth image which is used in several robotics applications. The distinction of Grid Observations is what the data within the channels represents. Instead of limiting the channels to color intensities, the channels within a cell of a Grid Observation generalize to any data that can be represented by a single number (float or int).

Before jumping into the details of the Grid Sensor, an important thing to note is the agent performance and qualitatively different behavior over raycasts. Unity MLAgent's comes with a suite of example environments. One in particular, the [Food Collector](https://github.com/Unity-Technologies/ml-agents/tree/release_15_docs/docs/Learning-Environment-Examples.md#food-collector), has been the focus of the Grid Sensor development.
Before jumping into the details of the Grid Sensor, an important thing to note is the agent performance and qualitatively different behavior over raycasts. Unity MLAgent's comes with a suite of example environments. One in particular, the [Food Collector](https://github.com/Unity-Technologies/ml-agents/tree/release_16_docs/docs/Learning-Environment-Examples.md#food-collector), has been the focus of the Grid Sensor development.

The Food Collector environment can be described as:
* Set-up: A multi-agent environment where agents compete to collect food.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,24 +29,24 @@ The ML-Agents Extensions package is not currently available in the Package Manag
recommended ways to install the package:

### Local Installation
[Clone the repository](https://github.com/Unity-Technologies/ml-agents/tree/release_15_docs/docs/Installation.md#clone-the-ml-agents-toolkit-repository-optional) and follow the
[Local Installation for Development](https://github.com/Unity-Technologies/ml-agents/tree/release_15_docs/docs/Installation.md#advanced-local-installation-for-development-1)
[Clone the repository](https://github.com/Unity-Technologies/ml-agents/tree/release_16_docs/docs/Installation.md#clone-the-ml-agents-toolkit-repository-optional) and follow the
[Local Installation for Development](https://github.com/Unity-Technologies/ml-agents/tree/release_16_docs/docs/Installation.md#advanced-local-installation-for-development-1)
directions (substituting `com.unity.ml-agents.extensions` for the package name).

### Github via Package Manager
In Unity 2019.4 or later, open the Package Manager, hit the "+" button, and select "Add package from git URL".

![Package Manager git URL](https://github.com/Unity-Technologies/ml-agents/blob/release_15_docs/docs/images/unity_package_manager_git_url.png)
![Package Manager git URL](https://github.com/Unity-Technologies/ml-agents/blob/release_16_docs/docs/images/unity_package_manager_git_url.png)

In the dialog that appears, enter
```
git+https://github.com/Unity-Technologies/ml-agents.git?path=com.unity.ml-agents.extensions#release_15
git+https://github.com/Unity-Technologies/ml-agents.git?path=com.unity.ml-agents.extensions#release_16
```

You can also edit your project's `manifest.json` directly and add the following line to the `dependencies`
section:
```
"com.unity.ml-agents.extensions": "git+https://github.com/Unity-Technologies/ml-agents.git?path=com.unity.ml-agents.extensions#release_15",
"com.unity.ml-agents.extensions": "git+https://github.com/Unity-Technologies/ml-agents.git?path=com.unity.ml-agents.extensions#release_16",
```
See [Git dependencies](https://docs.unity3d.com/Manual/upm-git.html#subfolder) for more information. Note that this
may take several minutes to resolve the packages the first time that you add it.
Expand All @@ -68,4 +68,4 @@ If using the `InputActuatorComponent`
- No way to customize the action space of the `InputActuatorComponent`

## Need Help?
The main [README](https://github.com/Unity-Technologies/ml-agents/tree/release_15_docs/README.md) contains links for contacting the team or getting support.
The main [README](https://github.com/Unity-Technologies/ml-agents/tree/release_16_docs/README.md) contains links for contacting the team or getting support.
2 changes: 1 addition & 1 deletion com.unity.ml-agents.extensions/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "com.unity.ml-agents.extensions",
"displayName": "ML Agents Extensions",
"version": "0.3.0-preview",
"version": "0.3.1-preview",
"unity": "2019.4",
"description": "A source-only package for new features based on ML-Agents",
"dependencies": {
Expand Down
36 changes: 23 additions & 13 deletions com.unity.ml-agents/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,6 @@ sizes and will need to be retrained. (#5181)
different sizes using the same model. For a summary of the interface changes, please see the Migration Guide. (##5189)
- Updated the Barracuda package to version `1.3.3-preview`(#5236)

#### ml-agents / ml-agents-envs / gym-unity (Python)
- The `--resume` flag now supports resuming experiments with additional reward providers or
loading partial models if the network architecture has changed. See
[here](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Training-ML-Agents.md#loading-an-existing-model)
for more details. (#5213)

### Minor Changes
#### com.unity.ml-agents / com.unity.ml-agents.extensions (C#)
- The `.onnx` models input names have changed. All input placeholders will now use the prefix `obs_` removing the distinction between visual and vector observations. In addition, the inputs and outputs of LSTM changed. Models created with this version will not be usable with previous versions of the package (#5080, #5236)
Expand Down Expand Up @@ -68,10 +62,26 @@ This results in much less memory being allocated during inference with `CameraSe
settings. Unfortunately, this may require retraining models if it changes the resulting order of the sensors
or actuators on your system. (#5194)
- Removed additional memory allocations that were occurring due to assert messages and iterating of DemonstrationRecorders. (#5246)

## [1.9.1-preview] - 2021-04-13
### Major Changes
#### ml-agents / ml-agents-envs / gym-unity (Python)
- The `--resume` flag now supports resuming experiments with additional reward providers or
loading partial models if the network architecture has changed. See
[here](https://github.com/Unity-Technologies/ml-agents/blob/release-16_docs/docs/Training-ML-Agents.md#loading-an-existing-model)
for more details. (#5213)

### Bug Fixes
#### com.unity.ml-agents (C#)
- Fixed erroneous warnings when using the Demonstration Recorder. (#5216)

#### ml-agents / ml-agents-envs / gym-unity (Python)
- Fixed an issue which was causing increased variance when using LSTMs. Also fixed an issue with LSTM when used with POCA and `sequence_length` < `time_horizon`. (#5206)
- Fixed a bug where the SAC replay buffer would not be saved out at the end of a run, even if `save_replay_buffer` was enabled. (#5205)

- ELO now correctly resumes when loading from a checkpoint. (#5202)
- In the Python API, fixed `validate_action` to expect the right dimensions when `set_action_single_agent` is called. (#5208)
- In the `GymToUnityWrapper`, raise an appropriate warning if `step()` is called after an environment is done. (#5204)
- Fixed an issue where using one of the `gym` wrappers would override user-set log levels. (#5201)
## [1.9.0-preview] - 2021-03-17
### Major Changes
#### com.unity.ml-agents (C#)
Expand Down Expand Up @@ -150,7 +160,7 @@ or actuators on your system. (#5194)
- Added a `--torch-device` commandline option to `mlagents-learn`, which sets the default
[`torch.device`](https://pytorch.org/docs/stable/tensor_attributes.html#torch.torch.device) used for training. (#4888)
- The `--cpu` commandline option had no effect and was removed. Use `--torch-device=cpu` to force CPU training. (#4888)
- The `mlagents_env` API has changed, `BehaviorSpec` now has a `observation_specs` property containing a list of `ObservationSpec`. For more information on `ObservationSpec` see [here](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Python-API.md#behaviorspec). (#4763, #4825)
- The `mlagents_env` API has changed, `BehaviorSpec` now has a `observation_specs` property containing a list of `ObservationSpec`. For more information on `ObservationSpec` see [here](https://github.com/Unity-Technologies/ml-agents/blob/release_13_docs/docs/Python-API.md#behaviorspec). (#4763, #4825)

### Bug Fixes
#### com.unity.ml-agents (C#)
Expand Down Expand Up @@ -216,7 +226,7 @@ discrete actions for shooting. (#4746)
#### com.unity.ml-agents (C#)
#### ml-agents / ml-agents-envs / gym-unity (Python)
- PyTorch trainers are now the default. See the
[installation docs](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Installation.md) for
[installation docs](https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Installation.md) for
more information on installing PyTorch. For the time being, TensorFlow is still available;
you can use the TensorFlow backend by adding `--tensorflow` to the CLI, or
adding `framework: tensorflow` in the configuration YAML. (#4517)
Expand All @@ -225,7 +235,7 @@ discrete actions for shooting. (#4746)
#### com.unity.ml-agents / com.unity.ml-agents.extensions (C#)
- The Barracuda dependency was upgraded to 1.1.2 (#4571)
- Utilities were added to `com.unity.ml-agents.extensions` to make it easier to
integrate with match-3 games. See the [readme](https://github.com/Unity-Technologies/ml-agents/blob/main/com.unity.ml-agents.extensions/Documentation~/Match3.md)
integrate with match-3 games. See the [readme](https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/com.unity.ml-agents.extensions/Documentation~/Match3.md)
for more details. (#4515)
#### ml-agents / ml-agents-envs / gym-unity (Python)
- The `action_probs` node is no longer listed as an output in TensorFlow models (#4613).
Expand All @@ -248,7 +258,7 @@ goes larger than 2^31. Previous Tensorflow checkpoints will become incompatible
#### ml-agents / ml-agents-envs / gym-unity (Python)
- Added the Random Network Distillation (RND) intrinsic reward signal to the Pytorch
trainers. To use RND, add a `rnd` section to the `reward_signals` section of your
yaml configuration file. [More information here](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Training-Configuration-File.md#rnd-intrinsic-reward) (#4473)
yaml configuration file. [More information here](https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Training-Configuration-File.md#rnd-intrinsic-reward) (#4473)
### Minor Changes
#### com.unity.ml-agents (C#)
- Stacking for compressed observations is now supported. An additional setting
Expand Down Expand Up @@ -367,11 +377,11 @@ first trajectory processed. (#4299)
### Major Changes
#### ml-agents / ml-agents-envs / gym-unity (Python)
- The Parameter Randomization feature has been refactored to enable sampling of new parameters per episode to improve robustness. The
`resampling-interval` parameter has been removed and the config structure updated. More information [here](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Training-ML-Agents.md). (#4065)
`resampling-interval` parameter has been removed and the config structure updated. More information [here](https://github.com/Unity-Technologies/ml-agents/blob/release_5_docs/docs/Training-ML-Agents.md). (#4065)
- The Parameter Randomization feature has been merged with the Curriculum feature. It is now possible to specify a sampler
in the lesson of a Curriculum. Curriculum has been refactored and is now specified at the level of the parameter, not the
behavior. More information
[here](https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Training-ML-Agents.md).(#4160)
[here](https://github.com/Unity-Technologies/ml-agents/blob/release_5_docs/docs/Training-ML-Agents.md).(#4160)

### Minor Changes
#### com.unity.ml-agents (C#)
Expand Down
Loading