WIP Made initial changes to enable dimension properties and added attention module #4763

vincentpierre · 2020-12-16T06:09:46Z

Proposed change(s)

Implementation according to this design : https://docs.google.com/document/d/1EvgrFjICvz9o3ymAVnR1O7nNxeH-4UknahQTGcjG3ww/edit#

When trying to train with a BufferSensor, the user will see the error message :

mlagents.trainers.exception.UnityTrainerException: Unsupported shape of (A, B) for observation i

This Pr is just setting the ground work for allowing variable size observation (no trainer changes)

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

Checklist

Added tests that prove my fix is effective or that my feature works
Updated the changelog (if applicable)
Updated the documentation (if applicable)
Updated the migration guide (if applicable)

Other comments

…ntion module

com.unity.ml-agents/Runtime/Sensors/IDimensionPropertiesSensor.cs

com.unity.ml-agents/Runtime/Sensors/BufferSensor.cs

chriselion · 2020-12-18T01:06:50Z

com.unity.ml-agents/Runtime/Sensors/BufferSensor.cs

+        {
+            for (int i = 0; i < m_ObsSize * m_MaxNumObs; i++)
+            {
+                writer[i] = m_ObservationBuffer[i];


So this is implicitly zero padded, right?

Yes, the assumption is that if all the floats in an obs are 0, then the observation is padded. This padding will be ignored by the trainers, but still fed as input during inference / training.
We can pad with another value, but I chose 0.

chriselion · 2020-12-18T01:18:02Z

com.unity.ml-agents/Runtime/Communicator/GrpcExtensions.cs

+                if (dimensionPropertySensor != null)
+                {
+                    var dimensionProperties = dimensionPropertySensor.GetDimensionProperties();
+                    int[] intDimensionProperties = new int[dimensionProperties.Length];


nit: can you just loop and call observationProto.DimensionProperties.Add()?

com.unity.ml-agents/Runtime/Sensors/IDimensionPropertiesSensor.cs

dongruoping · 2020-12-21T19:59:11Z

com.unity.ml-agents/Runtime/Sensors/IDimensionPropertiesSensor.cs

+        None = 1,
+
+        /// <summary>
+        /// Means it is possible to do a convolution in this dimension.


I wonder if it will be better by saying something like "suitable/meaningful to apply a convolution".

(It's possible to do a conv anyways.)

Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>

…RIANCE

ml-agents-envs/mlagents_envs/base_env.py

com.unity.ml-agents/Runtime/Sensors/BufferSensor.cs

ml-agents/mlagents/trainers/torch/networks.py

ervteng · 2020-12-22T18:46:15Z

ml-agents-envs/mlagents_envs/base_env.py

    """

-    observation_shapes: List[Tuple]
+    sensor_spec: List[SensorSpec]


Suggested change

sensor_spec: List[SensorSpec]

sensor_specs: List[SensorSpec]

okay, will change throughout.

Any other issues you see with naming or design ?

vincentpierre · 2020-12-22T19:52:47Z

ml-agents-envs/mlagents_envs/environment.py

@@ -129,7 +129,7 @@ def _warn_csharp_base_capabilities(
        if not caps.baseRLCapabilities:
            logger.warning(
                "WARNING: The Unity process is not running with the expected base Reinforcement Learning"
-                " capabilities. Please be sure upgrade the Unity Package to a version that is compatible with this "
+                " capabilities. Please be sure upgrade the Unity Package sensor_specsn that is compatible with this "


I think I have a problem with my IDE, this should not have changed

ervteng · 2020-12-23T17:44:46Z

ml-agents-envs/mlagents_envs/tests/test_registry.py

@@ -36,7 +36,7 @@ def test_basic_in_registry():
    for worker_id in range(2):
        assert BASIC_ID in registry
        env = registry[BASIC_ID].make(
-            base_port=6005, worker_id=worker_id, no_graphics=True
+            base_port=6002, worker_id=worker_id, no_graphics=True


just checking if this change was intentional

ervteng

Super useful PR, LGTM

WIP Made initial changes to enale dimension properties and added atte…

66501bf

…ntion module

vincentpierre self-assigned this Dec 16, 2020

Fixing the meta files

1596d45

chriselion reviewed Dec 18, 2020

View reviewed changes

com.unity.ml-agents/Runtime/Sensors/IDimensionPropertiesSensor.cs Outdated Show resolved Hide resolved

chriselion reviewed Dec 18, 2020

View reviewed changes

com.unity.ml-agents/Runtime/Sensors/BufferSensor.cs Outdated Show resolved Hide resolved

chriselion reviewed Dec 18, 2020

View reviewed changes

com.unity.ml-agents/Runtime/Sensors/BufferSensor.cs Outdated Show resolved Hide resolved

chriselion reviewed Dec 18, 2020

View reviewed changes

vincentpierre changed the title ~~WIP Made initial changes to enale dimension properties and added attention module~~ WIP Made initial changes to enable dimension properties and added attention module Dec 18, 2020

vincentpierre added 3 commits December 18, 2020 16:18

addressing preliminary comments

d54ea91

Fixing bugs and started integration

cd49289

Removed a debug message

3cddfac

vincentpierre marked this pull request as ready for review December 21, 2020 18:30

dongruoping reviewed Dec 21, 2020

View reviewed changes

com.unity.ml-agents/Runtime/Sensors/IDimensionPropertiesSensor.cs Outdated Show resolved Hide resolved

dongruoping reviewed Dec 21, 2020

View reviewed changes

com.unity.ml-agents/Runtime/Sensors/IDimensionPropertiesSensor.cs Outdated Show resolved Hide resolved

dongruoping reviewed Dec 21, 2020

View reviewed changes

vincentpierre and others added 3 commits December 21, 2020 12:13

Update com.unity.ml-agents/Runtime/Sensors/IDimensionPropertiesSensor.cs

1268e9a

Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>

Update com.unity.ml-agents/Runtime/Sensors/IDimensionPropertiesSensor.cs

428763b

Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>

Replace possible with suitable in description of TRANSLATIONAL_EQUIVA…

5e907c3

…RIANCE

ervteng reviewed Dec 21, 2020

View reviewed changes

ml-agents-envs/mlagents_envs/base_env.py Outdated Show resolved Hide resolved

ervteng reviewed Dec 21, 2020

View reviewed changes

com.unity.ml-agents/Runtime/Sensors/BufferSensor.cs Show resolved Hide resolved

vincentpierre added 6 commits December 22, 2020 08:40

addressing comments. ObservationSpec is no longer a list

4c6e28d

Renaming ObservationSpec to SensorSpec

ce0e837

Modifying the documentation

1b416f2

Fixing the tests

e9319a6

renaming obs_spec variables

0338e70

Merging master

979bf58

ervteng reviewed Dec 22, 2020

View reviewed changes

ml-agents/mlagents/trainers/torch/networks.py Outdated Show resolved Hide resolved

ervteng reviewed Dec 22, 2020

View reviewed changes

Merge branch 'master' into develop-att

7c5a9ae

renaming sensor_spec to sensor_specS

c39955f

vincentpierre commented Dec 22, 2020

View reviewed changes

vincentpierre added 2 commits December 22, 2020 11:54

Fixing strange issues

618e981

fixing tests

dde2cb1

ervteng reviewed Dec 23, 2020

View reviewed changes

ervteng approved these changes Dec 23, 2020

View reviewed changes

vincentpierre merged commit 953366b into master Dec 23, 2020

delete-merged-branch bot deleted the develop-att branch December 23, 2020 19:20

github-actions bot locked as resolved and limited conversation to collaborators Dec 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP Made initial changes to enable dimension properties and added attention module #4763

WIP Made initial changes to enable dimension properties and added attention module #4763

vincentpierre commented Dec 16, 2020 •

edited

Loading

chriselion Dec 18, 2020

vincentpierre Dec 19, 2020

chriselion Dec 18, 2020

dongruoping Dec 21, 2020

ervteng Dec 22, 2020

vincentpierre Dec 22, 2020

vincentpierre Dec 22, 2020

vincentpierre Dec 22, 2020

ervteng Dec 23, 2020

ervteng left a comment

WIP Made initial changes to enable dimension properties and added attention module #4763

WIP Made initial changes to enable dimension properties and added attention module #4763

Conversation

vincentpierre commented Dec 16, 2020 • edited Loading

Proposed change(s)

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

Checklist

Other comments

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ervteng left a comment

Choose a reason for hiding this comment

vincentpierre commented Dec 16, 2020 •

edited

Loading