setuptools-based plugin for StatsWriters

[chriselion]

Proposed change(s)

Adds a setuptools-based plugin system, and an interface for registering custom StatsWriters. Also refactored some of the output directory settings so that TensorBoardWriter just needs the RunOptions.

For more background, see this video and example code.

Example output:

$ mlagents-learn --force
...
2020-12-21 15:06:51 INFO [learn.py:274] run_seed set to 249
registering default
registering example
Creating a new stats writer! This is so exciting!

...

ExampleStatsWriter category: 3DBall values: {'Policy/Entropy': StatsSummary(mean=1.4183114, std=0.00084836915, num=50172), ...}

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

https://github.com/anthonywritescode/explains/tree/master/sample_code/ep128
https://www.youtube.com/watch?v=fY3Y_xPKWNA

Types of change(s)

• New feature

Checklist

• Added tests that prove my fix is effective or that my feature works
• Updated the changelog (if applicable)
• Updated the documentation (if applicable)

[chriselion]

No plans to publish this package, but we could use it to set up tests, e.g. for bad imports.

[chriselion]

We would need to add a new key for each type of plugin interface we want, e.g. mlagents.demonstration_provider: [...]

[chriselion]

The form of this is {entry point name}={plugin module}:{plugin_function}

[vincentpierre]

Make the see below a link to line 41

[chriselion]

Done

[vincentpierre]

Is this relevant to this PR? Why are we adding this flag ?

[chriselion]

This was previously hardcoded in learn.py
https://github.com/Unity-Technologies/ml-agents/pull/4788/files#diff-4e0d7b8aef419254e6fc3c63f5baa53f222afda7c908705e857a22f7d4792c47L68

and is now used in CheckpointSettings.maybe_init_path()
https://github.com/Unity-Technologies/ml-agents/pull/4788/files#diff-546e90789e914f8707fd97391f78c4bba39ae69a965858bbb69c2d324db1ec51R719

I can keep it hardcoded if you prefer, but I do think moving all the path logic to part of RunOptions is a win (and basically a pre-req to make TensorBoardWriter act like a plugin)

[ervteng]

If a user changes this from the default results will it mess with how they're saved/synced in cloud? cc: @hvpeteet

[hvpeteet]

If a user can set it in the run_options then yes this could mess with cloud since we don't already take it into account. I don't think it is a reason not to make the change though. We can add logic to overwrite this field since we already modify run_options.

Just a couple of questions to confirm my understanding:

1. Would this be set under checkpoint_settings --> write_path?
2. Could we (cloud) blindly set this for every experiment and it would be ignored by older mlagents versions?

[chriselion]

1. It would be checkpoint_settings -> results_dir (at the top level)
2. This would trigger an error here

   ml-agents/ml-agents/mlagents/trainers/settings.py

   Line 37 in 5db6efe

   def check_and_structure(key: str, value: Any, class_type: type) -> Any:

[hvpeteet]

Thanks, we can check for it and only overwrite it if already set then. When you check this in can you file a ticket against me to make sure we get this added at some point?

[chriselion]

Will do