1.3.0

Memory networks in BenchMARL

BenchMARL release paired with TorchRL 0.6.0

Highlights

RNNs in BenchMARL

We now support RNNs as models!

We have implemented and added to the library GRU and LSTM.

These can be used in the policy or in the critic (both from local agent inputs and from global centralized inputs). They are also compatible with any parameter sharing choice.

We have benchmarked GRU on a multiagent version of repeat_previous and it does solve the task, while just MLP fails

In cotrast to traditional RNNs, we do not do any time padding. Instead we do a for loop over time in the sequence while reading the “is_init” flag. This approach is slower but leads to better results.

Oh also as always you can chain them with as many models as you desire (CNN, GNN, ...)

Simplified model reloading and evaluation

We have added some useful tools described at here

In particular, we have added experiment.evaluate() and some useful command line tols like benchmarl/evaluate.py and benchmarl/resume.py that just take the path to a checkpoint file.

Basically now you can reload models from the hydra run without giving again all the config, the scripts will find the configs you used in the hydra folders automatically.

Better logging of episode metrics

BenchaMARL will now consider an episode done when the global task done is set. Thus, it will allow for agents done early (as long as the global done is set on all()

Here is an overview of how episode metrics are computed:

BenchMARL will be looking at the global done (always assumed to be set), which can usually be computed using any or all over the single agents dones.

In all cases the global done is what is used to compute the episode reward.

We log episode_reward min, mean, max over episodes at three different levels:

• agent, (disabled by default, can be turned on manually)
• group, averaged over agents in group
• global, averaged over agents in groups and gropus

What's Changed

• [BugFix] Fix collect with grad by @matteobettini in #114
• [Refactor] Update values headed to deprecation by @matteobettini in #118
• [Docs] Citation by @matteobettini in #119
• [Model] GRU and general RNN compatibility by @matteobettini in #116
• [Model] LSTM by @matteobettini in #120
• [BugFix, Feature] GNN position_key and velocity_key not in observation_spec by @matteobettini in #125
• [BugFix] Small fix to multi-group eval and add wandb project_name by @matteobettini in #126
• [Feature] Improved experiment reloading and evaluation by @matteobettini in #127
• [BugFix] GNN position and velocity key by @matteobettini in #132
• [BugFix] More flexible episode_reward computation in logger by @matteobettini in #136
• [Feature] RNNs in SAC losses by @matteobettini in #139
• [Version] 1.3.0 by @matteobettini in #140

Full Changelog: 1.2.1...1.3.0

1.2.1

BenchMARL release paired with TorchRL 0.4.0

Many exctiting updates in this new release!

Most importantly there have been major inmpovements to the GNN and the addition of a new model: DeepSets

New features

• [Feature] Allow multiple observation keys in specs by @matteobettini in #82 #83
• [Feature] Buffer device by @matteobettini in #87
• [Feature] More VMAS tasks by @matteobettini in #88
• [Feature] Simplify extending benchmarl and improve extending examples by @matteobettini in #89
• [Example] Collecting with gradient by @matteobettini in #77
• [Feature] Share parameters between models by @matteobettini in #95
• [Feature] GNN build topology dynamically from positions by @matteobettini in #98
• [Model] DeepSets by @matteobettini in #96
• [Feature] PPO minibatch advantage by @matteobettini in #100
• [Feature] keep_checkpoints_num and checkpoint_at_end by @matteobettini in #102
• [Feature] Run evaluation on first iteration by @matteobettini in #108
• [Feature] Iteration timer includes evaluation by @matteobettini in #110

New Contributors

• @mchoilab made their first contribution in #105

Full Changelog: 1.2.0...1.2.1

1.2.0

BenchMARL release paired with TorchRL 0.4.0

BenchMARL just got extended with 49 vision-based tasks from Google DeepMind MeltingPot 🎉

But wait there is more! We have added a brand new CNN model 🚀 that can be used in actors and critics (fully compatible with any parameter sharing and centralization choices)

What's Changed

• [Feature] Allow any name for observation and global_state keys by @matteobettini in #72
• [Model] CNN by @matteobettini in #74
• [Feature] Allow multipe inputs to models by @matteobettini in #73
• [Environment] Melitngpot by @matteobettini in #75

Full Changelog: 1.1.1...1.2.0

1.1.1

Align to torchrl 0.3.1

What's Changed

• [CI] Fix CI by @matteobettini in #65
• [Test] Lighter tests by @matteobettini in #66
• [CI] Fix CI by @matteobettini in #67
• [Release] 1.1.1 by @matteobettini in #69

Full Changelog: 1.1.0...1.1.1

1.1.0

Highlights

• New GNN Model
• New VMAS scenarios and VMAS MPE scenarios
• Multiple bug fixes and new configuration parameters
• BenchMARL 1.1.0 is paired to TorchRL 0.3.0

What's Changed

• [Model] GNN by @matteobettini in #30
• [Docs] Citation by @matteobettini in #39
• [BugFix] Fix discrete SAC by @matteobettini in #40
• [Tasks] VMAS MPE (Vectorized) by @matteobettini in #42
• [Feature] Optional Tanh transfromed distributions in algorithms by @matteobettini in #45
• [Feature] Choose if evaluation is run on sampled actions or on mode by @matteobettini in #49
• [BugFix] Use correct types to prevent multiwalker init error. by @KaleabTessera in #47
• [CI] Fix code cov by @matteobettini in #51
• [Feature] More VMAS tasks by @matteobettini in #48
• [Doc] Fix Typo by @matteobettini in #57
• [BugFix] Rename PPO params by @matteobettini in #59
• [Doc] Fix typo by @matteobettini in #60
• [CI] Run main tests with torchrl nightly and have a separate CI for testing torchrl stable by @matteobettini in #53
• [Refactor] Adapt PPO parameter names by @matteobettini in #62
• [Versioning] 1.1.0 by @matteobettini in #61

New Contributors

• @KaleabTessera made their first contribution in #47

Full Changelog: 1.0.0...1.1.0

1.0.0

BenchMARL docs

The BenchMARL docs are now available at https://benchmarl.readthedocs.io/.

What's Changed

• [Feature] More callbacks by @matteobettini in #35
• [Docs] Link video presentation by @matteobettini in #36
• [BugFix] Task loading schema validation by @matteobettini in #37
• [Docs] Build the docs by @matteobettini in #34

Full Changelog: 0.0.4...1.0.0

0.0.4

0.0.4

0.0.3

• Added yaml configuration files to PyPi package
• Addded possibility to access the experiment from the algorithm

0.0.2

First release