Version 0.2.0

• New graph neural network building blocks (message passing based on torch-scatter in addition to existing graph convolutions)
• Support for action recording, replay from pre-computed action records and feature collection.
• Improved wrapper hierarchy semantics: Previously values were assigned to the outermost wrapper. Now values are assigned to existing attributes by traversing the wrapper hierarchy.
• Removal of deprecated modules (APIContext and Maze models for RLlib)
• Reflecting changes in upstream dependencies (Gym version pinned to <0.23)

Version 0.1.8

New Features

• Agent Deployment Workflow
• Soft Actor Critic from Demonstrations (SACfD)
• Locally Distributed ES Runner
• SpacesRecordingWrapper: Records and dumps processed trajectories to pickle files
• Fixes event logging for environment resets and policy events

Version 0.1.7

• Adds Soft Actor-Critic (SAC) Trainer (supporting Dictionary Observations and Actions)
• Simplifies the reward aggregation interface (now also supports multi-agent training)
• Extends PPO and A2C to multi-agent capable actor-critic trainers (individual agents vs. centralized critic)
• Adds option for custom rollout evaluators
• Adds option for shared weights in actor-critic settings
• Adds experiment and multi-run support for RunContext Python API
• Compatibility with PyTorch 1.9

Version 0.1.6

Changes

• made Maze compatible to Rllib 1.4
• updated to the recently released hydra 1.1.0
• Simplified API (RunContext): Experiment and evaluation support
• Fixed support of the nevergrad sweeper: made the LocalLauncher hydra plugin part of the wheel
• Replaced the (policy id, actor id) tuple with an ActorID class

Other

• various documentation improvements
• added ready-to-go Docker containers
• contribution guidelines, pull request templates etc. on GitHub

Version 0.1.5

Features:

• adds RunContext (Maze Python API)
• adds seeding to environments, models and trainers
• changes of simulated environment interfaces step_without_observation -> fast_step

Improvements:

• adds an ExportGifWrapper
• adds network architecture visualizations to Tensorboard Images
• adds incremental min/max stats
• adds categorical (support-based) value networks
• adds value transformations

Version 0.1.4

• switch to RLlib version 1.3.0.
• full structured env support
  • policy interface now selects policy based on actor_id
  • interfaces support collaborative multi-agent actor critic
• improved docs
• added testing dependencies to main package

Version 0.1.3

Improvements:

• Enable event collection from within the Wrapper stack
• Aligned StepSkipWrapper with the event system
• MonitoringWrapper: Logging of observations, actions and rewards throughout the wrapper stack, useful for diagnosis
• Make _recursive_ in Hydra config files compatible with Maze object instantiation

Version 0.1.2

Features:

• Imitation Learning:
  • Added Evaluation Rollouts
  • Unified dataset structures (InMemoryDataset)
• GlobalPoolingBlock: now supports sum and max pooling
• ObservationNormalizationWrapper: Adds observation and observation distribution visualization to Tensorboard logging.
• Distribution: Introduced VectorEnv, refactored the single and multi process parallelization wrappers.

Maze 0.1.1

Features:

• hyper parameter optimization via grid search and Nevergrad
• plain python training example
• local hydra job launcher
• extend attention/transformer perception blocks
• adds MazeEnvMonitoringWrapper as a default to wrapper stacks

Fixes:

• cumulative stats logging

Maze 0.1.0

Documentation updates:

• Integrating existing Gym environments
• Factory documentation
• Experiments workflow, ...

Updated to Hydra 1.1.0:

• Using Hydra.instantiate instead of custom registry implementation

Added Rollout evaluator