Refactor and Upgrade to Gymnasium #75

D-o-d-o-x · 2023-09-19T15:33:20Z

This PR upgrades fancy_gym from gym to gymnasium and refactors some parts.
This PR replaces #68 and should be merged instead.

Gymnasium is a fork of OpenAIs Gym (v21) by the Farama Foundation, since the original is no longer maintained.
fancy_gym previously used gym v25, the upgrade to Gymnasium v26 brought some breaking changes.

Our new fancy registry allows direct integration into gymnasium's env management; fancy_gym.make is no longer required and gym.make can be used for all our environments and mp-variants instead:
E.g. gym.make(fancy/Reacher5d-v0) or gym.make(fancy_ProDMP/Reacher5d-v0).
We also make all metaworld ML1 tasks avaible under gym.make(metaworld/...) / gym.make(metaworld_ProMP/...) / etc.
Registering mp-variants of environments and MP configuration management has been streamlined (details in the new README, all fancy envs and mp-wrappers were ported over to this new format).

We still fail 12 / 4630 Tests, but these are caused by bugs in other projects and should be fixed there.
These are listed here.

Additional changes:

Better README.
Switched to newest (as of now) Metaworld version.
Switched to new mujoco binding (except when some old OpenAI envs need to be used; old mujoco-py binding is a optional dependency).
Rudimentary support for envs, that have a Dict observation space has been added (since this is used by shimmys dm_control bindings).
Dependency versions have been made stricter (I had issues with getting tests to pass on some allowed versions).
Some of the custom envs still used an old coordinate format (global coordinates), that will no longer supported by future mujoco versions; these were converted to use local coordinates.
Now uses shimmy as a compat wrapper for dmc instead of custom one.
Since Metaworld now support gymnasium natively, we only have a thin wrapper, to mitigate some issues.
Mujoco envs seem to no longer allow supplying a custom XML file, so I build a wrapper, that re-implements this feature for the custom envs.
Many custom envs also lacked some metadata and did not expose an observation-space.
Fixed some issues with incorrect mp-args (e.g. weights_scale vs weight_scale)

This PR is based on Fabian's '47-update-to-new-gym-api'-branch.

This PR addresses the open issues #47, #62, #34 and PR#71.

In prevous gym versions executing a step returned obs, reward, done, info = env.step(...) With the switch to gymnasium this has changed to obs, reward, terminated, truncated, info = env.step(...) We also made the code a bit more self explainatory.

pip install .[all] will no longer install components only required for testing pip install .[testing] will also install all compionents required to run all tests

(e.g. 'dmc:...' by shimmy)

…den access private attribute.

Gymnasium Mujoco Envs no longer allow overriding the used xml_file We therefore implement intermediate classes, that reimplement this feature.

(Thanks to ChatGPT for helping)

fancy_gym/envs/mujoco/table_tennis/table_tennis_env.py

fancy_gym/envs/registry.py

fancy_gym/meta/README.MD

libs (like atari envs added by shimmy), only those included in gym and those added by us.

Fabian and others added 30 commits October 20, 2022 10:10

first updats to reacher and env creation

0c7ac83

updated examples to new api,

fbe3ef4

updated custom tasks to new api

ed72404

updated dm_control envs to use shimmy

9ebc021

updated tests for dm_control

ec2063a

updated to new API, so tests still failing

c53924d

num_basis is not implemented; tests are expected to fail

b695239

Updating gym(nasium) and metaworld; sticter versions

9de1257

Added box2d as a dependency

2cbfff7

Optional Dependency: PyTest

1c092d0

Update URL for Metaworld (rlworkgroup -> Farama)

826f22b

Merge branch '47-update-to-new-gym-api' into gym_upgrade

228e343

Adapted test to new gym interface

a559f92

In prevous gym versions executing a step returned obs, reward, done, info = env.step(...) With the switch to gymnasium this has changed to obs, reward, terminated, truncated, info = env.step(...) We also made the code a bit more self explainatory.

Use shimmy as a binding for dmc, also 'testing' seperate from 'all'

1e62da0

pip install .[all] will no longer install components only required for testing pip install .[testing] will also install all compionents required to run all tests

Buf Fix: PyTests still used binding to old gym

d4a8306

Fixed Bug: env_id was broken, when prefix is handled by other library

0dfd8e1

(e.g. 'dmc:...' by shimmy)

Fixed Bug: ToyEnv did not follow new gym spec

21d5ebb

Allow dmc envs to be accessed via dmc:... (for backwards compatibility)

07de655

Fix: TimeAwareObservation did not support float64 and tried to forbid…

bf3ed8a

…den access private attribute.

Ported box_pushing to new mujoco bindings

e75ab89

Fixed: Typos

e70a56a

Fixed: test_black_box.py still used old gym specs in two places

3bbf101

More verbose errors for run_env_determinism

bc9d4cf

Warning no longer applicable

1c002a2

Adapted Mujoco Envs to new gymnasium spec

dabfc7c

Gymnasium Mujoco Envs no longer allow overriding the used xml_file We therefore implement intermediate classes, that reimplement this feature.

Auto convert output spaces.Dict to Box for BB-Envs

d6df677

Merge branch 'master' into gym_upgrade

35680ab

Fix: NameError: name 'd' is not defined

a8a6760

Fix: hopper_jump_on_box still used unpatched HopperEnv

29b458c

Also support old gym Box as observation_space (backwards compat)

dbd7c37

Dominik Roth added 11 commits September 20, 2023 12:49

minor changes to README

d6ecc0d

Better doumentation of fancy registry fucntions (register & upgrade)

56c1c65

Updated dmc README

8bba78b

Improved and Updated README for custom mujoco envs

e55905a

Improved README for classical controll envs

e305556

Slighly better docs for OpenAI envs

a803d92

Fixed open_ai README

e633b1e

Expanded README for Metaworld

b3de71e

(Thanks to ChatGPT for helping)

Tiny update to Metaworld README

27f3a82

README: Justify text about MP envs (looks way better)

1eeccff

README: Better Justifycation of text

0ae8708

ottofabian requested changes Oct 11, 2023

View reviewed changes

Dominik Roth added 16 commits October 11, 2023 11:36

Fix: Typo in setup.py

9fee527

Added docstring to deprecated 'fancy_gym.make'

3180154

Added Hongyi & Hongyi as authors to setup.py

fcc79cd

Added bibtex for citations to README

5f3c28e

Fix: Bibtex in README mixed between tabs & spaces

bc150f5

Documnenting merging behavior for mp_configs (and remove TODOs)

757072c

README: @ is not a legal symbol in BibTex attribs

3317ffb

Development status alpha -> beta; version to 1.0 (for pub release)

db5e032

Add docstring to DefaultMPWrapper

980bb0d

Fix: Wrong horizon for Metaworld tasks

877a7ea

Dont run determinism tests on arbitrary external envs added by other

600575c

libs (like atari envs added by shimmy), only those included in gym and those added by us.

README: Minor cosmetic changes

fc73824

READE: Minor cosmetic changes pt.2

4f53477

READE: Minor cosmetic changes pt.3

614d2a4

Fix: Did not define MP_Wrapper for new RandomInit version of BoxPushing

f8dcf5b

Link legacy branch in README

a04a4b6

D-o-d-o-x merged commit c420a96 into ALRhub:master Oct 11, 2023

D-o-d-o-x mentioned this pull request Oct 11, 2023

Fix: DMC ProMP environments were actually all using DMP. #71

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor and Upgrade to Gymnasium #75

Refactor and Upgrade to Gymnasium #75

D-o-d-o-x commented Sep 19, 2023

Refactor and Upgrade to Gymnasium #75

Refactor and Upgrade to Gymnasium #75

Conversation

D-o-d-o-x commented Sep 19, 2023