Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JSON Serializable error #2328

Closed
bara-bba opened this issue Jun 1, 2022 · 6 comments · May be fixed by #2329
Closed

JSON Serializable error #2328

bara-bba opened this issue Jun 1, 2022 · 6 comments · May be fixed by #2329

Comments

@bara-bba
Copy link

bara-bba commented Jun 1, 2022

Hi everyone!

I installed garage on my new pc but in doing so I encountered this error. My environment hasn't changed at all so I think it is due to a version incompatibility of packages.

pip list looks like this:

Package                      Version
---------------------------- ---------------
absl-py                      1.0.0
aiosignal                    1.2.0
akro                         0.0.8
argon2-cffi                  21.3.0
argon2-cffi-bindings         21.2.0
astunparse                   1.6.3
attrs                        21.4.0
bleach                       4.1.0
cachetools                   5.2.0
certifi                      2022.5.18.1
charset-normalizer           2.0.12
click                        8.1.3
cloudpickle                  2.1.0
cma                          2.7.0
cycler                       0.11.0
debugpy                      1.5.1
decorator                    5.1.1
defusedxml                   0.7.1
Deprecated                   1.2.13
distlib                      0.3.4
dm-env                       1.5
dm-tree                      0.1.7
dowel                        0.0.3
filelock                     3.7.1
flatbuffers                  1.12
fonttools                    4.33.3
frozenlist                   1.3.0
future                       0.18.2
garage                       2021.3.0
gast                         0.4.0
glfw                         2.5.3
google-auth                  2.6.6
google-auth-oauthlib         0.4.6
google-pasta                 0.2.0
grpcio                       1.43.0
gym                          0.24.0
gym-notices                  0.0.6
h5py                         3.7.0
idna                         3.3
imageio                      2.19.3
importlib-metadata           4.11.4
importlib-resources          5.7.1
ipython-genutils             0.2.0
jsonschema                   4.5.1
jupyter-core                 4.9.2
jupyterlab-pygments          0.1.2
keras                        2.9.0
Keras-Preprocessing          1.1.2
kiwisolver                   1.4.2
labmaze                      1.0.5
libclang                     14.0.1
lxml                         4.7.1
Markdown                     3.3.7
matplotlib                   3.5.2
mistune                      0.8.4
mkl-fft                      1.3.1
mkl-random                   1.2.2
mkl-service                  2.4.0
msgpack                      1.0.3
mujoco                       2.1.2
nbclient                     0.5.11
nest-asyncio                 1.5.4
networkx                     2.8.2
notebook                     6.4.8
numpy                        1.22.3
oauthlib                     3.2.0
opt-einsum                   3.3.0
packaging                    21.3
pandocfilters                1.5.0
pathlib                      1.0.1
pathlib2                     2.3.6
Pillow                       9.0.1
pip                          22.1.2
platformdirs                 2.5.2
prometheus-client            0.13.1
protobuf                     3.19.4
psutil                       5.9.1
ptyprocess                   0.7.0
pyasn1                       0.4.8
pyasn1-modules               0.2.8
PyOpenGL                     3.1.5
pyparsing                    3.0.9
pyrsistent                   0.18.1
python-dateutil              2.8.2
PyWavelets                   1.3.0
PyYAML                       6.0
pyzmq                        22.3.0
ray                          1.12.1
redis                        4.1.1
requests                     2.27.1
requests-oauthlib            1.3.1
rlkit                        0.2.1.dev0
rsa                          4.8
scikit-image                 0.19.2
scipy                        1.8.1
Send2Trash                   1.8.0
setproctitle                 1.2.3
setuptools                   61.2.0
six                          1.16.0
tabulate                     0.8.9
tensorboard                  2.9.0
tensorboard-data-server      0.6.1
tensorboard-plugin-wit       1.8.1
tensorboardX                 2.5
tensorflow                   2.9.1
tensorflow-estimator         2.9.0
tensorflow-io-gcs-filesystem 0.26.0
tensorflow-probability       0.16.0
termcolor                    1.1.0
terminado                    0.13.1
testpath                     0.6.0
tifffile                     2022.5.4
torch                        1.7.1
torchaudio                   0.7.0a0+a853dff
torchvision                  0.8.0a0
tqdm                         4.62.3
typing_extensions            4.1.1
urllib3                      1.26.9
virtualenv                   20.14.1
webencodings                 0.5.1
Werkzeug                     2.1.2
wheel                        0.37.1
wrapt                        1.14.1
zipp                         3.8.0

And this is the error:

/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Only `cma.purecma` has been imported. Install `numpy` ("pip install numpy") if you want to import the entire `cma` package.

/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py:551: UserWarning: Archiving a launch repo larger than 8MiB. This may be slow. Set archive_launch_repo=False in wrap_experiment to disable this behavior.
  warnings.warn('Archiving a launch repo larger than 8MiB. This may be '
2022-06-01 09:21:47 | [garage_sac] Logging to /home/bara/PycharmProjects/garage/data/local/experiment/garage_sac_2
/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py:57: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if (distutils.version.LooseVersion(tf.__version__) <
/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/deterministic.py:36: UserWarning: Enabeling deterministic mode in PyTorch can have a performance impact when using GPU.
  warnings.warn(
Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Trainer is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type NormalizedEnv is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type GymEnv is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type PandaEnv is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type dict is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type dict is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type timedelta64 is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "scripts/garage_sac.py", line 112, in <module>
    garage_sac(seed=521)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 369, in __call__
    result = self.function(ctxt, **kwargs)
  File "scripts/garage_sac.py", line 108, in garage_sac
    trainer.train(n_epochs=10000, batch_size=1024, plot=True)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/trainer.py", line 400, in train
    dump_json(summary_file, self)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 464, in dump_json
    json.dump(data,
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 643, in _default_inner
    data[k] = self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in _default_inner
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 670, in <dictcomp>
    data = {k: self.default(v) for (k, v) in obj_dict.items()}
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 643, in _default_inner
    data[k] = self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 657, in _default_inner
    return float(o)
TypeError: float() argument must be a string or a number, not 'NoneType'

Thanks in advance for the help!
@krzentner
Copy link
Contributor

krzentner commented Jun 1, 2022 •
edited
Loading

It's difficult for me to debug this with certainty given the information, since it's not clear exactly what the JSON encoder is failing on.

Are you trying to pass the Trainer as an argument to an experiment function (i.e. a function that has wrap_experiment applied to it)?
I've tried to make that work, but I wouldn't be surprised if there were corner cases that still didn't work.

The final error here appears to be that isinstance(o, np.number) is True, and yet float(o) crashes with float() argument must be a string or a number, not 'NoneType', which is quite strange. My guess is that somehow a np.timedelta64(None) is being created, but I have no clue where it would be coming from. Perhaps such a value is being created inside your environment and you're passing it as an argument to your experiment function?

@krzentner krzentner mentioned this issue Jun 1, 2022
Open
@krzentner
Copy link
Contributor

I've made a PR (linked above) that should hopefully fix this. Please install it and let me know if it does:
pip uninstall garage && pip install git+https://github.com/rlworkgroup/garage.git@fix-timedelta64-json

@bara-bba
Copy link
Author

bara-bba commented Jun 1, 2022

Thanks for your reply!
Unfortunately, I don't exactly understand what you mean by passing the Trainer to the experiment function. I'm using almost the same code as sac_half_cheetah.py:


#!/usr/bin/env python3
"""This is an example to train a task with SAC algorithm written in PyTorch."""
import numpy as np
import torch
from torch import nn
from torch.nn import functional as F
import warnings
from garage import wrap_experiment
from garage.envs import GymEnv, normalize
from garage.experiment import deterministic
from garage.replay_buffer import PathBuffer
from garage.sampler import FragmentWorker, LocalSampler, DefaultWorker
from garage.torch import set_gpu_mode
from garage.torch.algos import SAC
from garage.torch.policies import TanhGaussianMLPPolicy
from garage.torch.q_functions import ContinuousMLPQFunction
from garage.trainer import Trainer

from panda_env import PandaEnv

"""Snapshotter snapshots training data.

When training, it saves data to binary files. When resuming,
it loads from saved data.

Args:
    snapshot_dir (str): Path to save the log and iteration snapshot.
    snapshot_mode (str): Mode to save the snapshot. Can be either "all"
        (all iterations will be saved), "last" (only the last iteration
        will be saved), "gap" (every snapshot_gap iterations are saved),
        "gap_and_last" (save the last iteration as 'params.pkl' and save
        every snapshot_gap iteration separately), "gap_overwrite" (same as
        gap but overwrites the last saved snapshot), or "none" (do not
        save snapshots).
    snapshot_gap (int): Gap between snapshot iterations. Wait this number
        of iterations before taking another snapshot.

"""


@wrap_experiment(snapshot_mode='gap', snapshot_gap=100)
def garage_sac(ctxt=None, seed=1):
    """Set up environment and algorithm and run the task.

    Args:
        ctxt (garage.experiment.ExperimentContext): The experiment
            configuration used by Trainer to create the snapshotter.
        seed (int): Used to seed the random number generator to produce
            determinism.

    """
    deterministic.set_seed(seed)

    trainer = Trainer(snapshot_config=ctxt)
    env = normalize(GymEnv(PandaEnv(), max_episode_length=300), normalize_obs=True)

    policy = TanhGaussianMLPPolicy(
        env_spec=env.spec,
        hidden_sizes=[256, 256],
        hidden_nonlinearity=nn.ReLU,
        output_nonlinearity=None,
        min_std=np.exp(-20.),
        max_std=np.exp(2.),
    )

    qf1 = ContinuousMLPQFunction(env_spec=env.spec,
                                 hidden_sizes=[512, 512],
                                 hidden_nonlinearity=F.relu)

    qf2 = ContinuousMLPQFunction(env_spec=env.spec,
                                 hidden_sizes=[512, 512],
                                 hidden_nonlinearity=F.relu)

    replay_buffer = PathBuffer(capacity_in_transitions=int(5e6),
                               env_spec=env.spec)

    sampler = LocalSampler(agents=policy,
                           envs=env,
                           max_episode_length=env.spec.max_episode_length,
                           worker_class=DefaultWorker)

    sac = SAC(env_spec=env.spec,
              policy=policy,
              policy_lr=0.001,
              qf1=qf1,
              qf2=qf2,
              qf_lr=0.001,
              sampler=sampler,
              optimizer=torch.optim.Adam,
              gradient_steps_per_itr=10,
              replay_buffer=replay_buffer,
              min_buffer_size=1e3,
              num_evaluation_episodes=100,
              target_update_tau=0.005,
              discount=0.99,
              buffer_batch_size=512,
              initial_log_entropy=0.,
              reward_scale=1.,
              steps_per_epoch=1)

    if torch.cuda.is_available():
        set_gpu_mode(True)
    else:
        set_gpu_mode(False)

    sac.to()
    trainer.setup(algo=sac, env=env)
    trainer.train(n_epochs=10000, batch_size=1024, plot=True)


s = np.random.randint(0, 1000)
garage_sac(seed=521)

I also tried to install your fix but I still get:

/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py:551: UserWarning: Archiving a launch repo larger than 8MiB. This may be slow. Set archive_launch_repo=False in wrap_experiment to disable this behavior.
  warnings.warn('Archiving a launch repo larger than 8MiB. This may be '
2022-06-01 11:41:00 | [garage_sac] Logging to /home/bara/PycharmProjects/garage/data/local/experiment/garage_sac
/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py:57: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if (distutils.version.LooseVersion(tf.__version__) <
/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/deterministic.py:36: UserWarning: Enabeling deterministic mode in PyTorch can have a performance impact when using GPU.
  warnings.warn(
/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py:126: UserWarning: torch.distributed.reduce_op is deprecated, please use torch.distributed.ReduceOp instead
  warnings.warn("torch.distributed.reduce_op is deprecated, please use "
Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Trainer is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type NormalizedEnv is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type GymEnv is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type PandaEnv is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type dict is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 637, in _default_inner
    return json.JSONEncoder.default(self, o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type module is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "scripts/garage_sac.py", line 112, in <module>
    garage_sac(seed=521)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 369, in __call__
    result = self.function(ctxt, **kwargs)
  File "scripts/garage_sac.py", line 108, in garage_sac
    trainer.train(n_epochs=10000, batch_size=1024, plot=True)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/trainer.py", line 394, in train
    dump_json(summary_file, self)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 464, in dump_json
    json.dump(data,
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 644, in _default_inner
    data[k] = self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 675, in <dictcomp>
    k: self.default(v)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 611, in default
    return self._default_inner(o)
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in _default_inner
    data = {
  File "/home/bara/anaconda3/envs/garage2020/lib/python3.8/site-packages/garage/experiment/experiment.py", line 674, in <dictcomp>
    data = {
RuntimeError: dictionary changed size during iteration

@bara-bba
Copy link
Author

bara-bba commented Jun 1, 2022

Ok, I installed a previous version of gym and now it is working. I think the main problem is in the changes with _mujoco_bindings in gym==0.24.0

@krzentner
Copy link
Contributor

Ah, yeah. The recent gym changes have broken a lot of things. It appears I forgot about another one of the calls to dump_json, which dumps the Trainer, hence my confusion earlier. I've made a version that's even more robust now, so hopefully this issue shouldn't come up in the future.

@bara-bba
Copy link
Author

bara-bba commented Jun 3, 2022

Many thanks for the help!

@bara-bba bara-bba closed this as completed Jun 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make LogEncoder more robust rlworkgroup/garage
2 participants
@bara-bba @krzentner