Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MarkovVectorEnv does not set is_vector_env = True #216

Closed
singh-jayant opened this issue May 22, 2023 · 4 comments
Closed

MarkovVectorEnv does not set is_vector_env = True #216

singh-jayant opened this issue May 22, 2023 · 4 comments

Comments

@singh-jayant
Copy link

I was trying to set up a pettingzoo multi-agent environment for use as a vectorized environment through the wrapper supersuit.pettingzoo_env_to_vec_env_v1 and I realized that MarkovVectorEnv does not set ìs_vector_env to True. Is this a design choice since this isn't a "real" multi-env class ? Or a bug? Or am I missing something?

Here are my package versions:

gymnasium==0.28.1
PettingZoo==1.22.3
SuperSuit==3.8.0
@elliottower
Copy link
Contributor

I'm not 100% familiar with Gymnasium's vector environments, but I believe the distinction is that the vector environments in SuperSuit are intended to be used concatenated together (concat_vec_env), where they will return a list of observations, list of infos, etc. In the Gymnasium vector environments I believe they do the vectorization on the numpy level, meaning they will add a new dimension. So for example, I believe reward turns into rewards and gets indexed by the environment number. My guess is that it's more efficient to do it with the array itself, and simpler because it's ultimately just a single environment, rather than a messy amalgamation of multiple environments

We intend to copy over the functionality of Gymnasium's vector environments to PettingZoo natively at some point, as it would be cleaner to have that versus these wrappers which aren't tested in the same CI as PettingZoo (so we could very easily mess something up in the future). It's much harder to maintain two separate libraries, so these wrappers will all eventually be moved and most likely changed quite a bit to be more efficient, etc.

@elliottower
Copy link
Contributor

Running all of the tests locally they still pass with the is_vector_env attribute set to True, but the behavior with various Gymnasium wrappers will likely change. Will test a bit more to see. Just For what it's worth, as an example with pistonball if you concat 1 vector env the first dimension is 20 (20 agents), with two the first dimension is 40 (20 agents * 2) and so on. The env.reset_infos which is returned on reset instead of info (see #226) is a list of length 20 with an empty dict for each agent. The base environment has infos indexed by the agent name and is a dict.

So the point is basically that the wrapper flattens the environments and removes the names of agents.

@elliottower
Copy link
Contributor

Just did some testing with that attribute added and it then when you try to use Gymnasium wrappers it throws errors for missing the attributes single_observation_space etc. You can use them as is currently though, for example the normalization one:

from pettingzoo.butterfly import pistonball_v6
import supersuit as ss
import numpy as np
env = pistonball_v6.parallel_env()
env = ss.pettingzoo_env_to_vec_env_v1(env)
env = ss.concat_vec_envs_v1(env, 10, base_class="gymnasium")
obs, info = env.reset()
print("Pre-normalization maximum: ", np.amax(obs)) # 221

from gymnasium.wrappers import normalize
env = normalize.NormalizeObservation(env)
obs, info = env.reset()
print("Post-normalization maximum: ", np.amax(obs)) # 0.009999897608091765

@elliottower
Copy link
Contributor

Adding a test to ensure that these work, so thanks for the heads up about this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants