Multiagent Bug in Gym-Unity 14.1

[sahandrez]

Hello,

There seems to be a bug with multiagent environments in gym-unity release 14.1.
Some of the example environments work fine (e.g. 3D ball) while some of them (e.g. WallJump) fail with this error:

"The environment does not have the expected amount of agents. "
gym_unity.envs.UnityGymException: The environment does not have the expected amount of agents. Some agents did not request decisions at the same time.

It is possible that the issue is originated in the Unity environment side, but I can't seem to find the reason behind it. Single Agent environments work fine.

I'm using the provided notebooks here.

Thanks,
Sahand

[ervteng]

Hi @sahandrez, the multi-agent gym currently doesn't support environments with multiple types of behaviors (WallJump has two), or on-demand decisions (e.g. Bouncer). This is due to how gym expects all data per step to be from the same observation/action space.

[sahandrez]

Hi @ervteng,
I see. Thanks. I'm having another problem after fixing what you mentioned. Now in our custom environment, we are getting this error as soon as an agent is done in the environment:

File "\workspace\ml-agents\gym-unity\gym_unity\envs\__init__.py", line 531, in mark_agent_done
    gym_index = self._agent_id_to_gym_index.pop(agent_id)
KeyError: 7

Any idea what might have caused it? Thanks.

[ervteng]

Hi @sahandrez, are the agents done at different times/do they use on demand decisions?

Could you post the Inspector for your agents? Would be really helpful for us to debug. Thanks!

[sahandrez]

Hi @ervteng,
Thanks. We solved our problem. There was a bug in our environment which was not flagged in V0.13 but with the new multiagent system in V0.14 we figured it out.

[batu]

Hello @sahandrez ,
Can you tell me how you fixed the problem? I am facing the same KeyError issue in mark_agent_done

[sahandrez]

Hi @batu,
Sorry I no longer have access to the code. You can ask @ReinforcedMan for more info.

[RealZST]

Hi @batu ,
Did you fix it? I got the same error when I use gym-unity-0.15.0. Looking forward to your reply. Thank you.

[batu]

Unfortunately no. But I have something that is good enough.

In my case, I believe it happened when my agent got out of bounds and didn't reset properly. It was going out of bound because my implementation was acting weird when I push the simulation speed very high.

Currently, it happens only every 10-100 episodes, so when it does I just catch it and... pass.

[chriselion]

For this error:

File "\workspace\ml-agents\gym-unity\gym_unity\envs\__init__.py", line 531, in mark_agent_done
    gym_index = self._agent_id_to_gym_index.pop(agent_id)
KeyError: 7

We found a bug in the gym wrapper which would appear if an Agent called EndEpisode multiple times in the same step. The fix for this is in #3700, and we'll be doing a patch release next week that contains this fix (and a few others).

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.