Environments
The map contains apples and lemons. The first player (red) is very sensitive and scores 5 for the team for an apple (green square) and −5 for a lemon (yellow square). The second (blue), less sensitive player scores 1 for the team for an apple and −1 for a lemon. There is a wall of lemons between the players and the apples. Apples and lemons disappear when collected, and the environment resets when all apples are eaten. It is important that the sensitive agent eats the apples while the less sensitive agent should leave them to its team mate but clear the way by eating obstructing lemons.
Reference Paper : Value-Decomposition Networks For Cooperative Multi-Agent Learning ( Section 4.2)
Action Space: 0: Down, 1: Left, 2: Up , 3: Right, 4: Noop
Observation : Agent Coordinate + 3x3 mask around the agent + Steps in env.
Best Score: 0
Versions:
| Name | Description |
|---|---|
| Checkers-v0 | Each agent receives only it's local observation |
| Checkers-v1 | Each agent receives local observation all other agents |
It's a grid world environments having 2 agents (red and blue) where each agent wants to move their corresponding home location ( marked in boxes outlined in same colors). The challenging part of the game is to pass through the narrow corridor through which only one agent can pass at a time. They need to coordinate to not block he pathway for the other. A reward of +5 is given to each agent for reaching their home cell. The episode ends when both agent has reached their home state or for a maximum of 100 steps in environment.
Action Space: 0: Down, 1: Left, 2: Up , 3: Right, 4: Noop
Observation : Agent Coordinate + Steps in env.
Best Score: 0
Reference Paper : Value-Decomposition Networks For Cooperative Multi-Agent Learning ( Section 4.2)
Versions:
| Name | Description |
|---|---|
| CrossOver-v0 | Each agent receives only it's local position coordinates |
| CrossOver-v1 | Each agent receives position coordinates of all other agents |
Predator-prey involves a grid world, in which multiple predators attempt to capture randomly moving prey. Agents have a 5 × 5 view and select one of five actions ∈ {Left, Right, Up, Down, Stop} at each time step. Prey move according to selecting a uniformly random action at each time step. We define the “catching” of a prey as when the prey is within the cardinal direction of at least one predator. Each agent’s observation includes its own coordinates, agent ID, and the coordinates of the prey relative to itself, if observed. The agents can separate roles even if the parameters of the neural networks are shared by agent ID. We test with two different grid worlds: (i) a 5 × 5 grid world with two predators and one prey, and (ii) a 7 × 7 grid world with four predators and two prey. We modify the general predator-prey, such that a positive reward is given only if multiple predators catch a prey simultaneously, requiring a higher degree of cooperation. The predators get a team reward of 1 if two or more catch a prey at the same time, but they are given negative reward −P, if only one predator catches the prey as shown in Figure 7. We experimented with three varying P vales, where P = 0.5, 1.0, 1.5. The terminating condition of this task is when a prey is caught with more than one predator. The prey that has been caught is regenerated at random positions whenever the task terminates, and the game proceeds over fixed 100 steps.
Action Space: 0: Down, 1: Left, 2: Up , 3: Right, 4: Noop
Best Score: 0
| Name | Description |
|---|---|
| PredatorPrey5x5-v0 | |
| PredatorPrey5x5-v1 | |
| PredatorPrey5x5-v2 | |
| PredatorPrey5x5-v3 |
Same applies for PredatorPrey7x7
The task tests if two agents can synchronize their behaviour, when picking up objects and returning them to a drop point. In the Fetch task both players start on the same side of the map and have pickup points on the opposite side. A player scores 3 points for the team for pick-up, and another 5 points for dropping off the item at the drop point near the starting position. Then the pickup is available to either player again. It is optimal for the agents to cycle such that when one player reaches the pickup point the other returns to base, to be ready to pick up again.
Best Score: 0
Reference: Learning Multi-Agent communication with Backpropagation
Best Score: 0
We simulate a simple battle involving two opposing teams in a 15×15 grid as shown in Fig. 2(middle). Each team consists of m = 5 agents and their initial positions are sampled uniformly in a 5 × 5 square around the team center, which is picked uniformly in the grid. At each time step, an agent can perform one of the following actions: move one cell in one of four directions; attack another agent by specifying its ID j (there are m attack actions, each corresponding to one enemy agent); or do nothing. If agent A attacks agent B, then B’s health point will be reduced by 1, but only if B is inside the firing range of A (it's surrounding 3 × 3 areas). Agents need one time step of cooling down after an attack, during which they cannot attack. All agents start with 3 health points, and die when their health reaches 0. A team will win if all agents in the other team die. The simulation ends when one team wins, or neither of teams win within 40 time steps (a draw).
The model controls one team during training, and the other team consist of bots that follow a hardcoded policy. The bot policy is to attack the nearest enemy agent if it is within its firing range. If not, it approaches the nearest visible enemy agent within visual range. An agent is visible to all bots if it is inside the visual range of any individual bot. This shared vision gives an advantage to the bot team. When input to a model, each agent is represented by a set of one-hot binary vectors {i, t, l, h, c} encoding its unique ID, team ID, location, health points and cooldown. A model controlling an agent also sees other agents in its visual range (3 × 3 surrounding area). The model gets reward of -1 if the team loses or draws at the end of the game. In addition, it also get reward of −0.1 times the total health points of the enemy team, which encourages it to attack enemy bots.
Reference : Learning Multiagent Communication with Backpropagation