(KOR)StarCraft II Multi Agent Challenge

제공하는 멀티 에이전트 강화학습 모델들 _QMIX, COMA, LIIR, G2ANet, QTRAN, VDN, Central V, IQL, MAVEN, ROMA, RODE, DOP and Graph MIX .

본 저장소는 Windows OS에서 편리하게 실행시키기 위해 만들어짐.

첫번째로 StarCraft 2을 설치 해야 합니다. 체험판도 상관없습니다. 아래 링크에서 받으세요

https://starcraft2.com/ko-kr/

설치 후 미니게임을 위한 맵을 아래 링크에서 다운로드 받아야 합니다.

https://github.com/oxwhirl/smac/tree/master/smac/env/starcraft2/maps/SMAC_Maps

다운 받은 맵들을 아래 경로에 이동시키면 됩니다.

C:\Program Files (x86)\StarCraft II\Maps\SMAC_Maps

이제 부터는 환경 설정 입니다.

필요한 패키지들을 설치하기 위하여 아래대로 명령을 입력하세요

pip install -r requirements.txt

불행이도 한가지는 직접 설치해야 합니다.(어렵지 않습니다.)

pytorch만 아래와 같이 직접 설치해 주세요\

conda install pytorch==1.2.0 torchvision==0.4.0 -c pytorch

마지막으로 "main.py"을 실행 시키면 됩니다.

(ENG)StarCraft II Multi Agent Challenge

The algorithms provided are _QMIX, COMA, LIIR, G2ANet, QTRAN, VDN, Central V, IQL, ROMA, RODE, DOP and Graph MIX .

This repository has been edited for convenient execution in Windows OS.

First you need to install the StarCraft 2 game. Trial version does not matter. Download it from the link below

https://starcraft2.com/ko-kr/

After installation, you should download the map required for the minigame from the link below.

https://github.com/oxwhirl/smac/tree/master/smac/env/starcraft2/maps/SMAC_Maps

You can move all downloaded files to the path below.

C:\Program Files (x86)\StarCraft II\Maps\SMAC_Maps

From now on, this is the environment setting.

Enter the following command to install the packages you need first.

pip install -r requirements.txt

Unfortunately, you need to install the one below yourself (it is not difficult).

You just need to install pytorch.

conda install pytorch==1.2.0 torchvision==0.4.0 -c pytorch

Finally, run "main.py"

Python MARL framework

PyMARL is WhiRL's framework for deep multi-agent reinforcement learning and includes implementations of the following algorithms:

• QMIX: QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• COMA: Counterfactual Multi-Agent Policy Gradients
• VDN: Value-Decomposition Networks For Cooperative Multi-Agent Learning
• IQL: Independent Q-Learning
• G2ANet: Multi-Agent Game Abstraction via Graph Attention Neural Network
• QTRAN: QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning
• LIIR: LIIR: Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning
• ROMA: ROMA: Multi-Agent Reinforcement Learning with Emergent Roles
• RODE: RODE: Learning Roles to Decompose Multi-Agent Tasks
• DOP: DOP: Off-Policy Multi-Agent Decomposed Policy Gradients
• Graph MIX: Graph Convolutional Value Decomposition in Multi-Agent Reinforcement Learning

PyMARL is written in PyTorch and uses SMAC as its environment.

Installation instructions

Build the Dockerfile using

cd docker
bash build.sh

Set up StarCraft II and SMAC:

bash install_sc2.sh

This will download SC2 into the 3rdparty folder and copy the maps necessary to run over.

The requirements.txt file can be used to install the necessary packages into a virtual environment (not recomended).

Run an experiment

python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=2s3z

The config files act as defaults for an algorithm or environment.

They are all located in src/config. --config refers to the config files in src/config/algs --env-config refers to the config files in src/config/envs

To run experiments using the Docker container:

bash run.sh $GPU python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=2s3z

All results will be stored in the Results folder.

The previous config files used for the SMAC Beta have the suffix _beta.

Saving and loading learnt models

Saving models

You can save the learnt models to disk by setting save_model = True, which is set to False by default. The frequency of saving models can be adjusted using save_model_interval configuration. Models will be saved in the result directory, under the folder called models. The directory corresponding each run will contain models saved throughout the experiment, each within a folder corresponding to the number of timesteps passed since starting the learning process.

Loading models

Learnt models can be loaded using the checkpoint_path parameter, after which the learning will proceed from the corresponding timestep.

Watching StarCraft II replays

save_replay option allows saving replays of models which are loaded using checkpoint_path. Once the model is successfully loaded, test_nepisode number of episodes are run on the test mode and a .SC2Replay file is saved in the Replay directory of StarCraft II. Please make sure to use the episode runner if you wish to save a replay, i.e., runner=episode. The name of the saved replay file starts with the given env_args.save_replay_prefix (map_name if empty), followed by the current timestamp.

The saved replays can be watched by double-clicking on them or using the following command:

python -m pysc2.bin.play --norender --rgb_minimap_size 0 --replay NAME.SC2Replay

Note: Replays cannot be watched using the Linux version of StarCraft II. Please use either the Mac or Windows version of the StarCraft II client.

Documentation/Support

Documentation is a little sparse at the moment (but will improve!). Please raise an issue in this repo, or email Tabish

Citing PyMARL

If you use PyMARL in your research, please cite the SMAC paper.

M. Samvelyan, T. Rashid, C. Schroeder de Witt, G. Farquhar, N. Nardelli, T.G.J. Rudner, C.-M. Hung, P.H.S. Torr, J. Foerster, S. Whiteson. The StarCraft Multi-Agent Challenge, CoRR abs/1902.04043, 2019.

In BibTeX format:

@article{samvelyan19smac,
  title = {{The} {StarCraft} {Multi}-{Agent} {Challenge}},
  author = {Mikayel Samvelyan and Tabish Rashid and Christian Schroeder de Witt and Gregory Farquhar and Nantas Nardelli and Tim G. J. Rudner and Chia-Man Hung and Philiph H. S. Torr and Jakob Foerster and Shimon Whiteson},
  journal = {CoRR},
  volume = {abs/1902.04043},
  year = {2019},
}

License

Code licensed under the Apache License v2.0