KR-DL-UCT

This repository provides the source codes for KR-DL-UCT algorithm in the paper.

Deep Reinforcement Learning in Continuous Action Spaces: a Case Study in the Game of Simulated Curling by Kyowoon Lee, Sol-A Kim, Jaesik Choi and Seong-Whan Lee in ICML-2018

Abstract

Many real-world applications of reinforcement learning require an agent to select optimal actions from continuous action spaces. Recently, deep neural networks have successfully been applied to games with discrete actions spaces. However, deep neural networks for discrete actions are not suitable for devising strategies for games in which a very small change in an action can dramatically affect the outcome. In this paper, we present a new framework which incorporates a deep neural network that can be used to learn game strategies based on a kernel-based Monte Carlo tree search that finds actions within a continuous space. To avoid hand-crafted features, we train our network using supervised learning followed by reinforcement learning with a high-fidelity simulator for the Olympic sport of curling. The program trained under our framework outperforms existing programs equipped with several hand-crafted features and won an international digital curling competition.

Prerequisites

• Python 2.7 or Python 3.3+
• Tensorflow
• cython

Install

To get our code:

git clone --recursive https://github.com/leekwoon/KR-DL-UCT.git

To install:

python setup.py install build_ext --inplace

Running examples

In this code, you can run the game by using our algorithm. The game log will be located in ./data

python -m src.tests.game_test

You can download the latest simulator and watch the game from the log file.

Description of simulator are available from http://minerva.cs.uec.ac.jp/curling_en/wiki.cgi?page=Description+of+each+part

Authors

Kyowoon Lee*¹(leekwoon@unist.ac.kr), Sol-A Kim*¹(sol-a@unist.ac.kr), Jaesik Choi¹(jaesik@unist.ac.kr), Seong-Whan Lee²(sw.lee@korea.ac.kr)

¹UNIST @ Department of Computer Engineering, UNIST, Ulsan, Republic of Korea

²Korea University @ Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea

* Equal contribution