Solving 2048 with Value Iteration and Markov Decision Process

Our project is an extension of another repo (https://github.com/voldikss/EE369-2048-AI)

Below is a list of methods/classes that we either created or modified/added to:

Created:

• task/agents.py
  • MarkovModel class
  • LearningAgent class
  • GreedyAgent class
  • load_states_probs method
  • determine_rewards method
  • is_win method
  • is_loss method
  • is_mergeable method
  • step method
  • calc_reward method
• game2048/agents.py
  • play_learn method
  • convert_state method
• learn_probabilities.py
• learned_states_probs.txt

Modified/added to:

• webapp.py
• task/planning.py
• evaluate.py

This is the original repo's README:

For SJTU EE369 final project.

Use supervised learning (imitation learning) and tree searching approaches to solve the game of 2048.

Code structure

• game2048/: the main package.
  • game.py: the core 2048 Game class.
  • agents.py: the Agent class with instances.
  • displays.py: the Display class with instances, to show the Game state.
  • expectimax/: a powerful ExpectiMax agent by here.
• task/: the implementation of supervised learning and tree searching.
  • agents.py: the Agent classes of supervised learning and tree searching.
  • model.py: the convolutional neural network model.
  • offline_training.py: offline method for training.
  • online_training.py: online method for training.
  • planning.py: the tree searching approach solution of the game.
  • util.py: tools to process the game board.
  • model_0_1024.h5: the dumped CNN model.
• explore.ipynb: introduce how to use the Agent, Display and Game.
• static/: frontend assets (based on Vue.js) for web app.
• webapp.py: run the web app (backend) demo.
• evaluate.py: evaluate the self-defined agent.

To run the evaluation of the agents

• To evaluate the supervised learning model, run

# Will play the game for 50 times and return the average score
python evaluate.py --agent=cnnagent

P.S. Currently the max score is 1024, the average score is 541.44.

• To evaluate the tree searching method, run

python evaluate.py --agent=pagent

P.S. With the depth set to 3, the planning method can reach the score 2048.

To run the web app

python webapp.py 

You can also specify an agent by adding --agent. cnnagent, pagent, emagent are usable, RandomAgent by default.

For example, run the web app with the planning agent

python webapp.py --agent=pagen

To compile the pre-defined ExpectiMax agent

cd game2048/expectimax
bash configure
make

LICENSE

The code is under Apache-2.0 License. pp.