Project is about neural network determining hand and probability of winning with hand and game stage (preflop, flop, turn, river) in Poker Texas Hold'em. Requires python3.
- Deuces - used to determine hand and their strength. Available on: https://github.com/worldveil/deuces and compatible with python2
- Monte-Carlo - used to determine probability of winning with hand and game stage using Monte Carlo method. Available on: https://github.com/Blind0ne/Monte-Carlo. Requires deuces library.
To generate training and test set use the SetGenerator.py file. generate_sets method takes parameters: the number of all card at given stage: all_cards_no(2, 5, 6, 7). The number of sets depends on the parameter:
| Parameter | Test set | Training set |
|---|---|---|
| 2 | 133 |
1195 |
| 5 | 3979 | 35803 |
| 6 | 3979 | 35803 |
| 7 | 3979 | 35803 |
Method prints to STDOUT number of the generated sample (trening, test)
Part of the sample training set for the parameter all_cards_no = 2:
2,4,2,6,0.20840000000000003
1,6,4,13,0.23440000000000005
2,6,4,13,0.25
3,12,4,12,0.5028
1,6,3,6,0.24680000000000002
Card is represented by pair: suit and rank. Two pairs are cards in hand. Last number is the probability of winnig.
Part of the sample training set for the parameter all_cards_no = 6:
1,5,4,5,1,12,3,7,2,13,4,1,8,0.12680000000000002
3,4,4,4,4,5,4,9,2,4,1,4,2,0.9996
1,6,4,6,1,8,2,8,2,1,2,2,7,0.28080000000000005
1,2,2,5,4,7,3,6,3,13,1,9,9,0.03520000000000001
4,4,3,7,4,6,3,9,4,10,1,7,8,0.11599999999999999
Card is represented by pair: suit and rank. Last two numbers are: the strongest hand and probability of winning. The first two pairs are the cards in hand, the others - on board.
from Sets.SetGenerator import SetGenerator
all_cards_no = 5
generator = SetGenerator()
generator.generate_sets(all_cards_no)
You must import the SetImporter.py and SetGenerator.py files. The constructor of the SetImporter class is two-argument: the path to the file with the set and the number of all cards at a given stage of the game.
from Sets.SetImporter import SetImporter
from Sets.SetGenerator import SetGenerator
all_cards_no = 5
train_set = SetImporter("{}/{}.data".format(SetGenerator.dir_path, "training", all_cards_no), all_cards_no)
test_set = SetImporter("{}/{}.data".format(SetGenerator.dir_path, "test", all_cards_no), all_cards_no)
To train the model and test it, execute the code below. The create method parameter is the number of hidden layer neurons.
from Sets.SetImporter import SetImporter
from Sets.SetGenerator import SetGenerator
from Model.Model import Model
all_cards_no = 5
train_set = SetImporter("{}/poker-hand-{}-{}.data".format(SetGenerator.dir_path, "training", all_cards_no), all_cards_no)
test_set = SetImporter("{}/poker-hand-{}-{}.data".format(SetGenerator.dir_path, "test", all_cards_no), all_cards_no)
model = Model(train_set)
model.create(28)
model.train()
model.test(test_set)
The create method creates a neural network model using the tensorflow library. The network consists of an input layer with 17 neurons (13 rank + 4 suits), one hidden Dense layer, which uses the relu activation function. Depending on the selected stage of the game (2 or 5, 6, 7), the network has one (for 2) or 2 output layers (for 5, 6, 7). The output layer with one neuron determining the probability of winning (regression) uses the sigmoid activation function. The output layer with 10 neurons defining the hand (classification) uses the softmax activation function. The following metrics were used: acc (classification), mse (regression).
def compile(self):
if self.all_cards == 2:
# metrics for regression
self.model.compile(optimizer=tf.train.AdamOptimizer(), loss='mse', metrics=['mse'])
else:
# metrics for classification and regression
self.model.compile(optimizer=tf.train.AdamOptimizer(),
loss={'output_layer_1': 'sparse_categorical_crossentropy', 'output_layer_2': 'mse'},
metrics={'output_layer_1': 'acc', 'output_layer_2': 'mse'})
def create(self, neurons_in_hidden_layer):
if self.dataset is not None:
self.model = keras.models.Sequential()
# 17 = 13 ranks + 4 suits
input_layer = keras.layers.Input((17*self.dataset.all_cards,), name="input_layer")
hidden_layer = keras.layers.Dense(neurons_in_hidden_layer, activation=tf.nn.relu, name="hidden_layer")(input_layer)
# 1 output in NN
if self.dataset.all_cards == 2:
output_layer = keras.layers.Dense(1, activation=tf.nn.sigmoid, name="output_layer")(hidden_layer)
self.model = keras.models.Model(input_layer, output_layer)
# 2 outputs in NN
else:
output_1 = keras.layers.Dense(10, activation=tf.nn.softmax, name="output_layer_1")(hidden_layer)
output_2 = keras.layers.Dense(1, activation=tf.nn.sigmoid, name="output_layer_2")(hidden_layer)
self.model = keras.models.Model(input_layer, [output_1, output_2])
self.compile()
all_cards_no = 2
| neurons_no | loss | mean_squared_error |
|---|---|---|
| 8 | 0.0002728728868532926 |
0.0002728728868532926 |
| 16 | 0.00019329327915329486 | 0.00019329327915329486 |
| 24 | 0.0002716764865908772 | 0.0002716764865908772 |
| 28 | 0.00016631107428111136 | 0.00016631107428111136 |
| 29 | 0.00021212051797192544 | 0.00021212051797192544 |
| 30 | 0.00021880857821088284 | 0.00021880857821088284 |
| 32 | 0.00019229181634727865 | 0.00019229181634727865 |
all_cards_no = 5
| neurons_no | loss | output_layer_1_loss | output_layer_2_loss | output_layer_1_acc | output_layer_2_mean_squared_error |
|---|---|---|---|---|---|
| 8 | 0.8569035530090332 | 0.8307963013648987 | 0.026107240468263626 | 0.6337355375289917 | 0.026107240468263626 |
| 16 | 0.18359050154685974 | 0.1657908409833908 | 0.017799654975533485 | 0.9472096562385559 | 0.017799654975533485 |
| 24 | 0.23562563955783844 | 0.2202180176973343 | 0.015407627448439598 | 0.9678230285644531 | 0.015407627448439598 |
| 28 | 0.24058467149734497 | 0.2283773422241211 | 0.012207326479256153 | 0.9690799117088318 | 0.012207326479256153 |
| 29 | 0.29240182042121887 | 0.2786564528942108 | 0.01374536007642746 | 0.9658119678497314 | 0.01374536007642746 |
| 30 | 0.35394394397735596 | 0.34037789702415466 | 0.013566038571298122 | 0.9615384340286255 | 0.013566038571298122 |
| 32 | 0.39161327481269836 | 0.37983229756355286 | 0.011780978180468082 | 0.9610356688499451 | 0.011780978180468082 |
all_cards_no = 6
| neurons_no | loss | output_layer_1_loss | output_layer_2_loss | output_layer_1_acc | output_layer_2_mean_squared_error |
|---|---|---|---|---|---|
| 8 | 1.1386266946792603 | 1.100320816040039 | 0.03830586373806 | 0.5389643311500549 | 0.03830586373806 |
| 16 | 0.5025573968887329 | 0.4704950153827667 | 0.0320623554289341 | 0.8627451062202454 | 0.0320623554289341 |
| 24 | 0.2888332009315491 | 0.2606068551540375 | 0.02822634018957615 | 0.9263448715209961 | 0.02822634018957615 |
| 28 | 0.33630427718162537 | 0.3104614317417145 | 0.025842837989330292 | 0.9281045794487 | 0.025842837989330292 |
| 29 | 0.3946927785873413 | 0.3673554062843323 | 0.02733737602829933 | 0.9263448715209961 | 0.02733737602829933 |
| 30 | 0.5774040818214417 | 0.5508595705032349 | 0.026544496417045593 | 0.9301156401634216 | 0.026544496417045593 |
| 32 | 0.44998517632484436 | 0.4231707453727722 | 0.02681444026529789 | 0.9147812724113464 | 0.02681444026529789 |
all_cards_no = 7
| neurons_no | loss | output_layer_1_loss | output_layer_2_loss | output_layer_1_acc | output_layer_2_mean_squared_error |
|---|---|---|---|---|---|
| 8 | 1.3301163911819458 | 1.2595701217651367 | 0.0705462396144867 | 0.47561588883399963 | 0.0705462396144867 |
| 16 | 0.9917051196098328 | 0.928206741809845 | 0.063498355448246 | 0.6312217116355896 | 0.063498355448246 |
| 24 | 0.44883644580841064 | 0.39074623584747314 | 0.058090221136808395 | 0.8637506365776062 | 0.058090221136808395 |
| 28 | 0.6418846845626831 | 0.580848217010498 | 0.06103646755218506 | 0.8619909286499023 | 0.06103646755218506 |
| 29 | 0.6029195189476013 | 0.5420128703117371 | 0.06090662255883217 | 0.8501759767532349 | 0.06090662255883217 |
| 30 | 0.5111713409423828 | 0.4530506730079651 | 0.05812065303325653 | 0.8866264224052429 | 0.05812065303325653 |
| 32 | 0.571366548538208 | 0.5106329917907715 | 0.060733549296855927 | 0.8692810535430908 | 0.060733549296855927 |
model.save("{}/model-{}".format(Model.dir_path, all_cards_no))
model.load("{}/model-{}".format(Model.dir_path, all_cards_no), all_cards_no)
Hands are marked with a number. Cards are represented by concatenation of characters: suit and rank.
| Hand number |
Hand |
|---|---|
| 0 | Royal Straight Flush |
| 1 |
Straight Flush |
| 2 | Four of a Kind |
| 3 | Full House |
| 4 | Flush |
| 5 | Straight |
| 6 | Three of a Kind |
| 7 | Two Pair |
| 8 | Pair |
| 9 | High Card |
| Rank | Char |
|---|---|
| 2,3,...,9 | 2,3,...,9 |
| 10 |
10 |
| Jack | J |
| Queen | Q |
| King | K |
| Ace | A |
| Suit | Char |
|---|---|
| Spades | s |
| Hearts | h |
| Diamonds | d |
| Clubs | c |
The prediction for the game stage with 2 cards in hand returns the probability of winning. For other stages, the hand number (the strongest) is additionally returned.
print(model.predict(['Qs', 'Qh', 'Qd', 'Kc', 'Kh']))
from Model.Model import Model
all_cards_no = 5
model = Model()
model.load("{}/model-{}".format(Model.dir_path, all_cards_no), all_cards_no)
print(model.predict(['Qs', 'Qh', 'Qd', 'Kc', 'Kh']))
(3, 0.89730614)
- https://github.com/worldveil/deuces
- https://github.com/Blind0ne/Monte-Carlo
- Monika Grądzka
- Robert Kazimierczak