Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UniversalPoker/ACPC fixes: fullgame+abstracted MaxGameLength, fullgame LegalActions #1035

Merged
merged 7 commits into from
Mar 20, 2023
Merged

UniversalPoker/ACPC fixes: fullgame+abstracted MaxGameLength, fullgame LegalActions #1035

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3e1020a
add MaxGameLength for fullgame
VitamintK Mar 13, 2023
4a44987
give a tighter upperbound for MaxGameLength with abstractions
VitamintK Mar 13, 2023
3a7bb5c
acpc, fullgame, fix LegalActions so it does not return raises when th…
VitamintK Mar 13, 2023
4aa2431
add a test
VitamintK Mar 14, 2023
b3aa36f
also add an unabstracted universal poker playthrough
VitamintK Mar 14, 2023
ee5e6f5
add a testcase for unabstracted universal poker, and re-run playthrough
VitamintK Mar 14, 2023
e0f464a
acpc MaxGameLength: apply jhtschultz's simpler algorithm
VitamintK Mar 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 15 additions & 6 deletions open_spiel/games/universal_poker.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -755,6 +755,9 @@ std::vector<Action> UniversalPokerState::LegalActions() const {
}
return legal_actions;
} else {
if (acpc_state_.IsFinished()) {
return legal_actions;
}
if (acpc_state_.IsValidAction(
acpc_cpp::ACPCState::ACPCActionType::ACPC_FOLD, 0)) {
legal_actions.push_back(kFold);
Expand Down Expand Up @@ -972,8 +975,6 @@ UniversalPokerGame::UniversalPokerGame(const GameParameters &params)
potSize_(ParameterValue<int>("potSize")),
boardCards_(ParameterValue<std::string>("boardCards")),
handReaches_(ParameterValue<std::string>("handReaches")) {
max_game_length_ = MaxGameLength();
SPIEL_CHECK_TRUE(max_game_length_.has_value());
std::string betting_abstraction =
ParameterValue<std::string>("bettingAbstraction");
if (betting_abstraction == "fc") {
Expand All @@ -988,6 +989,8 @@ UniversalPokerGame::UniversalPokerGame(const GameParameters &params)
SpielFatalError(absl::StrFormat("bettingAbstraction: %s not supported.",
betting_abstraction));
}
max_game_length_ = MaxGameLength();
SPIEL_CHECK_TRUE(max_game_length_.has_value());
}

std::unique_ptr<State> UniversalPokerGame::NewInitialState() const {
Expand Down Expand Up @@ -1091,10 +1094,16 @@ int UniversalPokerGame::MaxGameLength() const {
maxBlind =
acpc_game_.BlindSize(p) > maxBlind ? acpc_game_.BlindSize(p) : maxBlind;
}

while (maxStack > maxBlind) {
maxStack /= 2.0; // You have always to bet the pot size
length += NumPlayers(); // Each player has to react
if ((betting_abstraction_==BettingAbstraction::kFULLGAME) || (betting_abstraction_==BettingAbstraction::kFCHPA)){
// with fullgame, the longest game comes from each player can bet/raise the big blind every action.
// with FCHPA, the longest game is when each player bets/raise half-pot every action.
// however, for now we'll just use the fullgame value for FCHPA too, although it is a big overestimate.
length += (maxStack+maxBlind-1)/maxBlind;
} else {
while (maxStack > maxBlind) {
maxStack /= 2.0; // You have always to bet the pot size
length += NumPlayers() - 1; // 1 player bets, and n-2 players call
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it n-2 players call and not n-1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. Here's my thought process. Also let me know if you see anything wrong with the reasoning.

The longest game (with a single betting round, for simplicity) consists of:

n-1 players checking,
1 player betting, n-2 players calling,
1 player raising, n-2 players calling,
etc...,
1 player raising, n-1 players calling

so each min-bet/min-raise is succeeded by n-2 calls (except for the last bet, which is succeeded by n-1). In addition, there are n-1 checks in the beginning. These n-1 checks + the final 1 call add up to n, which is accounted for a few lines above in the code (// Check Actions).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it's probably convenient for future readers of the code if I just put this comment as a code comment. What do ya think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification. This reasoning makes sense, but there's still a few points to resolve.

First, the calculation on line 1101 only accounts for the number of raises, not all of the n-2 calls in between. So I believe that has to be updated.

Second, maybe there's a simpler way to structure this that better handles all betting abstractions. The goal is to figure out what the max number of raises is and then apply your logic regarding max number of check/calls.

max_num_raises = 0
if betting_abstraction == kFC:
  pass  # no raises allowed
elif betting_abstraction == kFCPA:
  pot_size = maxBlind * num_players
  while pot_size / num_players < maxStack:
    max_num_raises += 1
    pot_size += pot_size * num_players
elif betting_abstraction == kFCHPA:
  pot_size = maxBlind * num_players
  while pot_size / num_players < maxStack:
    max_num_raises += 1
    pot_size += pot_size / 2 * num_players
elif betting_abstraction == fullgame:
  max_num_raises = (maxStack+maxBlind-1)/maxBlind

length += max_num_raises * (num_players - 1)

I think this is more intuitive than dividing the max stack size in half. I totally might be missing something on either of these points though so let me know what you think. Funny how even these simple things can be tricky!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First, the calculation on line 1101 only accounts for the number of raises, not all of the n-2 calls in between. So I believe that has to be updated.

ah yes, thanks. 😅 good catch!

I think this is more intuitive than dividing the max stack size in half.

Yup, definitely. The logic in your snippet all looks good, thanks!

Funny how even these simple things can be tricky!

On the bright side, this has finally made me learn how to calculate a pot-sized bet.

}
}
return length;
}
Expand Down
59 changes: 59 additions & 0 deletions open_spiel/games/universal_poker_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,6 +436,64 @@ void FullNLBettingTest3() {
":2c2d|2h2s|3c3d/3h3s4c/4d/4h"));
}

// Check that a max length game works and infostate tensors are all unique.
void FullNLBettingTest4() {
std::shared_ptr<const Game> game = LoadGame(
"universal_poker(betting=nolimit,"
"numPlayers=2,"
"numRounds=2,"
"blind=100 50,"
"numSuits=1,"
"numRanks=4,"
"numHoleCards=1,"
"numBoardCards=0 1,"
"stack=2000 2000,"
"bettingAbstraction=fullgame)");
std::set<std::vector<float>> information_state_tensor_set;
std::vector<float> tensor;
std::unique_ptr<State> state = game->NewInitialState();
SPIEL_CHECK_EQ(game->NumDistinctActions(), 2001);
// deal cards
while (state->IsChanceNode()) state->ApplyAction(state->LegalActions()[0]);
// check the infostate tensor and add to set
tensor = state->InformationStateTensor();
SPIEL_CHECK_FALSE(information_state_tensor_set.count(tensor));
information_state_tensor_set.insert(tensor);
state->ApplyAction(1); // check
// check the infostate tensor and add to set
tensor = state->InformationStateTensor();
SPIEL_CHECK_FALSE(information_state_tensor_set.count(tensor));
information_state_tensor_set.insert(tensor);
state->ApplyAction(200); //min bet
// check the infostate tensor and add to set
tensor = state->InformationStateTensor();
SPIEL_CHECK_FALSE(information_state_tensor_set.count(tensor));
information_state_tensor_set.insert(tensor);
state->ApplyAction(1); // call
state->ApplyAction(state->LegalActions()[0]); // deal flop
// check the infostate tensor and add to set
tensor = state->InformationStateTensor();
SPIEL_CHECK_FALSE(information_state_tensor_set.count(tensor));
information_state_tensor_set.insert(tensor);
state->ApplyAction(1); // check
// check the infostate tensor and add to set
tensor = state->InformationStateTensor();
SPIEL_CHECK_FALSE(information_state_tensor_set.count(tensor));
information_state_tensor_set.insert(tensor);
for (int i=300; i < 2000; i+=100){
state->ApplyAction(i); // min bet/raise
// check the infostate tensor and add to set
tensor = state->InformationStateTensor();
SPIEL_CHECK_FALSE(information_state_tensor_set.count(tensor));
information_state_tensor_set.insert(tensor);
}
state->ApplyAction(1); // call
SPIEL_CHECK_EQ(state->LegalActions().size(), 0);
std::cout << state->ToString() << std::endl;
SPIEL_CHECK_TRUE(absl::StrContains(state->ToString(),
"ACPC State: STATE:0:cr200c/cr300r400r500r600r700r800r900r1000r1100r1200r1300r1400r1500r1600r1700r1800r1900c:2c|3c/4c"));
}

void ChanceDealRegressionTest() {
std::shared_ptr<const Game> game = LoadGame(
"universal_poker(betting=nolimit,"
Expand Down Expand Up @@ -714,6 +772,7 @@ int main(int argc, char **argv) {
open_spiel::universal_poker::FullNLBettingTest1();
open_spiel::universal_poker::FullNLBettingTest2();
open_spiel::universal_poker::FullNLBettingTest3();
open_spiel::universal_poker::FullNLBettingTest4();
open_spiel::universal_poker::HulhMaxUtilityIsCorrect();
open_spiel::universal_poker::CanConvertActionsCorrectly();
open_spiel::universal_poker::TestFCHPA();
Expand Down
Loading