Chess Evaluation Positional Parameter Optimization
Parameters to be optimized:
'KingAttackWeight': {'default': 100, 'min': 50, 'max': 150, 'step': 5}
'MobilityWeight': {'default': 100, 'min': 50, 'max': 150, 'step': 3}
'PassedPawnWeightOp': {'default': 100, 'min': 50, 'max': 150, 'step': 5}The default values are Deuterium's default and considered to be the best. The base engine will start from these values and the test engine will take the suggested values from optuna optimizer. Tuning these parameters are tricky.
- First they are very sensitive to time control (unlike piece value parameters). That is why I use tc=20s+100ms in tuning slightly above my game verification tc of 15s+100ms. The tuning time control of 20s+100ms in each game takes time.
- Second in order for the parameter values to be considered the best and replace the old best, it has to defeat the old best by more than 0.55 or 55% score. Normally this is only 0.5 or 50%.
python tuner.py --engine ./engines/deuterium/deuterium.exe --opening-file ./start_opening/ogpt_chess_startpos.epd --concurrency 6 --games-per-trial 24 --base-time-sec 20 --inc-time-sec 0.10 --trials 1000 --study-name study_positional_1 --initial-best-value 0.55
trials: 1000, games_per_trial: 24
input param: OrderedDict([('KingAttackWeight', {'default': 100, 'min': 50, 'max': 150, 'step': 5}), ('MobilityWeight', {'default': 100, 'min': 50, 'max': 150, 'step': 3}), ('PassedPawnWeightOp', {'default': 100, 'min': 50, 'max': 150, 'step': 5})])
[I 2020-09-15 16:59:06,075] A new study created in RDB with name: study_positional_1
Warning, best param from previous trial is not found!, use an init param based from input param.
init best param: {'KingAttackWeight': 100, 'MobilityWeight': 100, 'PassedPawnWeightOp': 100}
Warning, init value is not found!, use init best value 0.55.
init best value: 0.55
starting trial: 0 ...
suggested param: {'KingAttackWeight': 90, 'MobilityWeight': 65, 'PassedPawnWeightOp': 60}
init param: {'KingAttackWeight': 100, 'MobilityWeight': 100, 'PassedPawnWeightOp': 100}
init value: 0.55
[I 2020-09-15 17:01:44,737] Trial 0 finished with value: 0.55 and parameters: {'KingAttackWeight': 90, 'MobilityWeight': 65, 'PassedPawnWeightOp': 60}. Best is trial 0 with value: 0.55.
starting trial: 1 ...
...
starting trial: 188 ...
suggested param: {'KingAttackWeight': 135, 'MobilityWeight': 71, 'PassedPawnWeightOp': 105}
best param: {'KingAttackWeight': 145, 'MobilityWeight': 62, 'PassedPawnWeightOp': 90}
best value: 0.5656524691289315
[I 2020-09-16 08:12:35,176] Trial 188 finished with value: 0.565923524298792 and parameters: {'KingAttackWeight': 135, 'MobilityWeight': 71, 'PassedPawnWeightOp': 105}. Best is trial 188 with value: 0.565923524298792.
starting trial: 189 ...
...
starting trial: 201 ...
suggested param: {'KingAttackWeight': 150, 'MobilityWeight': 68, 'PassedPawnWeightOp': 95}
best param: {'KingAttackWeight': 135, 'MobilityWeight': 71, 'PassedPawnWeightOp': 105}
best value: 0.565923524298792
[I 2020-09-16 08:53:36,684] Trial 201 finished with value: 0.479 and parameters: {'KingAttackWeight': 150, 'MobilityWeight': 68, 'PassedPawnWeightOp': 95}. Best is trial 188 with value: 0.565923524298792.
best param: {'KingAttackWeight': 135, 'MobilityWeight': 71, 'PassedPawnWeightOp': 105}
best value: 0.565923524298792
best trial number: 188The best parameters
best param: {'KingAttackWeight': 135, 'MobilityWeight': 71, 'PassedPawnWeightOp': 105}
after 201 trials are used by deuterium_positional_study_201_trials. It won against the default
init best param: {'KingAttackWeight': 100, 'MobilityWeight': 100, 'PassedPawnWeightOp': 100}
after 1000 games at tc=15s+100ms.
Score of deuterium_positional_study_201_trials vs deuterium_default: 250 - 203 - 547 [0.523] 1000
... deuterium_positional_study_201_trials playing White: 137 - 89 - 274 [0.548] 500
... deuterium_positional_study_201_trials playing Black: 113 - 114 - 273 [0.499] 500
... White vs Black: 251 - 202 - 547 [0.524] 1000
Elo difference: 16.3 +/- 14.5, LOS: 98.6 %, DrawRatio: 54.7 %
Finished match
Contour plot is crazy there are some areas that are still light, meaning optimization can still be continued and might still optimize the current best parameters. Or these areas are already good and probably more than 50% already since I started with more than 55%.
