Skip to content

Commit

Permalink
Update UCI_Elo parameterization
Browse files Browse the repository at this point in the history
The old parameterization (https://github.com/official-stockfish/Stockfish/pull/2225/files) has now become quite inaccurate.
This updates the formula based on updated results with master. The formula is based on a fit of the Elo results for games
played between master at various skill levels, and various versions of the Stash engine, which have been ranked at CCRL.

```
   # PLAYER             :  RATING  ERROR  POINTS  PLAYED   (%)
   1 master-skill-19    :  3191.1   40.4   940.0    1707    55
   2 master-skill-18    :  3170.3   39.3  1343.0    2519    53
   3 master-skill-17    :  3141.3   37.8  2282.0    4422    52
   4 master-skill-16    :  3111.2   37.1  2773.0    5423    51
   5 master-skill-15    :  3069.5   37.2  2728.5    5386    51
   6 master-skill-14    :  3024.8   36.1  2702.0    5339    51
   7 master-skill-13    :  2972.9   35.4  2645.5    5263    50
   8 master-skill-12    :  2923.1   35.0  2653.5    5165    51
   9 master-skill-11    :  2855.5   33.6  2524.0    5081    50
  10 master-skill-10    :  2788.3   32.0  2724.5    5511    49
  11 stash-bot-v25.0    :  2744.0   31.5  1952.5    3840    51
  12 master-skill-9     :  2702.8   30.5  2670.0    5018    53
  13 master-skill-8     :  2596.2   28.5  2669.5    4975    54
  14 stash-bot-v21.0    :  2561.2   30.0  1338.0    3366    40
  15 master-skill-7     :  2499.5   28.5  1934.0    4178    46
  16 stash-bot-v20.0    :  2452.6   27.7  1606.5    3378    48
  17 stash-bot-v19.0    :  2425.3   26.7  1787.0    3365    53
  18 master-skill-6     :  2363.2   26.4  2510.5    4379    57
  19 stash-bot-v17.0    :  2280.7   25.4  2209.0    4378    50
  20 master-skill-5     :  2203.7   25.3  2859.5    5422    53
  21 stash-bot-v15.3    :  2200.0   25.4  1757.0    4383    40
  22 stash-bot-v14      :  2145.9   25.5  2890.0    5167    56
  23 stash-bot-v13      :  2042.7   25.8  2263.5    4363    52
  24 stash-bot-v12      :  1963.4   25.8  1769.5    4210    42
  25 master-skill-4     :  1922.9   25.9  2690.0    5399    50
  26 stash-bot-v11      :  1873.0   26.3  2203.5    4335    51
  27 stash-bot-v10      :  1783.8   27.8  2568.5    4301    60
  28 master-skill-3     :  1742.3   27.8  1909.5    4439    43
  29 master-skill-2     :  1608.4   29.4  2064.5    4389    47
  30 stash-bot-v9       :  1582.6   30.2  2130.0    4230    50
  31 master-skill-1     :  1467.6   31.3  2015.5    4244    47
  32 stash-bot-v8       :  1452.8   31.5  1953.5    3780    52
  33 master-skill-0     :  1320.1   32.9   651.5    2083    31
```

Skill 0 .. 19, now covers CCRL Blitz Elo from 1320 to 3190, approximately.
Indeed, the Elo of stash in this analysis is only to within +- 100 Elo of CCRL,
probably because it depends quite a bit on the opponent pool.

To obtain a skill level for a given Elo number, the above data is fit as a 3rd
degree polynomial Skill(Elo). A quick test confirms the correspondence to the above table:

```
Score of master-elo-2721 vs stash-bot-v21.0: 51 - 16 - 19  [0.703] 86
Elo difference: 150.1 +/- 70.2, LOS: 100.0 %, DrawRatio: 22.1 %
```

closes #4341

No functional change.
  • Loading branch information
vondele committed Jan 22, 2023
1 parent da5bcec commit a08b8d4
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 3 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ change them via a chess GUI. This is a list of available UCI options in Stockfis

* #### UCI_Elo
If enabled by UCI_LimitStrength, aim for an engine strength of the given Elo.
This Elo rating has been calibrated at a time control of 60s+0.6s and anchored to CCRL 40/4.
This Elo rating has been calibrated at a time control of 120s+1.0s and anchored to +- 100 Elo to CCRL Blitz.

* #### Skill Level
Lower the Skill Level in order to make Stockfish play weaker (see also UCI_LimitStrength).
Expand Down
5 changes: 4 additions & 1 deletion src/search.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,10 @@ namespace {
struct Skill {
Skill(int skill_level, int uci_elo) {
if (uci_elo)
level = std::clamp(std::pow((uci_elo - 1346.6) / 143.4, 1 / 0.806), 0.0, 20.0);
{
double e = double(uci_elo - 1320) / (3190 - 1320);
level = std::clamp((((37.2473 * e - 40.8525) * e + 22.2943) * e - 0.311438), 0.0, 19.0);
}
else
level = double(skill_level);
}
Expand Down
2 changes: 1 addition & 1 deletion src/ucioption.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ void init(OptionsMap& o) {
o["UCI_Chess960"] << Option(false);
o["UCI_AnalyseMode"] << Option(false);
o["UCI_LimitStrength"] << Option(false);
o["UCI_Elo"] << Option(1350, 1350, 2850);
o["UCI_Elo"] << Option(1320, 1320, 3190);
o["UCI_ShowWDL"] << Option(false);
o["SyzygyPath"] << Option("<empty>", on_tb_path);
o["SyzygyProbeDepth"] << Option(1, 1, 100);
Expand Down

0 comments on commit a08b8d4

Please sign in to comment.