Skip to content

Commit

Permalink
Update default net to nn-8e47cf062333.nnue
Browse files Browse the repository at this point in the history
This net is the result of training on data used by the Leela project. More precisely,
we shuffled T60 and T74 data kindly provided by borg (for different Tnn, the data is
a result of Leela selfplay with differently sized Leela nets).

The data is available at vondele's google drive:
https://drive.google.com/drive/folders/1mftuzYdl9o6tBaceR3d_VBQIrgKJsFpl.

The Leela data comes in small chunks of .binpack files. To shuffle them, we simply
used a small python script to randomly rename the files, and then concatenated them
using `cat`. As validation data we picked a file of T60 data. We will further investigate
T74 data.

The training for the NNUE architecture used 200 epochs with the Python trainer from
the Stockfish project. Unlike the previous run we tried with this data, this run does
not have adjusted scaling — not because we didn't want to, but because we forgot.
However, this training randomly skips 40% more positions than previous run. The loss
was very spiky and decreased slower than it does usually.

Training loss: https://github.com/official-stockfish/images/blob/main/training-loss-8e47cf062333.png
Validation loss: https://github.com/official-stockfish/images/blob/main/validation-loss-8e47cf062333.png

This is the exact training command:
python train.py --smart-fen-skipping --random-fen-skipping 14 --batch-size 16384 --threads 4 --num-workers 4 --gpus 1 trainingdata\training_data.binpack validationdata\val.binpack

---

10k STC result:
ELO: 3.61 +-3.3 (95%) LOS: 98.4%
Total: 10000 W: 1241 L: 1137 D: 7622
Ptnml(0-2): 68, 841, 3086, 929, 76
https://tests.stockfishchess.org/tests/view/60c67e50457376eb8bcaae70

10k LTC result:
ELO: 2.71 +-2.4 (95%) LOS: 98.8%
Total: 10000 W: 659 L: 581 D: 8760
Ptnml(0-2): 22, 485, 3900, 579, 14
https://tests.stockfishchess.org/tests/view/60c69deb457376eb8bcaae98

Passed LTC:
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 9648 W: 685 L: 545 D: 8418
Ptnml(0-2): 22, 448, 3740, 596, 18
https://tests.stockfishchess.org/tests/view/60c6d41c457376eb8bcaaecf

---

closes #3550

Bench: 4877339
  • Loading branch information
JWmer authored and snicolet committed Jun 14, 2021
1 parent ce4c523 commit f8c779d
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 2 deletions.
3 changes: 2 additions & 1 deletion AUTHORS
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# List of authors for Stockfish, as of May 17, 2021
# List of authors for Stockfish, as of June 14, 2021

# Founders of the Stockfish project and fishtest infrastructure
Tord Romstad (romstad)
Expand Down Expand Up @@ -96,6 +96,7 @@ Joost VandeVondele (vondele)
Jörg Oster (joergoster)
Joseph Ellis (jhellis3)
Joseph R. Prostko
Julian Willemer (NightlyKing)
jundery
Justin Blanchard (UncombedCoconut)
Kelly Wilson
Expand Down
2 changes: 1 addition & 1 deletion src/evaluate.h
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ namespace Eval {
// The default net name MUST follow the format nn-[SHA256 first 12 digits].nnue
// for the build process (profile-build and fishtest) to work. Do not change the
// name of the macro, as it is used in the Makefile.
#define EvalFileDefaultName "nn-7e66505906a6.nnue"
#define EvalFileDefaultName "nn-8e47cf062333.nnue"

namespace NNUE {

Expand Down

0 comments on commit f8c779d

Please sign in to comment.