🧠 NN: Aurora-20581290fd

[unixwizard]

🎯 Summary

This PR improves upon the new Aurora architecture by optimizing some of the hyper-parameters. More experimentations are likely possible with batch size and learning rate.

🧬 Architecture and Hyper-Parameter

       IN          ACCUMULATOR                               HIDDEN                                    OUT
 ______________      _______      ______________________________________________________________      _____
| WHITE: (768) | -> | (384) | -> | ClippedReLU(0, 1) -> (384) \                                 |    |     |
|              |    |       |    |                             \                                |    |     |
|              |    |       |    |                              CONCATENATE(ColorToMove): (768) | -> | (1) |
|              |    |       |    |                             /                                |    |     |
| BLACK: (768) | -> | (384) | -> | ClippedReLU(0, 1) -> (384) /                                 |    |     |
 --------------      -------      --------------------------------------------------------------      -----

Codename: Aurora
ID: 20581290fd

Data: 1.7B FEN --- DEPTH: 9 (1.2B) + NODES: 5K (500M)
Batch Size: 750K
Epochs: 120
LR: 9e-3
LR Drop Step: 1
LR Drop Last: 120
LR Gamma: 0.985
WDL: 0.1
Scale: 400

👏 Acknowledgements

NA

📈 ELO

STC:

ELO   | 3.93 +- 3.14 (95%)
SPRT  | 10.0+0.10s Threads=1 Hash=16MB
LLR   | 2.97 (-2.94, 2.94) [0.00, 5.00]
GAMES | N: 28504 W: 8835 L: 8513 D: 11156

LTC:

ELO   | 3.71 +- 2.98 (95%)
SPRT  | 60.0+0.60s Threads=1 Hash=256MB
LLR   | 2.95 (-2.94, 2.94) [0.00, 5.00]
GAMES | N: 29136 W: 8295 L: 7984 D: 12857