Distilling Stockfish on a Billion Positions, Full 3.9B Dataset Available [P]

A project distilled the Stockfish value function into a ResNet/ViT model using 1 billion positions from the Gigafish dataset. The full 3.9 billion position dataset, built from 37 months of Lichess games, is available on Hugging Face. While a vision transformer was slow to understand the board, a CNN was more effective early in training due to its geometric inductive biases. The best results were achieved by combining both approaches.

This project is notable for distilling Stockfish on a billion positions and making the full 3.9 billion position dataset publicly available, unlike previous, smaller datasets.