The neural network of the Stockfish chess engine

Ironically, a lot of the tricks Stockfish is using here are reminiscent of tricks that were used in the original AlphaGo and later discarded in AlphaGoZero.<p>In particular, the AlphaGo paper mentioned four neural networks of significance:<p>- a policy network trained on human pro games. - a RL-enhanced policy network improving on the original SL-trained policy network. - a value network trained on games generated by the RL-enhanced policy network - a cheap policy network trained on human pro games, used only for rapid rollout simulations.<p>The cheap rollout policy network was discarded because DeepMind found that a &quot;slow evaluations of the right positions&quot; was better than &quot;rapid evaluations of questionable positions&quot;. The independently trained value network was discarded because co-training a value and policy head on a shared trunk saved a significant amount of compute, and helped regularize both objectives against each other. The RL-enhanced policy network was discarded in favor of training the policy network to directly replicate MCTS search statistics.<p>The depth and branching factor in chess and Go are different, so I won&#x27;t say the solutions ought to be the same, but it&#x27;s interesting nonetheless to see the original AlphaGo ideas be resurrected in this form.<p>The incremental updates are also related to Zobrist Hashing, which the Stockfish authors are certainly aware of.

If anyone wants to experiment with training these nets, it&#x27;s a great way to get exposed to a nice mix of chess and machine learning.<p>There are two trainers currently, the original one, which runs on CPU: <a href="https:&#x2F;&#x2F;github.com&#x2F;nodchip&#x2F;Stockfish" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;nodchip&#x2F;Stockfish</a>, and a pytorch one which runs on GPU: <a href="https:&#x2F;&#x2F;github.com&#x2F;glinscott&#x2F;nnue-pytorch" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;glinscott&#x2F;nnue-pytorch</a>.<p>The SF Discord is where all of the discussion&#x2F;development is happening: <a href="https:&#x2F;&#x2F;discord.gg&#x2F;KGfhSJd" rel="nofollow">https:&#x2F;&#x2F;discord.gg&#x2F;KGfhSJd</a>.<p>Right now there is a lot of experimentation to try adjusting the network architecture. The current leading approach is a much larger net which takes in attack information per square (eg. is this piece attacked by more pieces than it&#x27;s defended by?). That network is a little slower, but the additional information seems to be enough to be stronger than the current architecture.<p>Btw, the original Shogi developers really did something amazing. The nodchip trainer is all custom code, and trains extremely strong nets. There are all sorts of subtle tricks embedded in there as well that led to stronger nets. Not to mention, getting the quantization (float32 -&gt; int16&#x2F;int8) working gracefully is a huge challenge.

This is very nice. Reminds me a lot of tricks used for simple linear models. And it seems to work, given that Leela Chess Zero is losing with quite a gap.<p>Most of the times one would learn a model by just changing a single feature and then doing the whole sum made no sense.<p>A good example is learning a sequence of decisions, where each decision might have a cost associated to it, you can then say that the current decision depends on a previous one and vary the previous one to learn to recover from errors. If previous decision was bad, then you&#x27;d still like to make the best decision for current state.).<p>So even if your training data does not have this error-recovery example, you can just iterate through all previous decisions and make the model learn to recover.<p>An optimization in that case would be to just not redo the whole sum (for computing the decision function of a linear model).

It seems that the strategy is to use the neural network to score various moves and then a search strategy to try to find moves that result in a favourable score. And this post focuses on some of the technical engineering details to design such a scoring network. In particular the scoring is split into two parts: 1. A matrix multiplication by a sparse input vector to get a dense representation of a position, and 2. Some nonlinear and further layers after this first step. And it seems that step 1 is considered more expensive.<p>The way this is made cheap is by making it incremental: given some board s and output of the first layer b + Ws, it is cheap to compute b + Wt where a t is a board that is similar to s (the difference is W(t-s) but the vector t-s is 0 in almost every element.)<p>This motivates some of the engineering choices like using integers instead of floats. If you used floats then this incremental update wouldn’t work.<p>It seems to me that a lot of the smarts of stockfish will be in the search algorithm getting good results, but I don’t know if that just requires a bit of parallelism (surely some kind of work-stealing scheduler) and brute force or if it mostly relies on some more clever strategies. And maybe I’m wrong and the key is really in the scoring of positions.

Leela played stockfish 200 games and won with 106 - 94 [1]. Not sure which version of stockfish was used.<p>Some of the Leela-Stockfish games are analyzed by agadmator on YouTube [2].<p>[1] <a href="https:&#x2F;&#x2F;www.chess.com&#x2F;news&#x2F;view&#x2F;13th-computer-chess-championship-leela-chess-zero-stockfish" rel="nofollow">https:&#x2F;&#x2F;www.chess.com&#x2F;news&#x2F;view&#x2F;13th-computer-chess-champion...</a><p>[2] <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=YtXZjKItuC8" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=YtXZjKItuC8</a>

The gap between white and black piece performance is massive in these top engines if I&#x27;m reading it right. LCZero won 0&#x2F; lost 4 as black and won 24 &#x2F; lost zero as white (with lots of draws)? I had no idea the split was so big now. Do human tournaments look like this too these days? (From <a href="https:&#x2F;&#x2F;tcec-chess.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;tcec-chess.com&#x2F;</a>)

Using a previous Stockfish scorer they trained an NN without any labelling effort. This is also similar to how unsupervised translation is done in some methods. They start from word-&gt;word dictionary results and iteratively train lang1-&gt;lang2 and lang2-&gt;lang1 models feeding on each other&#x27;s output.

Great writeup!<p>One thing I don&#x27;t understand is why it would be smarter to augment the inputs with some of the missing board information - particularly the availability of castling. Even though this network is a quick-and-dirty evaluation, seems like there&#x27;s room for a few additional bits in the input and being able to castle very much changes the evaluation of some common positions.

I don&#x27;t know a lot about chess but I have one question: isn&#x27;t chess a solved game (in the sense that given a board state we always can compute the right move)? why use a neural network that can introduce, a very small percentage of the time, mistakes? I guess it is for performance reasons?

This is a really good article.<p>Lots of pieces about neural network design skip over the design of the representation in the input stage, which is one of the key design issues when building a custom neural network.<p>I love how much depth this articles goes into about that representation.

Analyses like this are a great indication that the main effect of refining neural networks around chess will make neural networks more exciting and ultimately make chess more boring.