Ablating 1 of a chess transformer's 128 attention heads causes the model to stop finding the queen sacrifice in a famous chess game. [P]

A recent study explored the impact of ablating a single attention head from a chess transformer model. Researchers used the chessformer_lens library to analyze the Maia-3 23m model. They found that removing just 1 of the model's 128 attention heads prevented it from identifying a queen sacrifice in a well-known chess game. This highlights the critical role individual attention heads play in complex decision-making processes within transformer architectures, even in highly specialized domains like chess.