What makes a chess move brilliant? Researchers use AI to find out

Researchers at the University of Toronto have designed a new AI model that understands how humans perceive creativity in chess.  

In a recent paper presented at an international conference, researchers in U of T’s Faculty of Applied Science & Engineering describe how they used techniques such as game trees and deep neural networks to enable chess engines to recognize brilliant moves.  

The development could lead to chess engines that can find the most creative and clever path to victory in game, rather than just making moves to maximize win rates. That, in turn, could have implications for other AI systems tasked with creative endeavours.  

“A chess move can be perceived as brilliant, or creative, when the strategic payoff isn’t clear at first, but in retrospect the player had to follow a precise path in gaming out all the possibilities to see so far into the future,” says paper co-author Michael Guerzhoy, an assistant professor, teaching stream, of mechanical and industrial engineering and engineering science who wrote about the research on his Substack.  

“We wanted our system to understand human perception of what constitutes brilliance in chess and distinguish that from just winning.”  

Most of the current research into chess AI is focused on enabling moves that create a higher chance of winning. But this doesn’t always make for an exciting game.   

Skilled human chess players, on the other hand, can play in a more dramatic or imaginative way by making moves that may break traditional rules –for example, sacrificing a piece in a way that may initially look like a mistake, but ultimately, paves the way to a win.  

The team worked with Leela Chess Zero, a top chess engine that learns through self-play and has played over 1.6 billion games against itself. They also employed Maia, a human-like neural network chess engine developed by U of T computer science researchers.   

“We used the two neural network chess engines to create our game trees at different levels of depth in a game,” says paper co-author Kamron Zaidi, a recent U of T Engineering graduate. 

“Using these game trees, we extracted many different features from it. We then fed the features into a neural network that we trained on the Lichess database of online chess games, which are labelled by human users of the database.” 

A game tree in chess represents the current state of a chess board along with all the possible moves and counter moves that can occur. Each board position is represented as a node and the game tree can be expanded on until the game is either won, drawn or lost. 

The researchers began with small game trees then slowly increased the size, adding more nodes to the tree. They found that when the neural network looks at all the game tree features and makes a prediction as to whether the move is brilliant or not, it reached an accuracy rate of 79 per cent using the test data set.  

The research – based on Zaidi’s undergraduate engineering science thesis, which was supervised by Guerzhoy – was presented at the International Conference on Computational Creativity in Jönköping, Sweden.   

“There were people from all over the world presenting research on more traditional aspects of creativity, but we were all focused on the same thing, which is, ‘How can we use AI to enhance our interactions and understandings of creativity?’” says Zaidi.  

The work has also received media coverage in outlets, including New Scientist, where English chess grandmaster Matthew Sadler says that a model that can understand brilliance could be used as a training tool for professionals and potentially lead to a more entertaining engine opponent for amateur players.  

The team sees their system as having broad applicability when it comes to perception of creativity and brilliance.  

“One of the biggest areas that is of interest to me is characterizing what we perceive as creativity,” says Guerzhoy.   

“Not just in board games but in other creative endeavours, including music and art, where there is a formal framework and rules that need to be followed. Highly creative work involves planning in advance and gaming out the possibilities. 

“But everyone I’ve talked to since the paper came out wants to know when they can play against our brilliant chess engine. So, I think making that possible is the obvious next step for us.”