Chess checkmates - a simple neural network

Introduction

Chess is a very old and very complex game. Humans have been playing chess and chess variants for thousands of years. I began playing chess somewhat seriously around 2015, and it has quickly become one of my favourite games. You can find a good primer for learning the rules of chess here.

Hobbyists, scientists, and professionals have long sought to look deeper into the nature of chess and explore it’s complexities. A natural course of study was to program computers to be able to play chess against humans and, eventually, beat them. This work reached a seminal moment in 1997 when IBMs Deep Blue supercomputer defeated the world champion of chess Garry Kasparov in a series of 6 games. From then on chess engines only became stronger and soon all top chess engines could easily defeat any human.

However, how typical chess engines play chess is very unlike how humans play. A chess engine such as Stockfish or Komodo will play chess through sheer calculation. That is, chess engines will utilize a computers ability to peform many ordered tasks quickly and efficiently to calculate the strength of thousands of series of moves in any given position, and then choose the best option. Such a strategey is far beyond any humans ability. Strong human chess players will choose their moves through a combination of experience, intuitition, and manual calculation.

In the past several years chess engines have been developed that do not rely purely on brute force calculation. The DeepMind company developed a program called AlphaZero that has mastered chess (along with go and shogi) through neural networks (NNs). A similar neural net based program, Leela Chess Zero, is open source and is an ongoing project. I will discuss neural networks in more detail later.

All of this recent work involving neural net based chess engines, as well as the (relatively) recent explosion of interest in artificial intelligence (AI), piqued my interest in further applications of AI in chess.

A particularly popular application of NNs is image classification, that is training a model to accurately predict a label across a given set of images. I have experience working with large sets of images for statistical applications before, and thought it would be fun to investigate a NN applied to large sets of chess images. I figured checkmates would be promising as a starting point. In chess, checkmate is the win condition of the game. It is achieved when a player’s king is attacked by an opposing piece, and every accessible ajoining square the king may move to is also attacked by an opposing piece.

*The black king is attacked by the white queen, and his escape route via f7 is controlled by the white pawn on e6*

Figure 1: The black king is attacked by the white queen, and his escape route via f7 is controlled by the white pawn on e6

I decided upon investigating checkmates because of their inherent simplicity. Any given chess position is either checkmate or not checkmate. This is a binary outcome, and binary classification is a well studied application of NN modelling. My current background is in undergraduate math and statistics, I have no formal training in AI and machine learning and have devoted little independent study to the field, so starting simple is prudent.

Part One - Data sourcing and cleaning

After deciding what I wanted to investigate, I needed to figure out a way to obtain the necessary data. My initial idea was to just surf the internet for random repositories of checkmate positions in FEN files, which could then be used to generate images. However I quickly discovered two problems.

  1. It is somewhat difficult to find repositories of FEN files that are of an actual checkmate (most are for checkmate in one move chess puzzles, more on this later).
  2. After locating a repository there were far too few actual positions! Image classification needs many many images for training.

My next idea was to obtain a large dataset of chess games, identify which games ended in a checkmate position, and extract those positions to generate images. Enter lichess. Lichess is a completely free and open source chess website, and I highly reccomend it if you are interested in playing online chess. Lichess conveniently archives every single game played on their site by month, with games as far back as 2013. To reduce computing time, I chose the month with fewest games, January 2013, during which a mere 121,332 games were played.

An aside, to give you an idea of how popular lichess has become over the years I scraped the number of standard (non-variant) games played per month.

*Games played vs. time on lichess*

Figure 2: Games played vs. time on lichess

The huge spike at the end is due to the ongoing covid-19 global pandemic.

Moving on, the games are bundled into a single .bz2 file, and are all in PGN format. After I downloaded the January 2013 file I needed a way to parse through the games and extract desired features. I decided to use the python library python-chess as it was reccomended on the lichess game archive.

I have fairly minimal experience working with python, so I am sure my approach is by no means the best or most efficient.

#load library
import chess.pgn 

#store downloaded pgn file
pgn = open("lichess_db_standard_rated_2013-01.pgn", encoding="utf-8-sig")

We can look at all the categories in a game from the pgn file.

chess.pgn.read_headers(pgn)
## Headers(Event='Rated Classical game', Site='https://lichess.org/j1dkb5dw', White='BFG9k', Black='mamalak', Result='1-0', BlackElo='1403', BlackRatingDiff='-8', ECO='C00', Opening='French Defense: Normal Variation', Termination='Normal', TimeControl='600+8', UTCDate='2012.12.31', UTCTime='23:01:03', WhiteElo='1639', WhiteRatingDiff='+5')

The goal is to extract positions with checkmate on the board. Lichess tags games that end in resignation or checkmate as “Normal” termination in pgn files. The initial parse will identify the locations of all these games across the main .pgn file.

#Create empty array to store games that ended in checkmate
checkmate_games = []

#1st parsing: get game locations that end normally
while True:
    offset = pgn.tell() #stores current file position
    headers = chess.pgn.read_headers(pgn) #stores current header
    
    if headers is None: #if no value go to next file location
        break
    if "Normal" in headers.get("Termination"):
        checkmate_games.append(offset)

Now that we have the file locations of games that ended normally, we need to store the actual games.

#2nd parsing: get games that ended normally
checkmate_games2 = []

for i in checkmate_games:
    pgn.seek(i)
    checkmate_games2.append(chess.pgn.read_game(pgn))

Now that we have stored all the actual games that ended normally, we need to find the subset of games that ended in checkmate.

#3rd parsing: get games that end in checkmate
checkmate_games3 = []

for i in range(len(checkmate_games2)):
    if(checkmate_games2[i].end().board().is_checkmate()):
        checkmate_games3.append(checkmate_games2[i])
        
len(checkmate_games3)
34474

Out of the initial 121,332 games in the file only 34,474 ended in checkmate.

The NN that will be trained still needs inputs, the next step is to get the checkmate positions in all 34,474 games and store these positions as images. Fortunately the python-chess library has a module that suits our needs, and will create an svg image of a specified chess position.

#Write an svg image of every checkmate position to disk
for i in range(len(checkmate_games3)):
    outfilename = '%smate.svg' % (str(i)) #create unique file name, e.g. 0mate.svg, 1mate.svg, ...
    outputFile = open(outfilename, mode = "w")
    outputFile.write(chess.svg.board(board = checkmate_games3[i].end().board(), coordinates=False))
    outputFile.close()

The .end() method will grab the position at the end of a game, .board() will generate a board representation for chess.svg.board() to render.

Now all the checkmate positions are saved as images as desired. However in order to train the NN, we will need more than just checkmate positions. To get chess positions that are not checkmate positions I looped through the stored games that ended normally, and extracted a random position from the game (that wasn’t checkmate), then saved that position again as a svg file.

import random as ran

for j in range(1,73552):

    board1 = checkmate_games2[j].board() #store jth game as a board
    moves = []
    
    
    for move in checkmate_games2[j].mainline_moves(): #append move list
        moves.append(move)
    
    if(len(moves) - 2 <= 1): #filter games that end after 2 moves or less
        continue
    
    r1 = ran.randint(1,len(moves)-2) #random total number of moves that won't result in checkmate.
    
    for i in range(r1): #play moves on current board
        board1.push(moves[i])
    
    outfilename = '%sNmate.svg' % (str(j))
    outputFile = open(outfilename, mode = "w")
    outputFile.write(chess.svg.board(board1, coordinates=False))
    outputFile.close()

Now there are 73,551 chess positions saved as svg files that are not checkmate. I chose this number somewhat randomly, it is a little over twice the number of checkmate images. I wanted a balance between computation time and sufficient data, as for training and testing I wanted to have more checkmate than non-checkmate positions.

However I was still not done, as svg files are not exactly friendly to out of box NN modules. To convert my chess positions to an image file format easier to work with I used ImageMagick. This section is already somewhat lengthy so I will skip over the details, but there exist many informative tutorials on the web about using ImageMagick for batch processing.


Part 3 - NN fitting coming soon!