Temporal Difference Learning Versus CoEvolution for Acquiring Othello Position Evaluation.pdfVIP

Temporal Difference Learning Versus Co-Evolution for Acquiring Othello Position Evaluation Simon M. Lucas Thomas P. Runarsson Department of Computer Science Science Institute University of Essex, Colchester, UK University of Iceland, Iceland sml@essex.ac.uk tpr@hi.is Abstract— This paper compares the use of temporal differ- The game playing strategies are encapsulated in the ence learning (TDL) versus co-evolutionary learning (CEL) for weights of a weighted piece counter (WPC). Each game acquiring position evaluation functions for the game of Othello. is played by using a 1-ply minimax search, with the WPC The paper provides important insights into the strengths and weaknesses of each approach. The main findings are that being used to estimate the value of the game-board after each for Othello, TDL learns much faster than CEL, but that possible move from the current board. properly tuned CEL can learn better playing strategies. For The paper is organised as follows. In section II a brief CEL, it is essential to use parent-child weighted averaging in description of the game Othello is given and some of the order to achieve good performance. Using this method a high more notable research on learning game strategies for Othello quality weighted piece counter was evolved, and was shown to significantly outperform a set of standard heuristic weights. listed. In section III the implementation of TDL and CEL is