TDLeaf(lambda): Combining Temporal Difference Learning with Game-Tree
  Search
  In this paper we present TDLeaf(lambda), a variation on the TD(lambda)
algorithm that enables it to be used in conjunction with minimax search. We
present some experiments in both chess and backgammon which demonstrate its
utility and provide comparisons with TD(lambda) and another less radical
variant, TD-directed(lambda). In particular, our chess program, ``KnightCap,''
used TDLeaf(lambda) to learn its evaluation function while playing on the Free
Internet Chess Server (FICS, fics.onenet.net). It improved from a 1650 rating
to a 2100 rating in just 308 games. We discuss some of the reasons for this
success and the relationship between our results and Tesauro's results in
backgammon.
