Learning for Adaptive Real-time Search
  Real-time heuristic search is a popular model of acting and learning in
intelligent autonomous agents. Learning real-time search agents improve their
performance over time by acquiring and refining a value function guiding the
application of their actions. As computing the perfect value function is
typically intractable, a heuristic approximation is acquired instead. Most
studies of learning in real-time search (and reinforcement learning) assume
that a simple value-function-greedy policy is used to select actions. This is
in contrast to practice, where high-performance is usually attained by
interleaving planning and acting via a lookahead search of a non-trivial depth.
In this paper, we take a step toward bridging this gap and propose a novel
algorithm that (i) learns a heuristic function to be used specifically with a
lookahead-based policy, (ii) selects the lookahead depth adaptively in each
state, (iii) gives the user control over the trade-off between exploration and
exploitation. We extensively evaluate the algorithm in the sliding tile puzzle
testbed comparing it to the classical LRTA* and the more recent weighted LRTA*,
bounded LRTA*, and FALCONS. Improvements of 5 to 30 folds in convergence speed
are observed.
