Learning Feature Relevance Through Step Size Adaptation in
  Temporal-Difference Learning
  There is a long history of using meta learning as representation learning,
specifically for determining the relevance of inputs. In this paper, we examine
an instance of meta-learning in which feature relevance is learned by adapting
step size parameters of stochastic gradient descent---building on a variety of
prior work in stochastic approximation, machine learning, and artificial neural
networks. In particular, we focus on stochastic meta-descent introduced in the
Incremental Delta-Bar-Delta (IDBD) algorithm for setting individual step sizes
for each feature of a linear function approximator. Using IDBD, a feature with
large or small step sizes will have a large or small impact on generalization
from training examples. As a main contribution of this work, we extend IDBD to
temporal-difference (TD) learning---a form of learning which is effective in
sequential, non i.i.d. problems. We derive a variety of IDBD generalizations
for TD learning, demonstrating that they are able to distinguish which features
are relevant and which are not. We demonstrate that TD IDBD is effective at
learning feature relevance in both an idealized gridworld and a real-world
robotic prediction task.
