A Markov Decision Process Approach to Active Meta Learning
  In supervised learning, we fit a single statistical model to a given data
set, assuming that the data is associated with a singular task, which yields
well-tuned models for specific use, but does not adapt well to new contexts. By
contrast, in meta-learning, the data is associated with numerous tasks, and we
seek a model that may perform well on all tasks simultaneously, in pursuit of
greater generalization. One challenge in meta-learning is how to exploit
relationships between tasks and classes, which is overlooked by commonly used
random or cyclic passes through data. In this work, we propose actively
selecting samples on which to train by discerning covariates inside and between
meta-training sets. Specifically, we cast the problem of selecting a sample
from a number of meta-training sets as either a multi-armed bandit or a Markov
Decision Process (MDP), depending on how one encapsulates correlation across
tasks. We develop scheduling schemes based on Upper Confidence Bound (UCB),
Gittins Index and tabular Markov Decision Problems (MDPs) solved with linear
programming, where the reward is the scaled statistical accuracy to ensure it
is a time-invariant function of state and action. Across a variety of
experimental contexts, we observe significant reductions in sample complexity
of active selection scheme relative to cyclic or i.i.d. sampling, demonstrating
the merit of exploiting covariates in practice.
