Using Time-Series Privileged Information for Provably Efficient Learning
  of Prediction Models
  We study prediction of future outcomes with supervised models that use
privileged information during learning. The privileged information comprises
samples of time series observed between the baseline time of prediction and the
future outcome; this information is only available at training time which
differs from the traditional supervised learning. Our question is when using
this privileged data leads to more sample-efficient learning of models that use
only baseline data for predictions at test time. We give an algorithm for this
setting and prove that when the time series are drawn from a non-stationary
Gaussian-linear dynamical system of fixed horizon, learning with privileged
information is more efficient than learning without it. On synthetic data, we
test the limits of our algorithm and theory, both when our assumptions hold and
when they are violated. On three diverse real-world datasets, we show that our
approach is generally preferable to classical learning, particularly when data
is scarce. Finally, we relate our estimator to a distillation approach both
theoretically and empirically.
