Online Learning of the Kalman Filter with Logarithmic Regret
  In this paper, we consider the problem of predicting observations generated
online by an unknown, partially observed linear system, which is driven by
stochastic noise. For such systems the optimal predictor in the mean square
sense is the celebrated Kalman filter, which can be explicitly computed when
the system model is known. When the system model is unknown, we have to learn
how to predict observations online based on finite data, suffering possibly a
non-zero regret with respect to the Kalman filter's prediction. We show that it
is possible to achieve a regret of the order of $\mathrm{poly}\log(N)$ with
high probability, where $N$ is the number of observations collected. Our work
is the first to provide logarithmic regret guarantees for the widely used
Kalman filter. This is achieved using an online least-squares algorithm, which
exploits the approximately linear relation between future observations and past
observations. The regret analysis is based on the stability properties of the
Kalman filter, recent statistical tools for finite sample analysis of system
identification, and classical results for the analysis of least-squares
algorithms for time series. Our regret analysis can also be applied for state
prediction of the hidden state, in the case of unknown noise statistics but
known state-space basis. A fundamental technical contribution is that our
bounds hold even for the class of non-explosive systems, which includes the
class of marginally stable systems, which was an open problem for the case of
online prediction under stochastic noise.
