Memory Bounds for the Experts Problem
  Online learning with expert advice is a fundamental problem of sequential
prediction. In this problem, the algorithm has access to a set of $n$ "experts"
who make predictions on each day. The goal on each day is to process these
predictions, and make a prediction with the minimum cost. After making a
prediction, the algorithm sees the actual outcome on that day, updates its
state, and then moves on to the next day. An algorithm is judged by how well it
does compared to the best expert in the set.
  The classical algorithm for this problem is the multiplicative weights
algorithm. However, every application, to our knowledge, relies on storing
weights for every expert, and uses $\Omega(n)$ memory. There is little work on
understanding the memory required to solve the online learning with expert
advice problem, or run standard sequential prediction algorithms, in natural
streaming models, which is especially important when the number of experts, as
well as the number of days on which the experts make predictions, is large.
  We initiate the study of the learning with expert advice problem in the
streaming setting, and show lower and upper bounds. Our lower bound for i.i.d.,
random order, and adversarial order streams uses a reduction to a custom-built
problem using a novel masking technique, to show a smooth trade-off for regret
versus memory. Our upper bounds show novel ways to run standard sequential
prediction algorithms in rounds on small "pools" of experts, thus reducing the
necessary memory. For random-order streams, we show that our upper bound is
tight up to low order terms. We hope that these results and techniques will
have broad applications in online learning, and can inspire algorithms based on
standard sequential prediction techniques, like multiplicative weights, for a
wide range of other problems in the memory-constrained setting.
