Soft-Bayes: Prod for Mixtures of Experts with Log-Loss
  We consider prediction with expert advice under the log-loss with the goal of
deriving efficient and robust algorithms. We argue that existing algorithms
such as exponentiated gradient, online gradient descent and online Newton step
do not adequately satisfy both requirements. Our main contribution is an
analysis of the Prod algorithm that is robust to any data sequence and runs in
linear time relative to the number of experts in each round. Despite the
unbounded nature of the log-loss, we derive a bound that is independent of the
largest loss and of the largest gradient, and depends only on the number of
experts and the time horizon. Furthermore we give a Bayesian interpretation of
Prod and adapt the algorithm to derive a tracking regret.
