Adam with Bandit Sampling for Deep Learning
  Adam is a widely used optimization method for training deep learning models.
It computes individual adaptive learning rates for different parameters. In
this paper, we propose a generalization of Adam, called Adambs, that allows us
to also adapt to different training examples based on their importance in the
model's convergence. To achieve this, we maintain a distribution over all
examples, selecting a mini-batch in each iteration by sampling according to
this distribution, which we update using a multi-armed bandit algorithm. This
ensures that examples that are more beneficial to the model training are
sampled with higher probabilities. We theoretically show that Adambs improves
the convergence rate of Adam---$O(\sqrt{\frac{\log n}{T} })$ instead of
$O(\sqrt{\frac{n}{T}})$ in some cases. Experiments on various models and
datasets demonstrate Adambs's fast convergence in practice.
