MURANA: A Generic Framework for Stochastic Variance-Reduced Optimization
  We propose a generic variance-reduced algorithm, which we call MUltiple
RANdomized Algorithm (MURANA), for minimizing a sum of several smooth functions
plus a regularizer, in a sequential or distributed manner. Our method is
formulated with general stochastic operators, which allow us to model various
strategies for reducing the computational complexity. For example, MURANA
supports sparse activation of the gradients, and also reduction of the
communication load via compression of the update vectors. This versatility
allows MURANA to cover many existing randomization mechanisms within a unified
framework. However, MURANA also encodes new methods as special cases. We
highlight one of them, which we call ELVIRA, and show that it improves upon
Loopless SVRG.
