A Generic online acceleration scheme for Optimization algorithms via   Relaxation and Inertia
  We propose generic acceleration schemes for a wide class of optimization and iterative schemes based on relaxation and inertia. In particular, we introduce methods that automatically tunes the acceleration coefficients online, and establish their convergence. This is made possible by considering the class of fixed-points iterations over averaged operators which encompass gradient methods, ADMM, primal dual algorithms, an so on. 
Franck Iutzeler and Julien M. Hendrickx
math.OC