Incremental Majorization-Minimization Optimization with Application to
  Large-Scale Machine Learning
  Majorization-minimization algorithms consist of successively minimizing a
sequence of upper bounds of the objective function. These upper bounds are
tight at the current estimate, and each iteration monotonically drives the
objective function downhill. Such a simple principle is widely applicable and
has been very popular in various scientific fields, especially in signal
processing and statistics. In this paper, we propose an incremental
majorization-minimization scheme for minimizing a large sum of continuous
functions, a problem of utmost importance in machine learning. We present
convergence guarantees for non-convex and convex optimization when the upper
bounds approximate the objective up to a smooth error; we call such upper
bounds "first-order surrogate functions". More precisely, we study asymptotic
stationary point guarantees for non-convex problems, and for convex ones, we
provide convergence rates for the expected objective function value. We apply
our scheme to composite optimization and obtain a new incremental proximal
gradient algorithm with linear convergence rate for strongly convex functions.
In our experiments, we show that our method is competitive with the state of
the art for solving machine learning problems such as logistic regression when
the number of training samples is large enough, and we demonstrate its
usefulness for sparse estimation with non-convex penalties.
