Model Function Based Conditional Gradient Method with Armijo-like Line
  Search
  The Conditional Gradient Method is generalized to a class of non-smooth
non-convex optimization problems with many applications in machine learning.
The proposed algorithm iterates by minimizing so-called model functions over
the constraint set. Complemented with an Amijo line search procedure, we prove
that subsequences converge to a stationary point. The abstract framework of
model functions provides great flexibility for the design of concrete
algorithms. As special cases, for example, we develop an algorithm for additive
composite problems and an algorithm for non-linear composite problems which
leads to a Gauss--Newton-type algorithm. Both instances are novel in non-smooth
non-convex optimization and come with numerous applications in machine
learning. Moreover, we obtain a hybrid version of Conditional Gradient and
Proximal Minimization schemes for free, which combines advantages of both. Our
algorithm is shown to perform favorably on a sparse non-linear robust
regression problem and we discuss the flexibility of the proposed framework in
several matrix factorization formulations.
