Revisiting Frank-Wolfe for Polytopes: Strict Complementarity and
  Sparsity
  In recent years it was proved that simple modifications of the classical
Frank-Wolfe algorithm (aka conditional gradient algorithm) for smooth convex
minimization over convex and compact polytopes, converge with linear rate,
assuming the objective function has the quadratic growth property. However, the
rate of these methods depends explicitly on the dimension of the problem which
cannot explain their empirical success for large scale problems. In this paper
we first demonstrate that already for very simple problems and even when the
optimal solution lies on a low-dimensional face of the polytope, such
dependence on the dimension cannot be avoided in worst case. We then revisit
the addition of a strict complementarity assumption already considered in
Wolfe's classical book \cite{Wolfe1970}, and prove that under this condition,
the Frank-Wolfe method with away-steps and line-search converges linearly with
rate that depends explicitly only on the dimension of the optimal face. We
motivate strict complementarity by proving that it implies sparsity-robustness
of optimal solutions to noise.
