Efficient Greedy Coordinate Descent for Composite Problems
  Coordinate descent with random coordinate selection is the current state of
the art for many large scale optimization problems. However, greedy selection
of the steepest coordinate on smooth problems can yield convergence rates
independent of the dimension $n$, and requiring upto $n$ times fewer
iterations.
  In this paper, we consider greedy updates that are based on subgradients for
a class of non-smooth composite problems, which includes $L1$-regularized
problems, SVMs and related applications. For these problems we provide (i) the
first linear rates of convergence independent of $n$, and show that our greedy
update rule provides speedups similar to those obtained in the smooth case.
This was previously conjectured to be true for a stronger greedy coordinate
selection strategy.
  Furthermore, we show that (ii) our new selection rule can be mapped to
instances of maximum inner product search, allowing to leverage standard
nearest neighbor algorithms to speed up the implementation. We demonstrate the
validity of the approach through extensive numerical experiments.
