Efficient Euclidean Projections onto the Intersection of Norm Balls
  Using sparse-inducing norms to learn robust models has received increasing
attention from many fields for its attractive properties. Projection-based
methods have been widely applied to learning tasks constrained by such norms.
As a key building block of these methods, an efficient operator for Euclidean
projection onto the intersection of $\ell_1$ and $\ell_{1,q}$ norm balls
$(q=2\text{or}\infty)$ is proposed in this paper. We prove that the projection
can be reduced to finding the root of an auxiliary function which is piecewise
smooth and monotonic. Hence, a bisection algorithm is sufficient to solve the
problem. We show that the time complexity of our solution is $O(n+g\log g)$ for
$q=2$ and $O(n\log n)$ for $q=\infty$, where $n$ is the dimensionality of the
vector to be projected and $g$ is the number of disjoint groups; we confirm
this complexity by experimentation. Empirical study reveals that our method
achieves significantly better performance than classical methods in terms of
running time and memory usage. We further show that embedded with our efficient
projection operator, projection-based algorithms can solve regression problems
with composite norm constraints more efficiently than other methods and give
superior accuracy.
