Algorithmic Analysis and Statistical Estimation of SLOPE via Approximate
  Message Passing
  SLOPE is a relatively new convex optimization procedure for high-dimensional
linear regression via the sorted l1 penalty: the larger the rank of the fitted
coefficient, the larger the penalty. This non-separable penalty renders many
existing techniques invalid or inconclusive in analyzing the SLOPE solution. In
this paper, we develop an asymptotically exact characterization of the SLOPE
solution under Gaussian random designs through solving the SLOPE problem using
approximate message passing (AMP). This algorithmic approach allows us to
approximate the SLOPE solution via the much more amenable AMP iterates.
Explicitly, we characterize the asymptotic dynamics of the AMP iterates relying
on a recently developed state evolution analysis for non-separable penalties,
thereby overcoming the difficulty caused by the sorted l1 penalty. Moreover, we
prove that the AMP iterates converge to the SLOPE solution in an asymptotic
sense, and numerical simulations show that the convergence is surprisingly
fast. Our proof rests on a novel technique that specifically leverages the
SLOPE problem. In contrast to prior literature, our work not only yields an
asymptotically sharp analysis but also offers an algorithmic, flexible, and
constructive approach to understanding the SLOPE problem.
