Max-Affine Regression: Provable, Tractable, and Near-Optimal Statistical
  Estimation
  Max-affine regression refers to a model where the unknown regression function
is modeled as a maximum of $k$ unknown affine functions for a fixed $k \geq 1$.
This generalizes linear regression and (real) phase retrieval, and is closely
related to convex regression. Working within a non-asymptotic framework, we
study this problem in the high-dimensional setting assuming that $k$ is a fixed
constant, and focus on estimation of the unknown coefficients of the affine
functions underlying the model. We analyze a natural alternating minimization
(AM) algorithm for the non-convex least squares objective when the design is
random. We show that the AM algorithm, when initialized suitably, converges
with high probability and at a geometric rate to a small ball around the
optimal coefficients. In order to initialize the algorithm, we propose and
analyze a combination of a spectral method and a random search scheme in a
low-dimensional space, which may be of independent interest. The final rate
that we obtain is near-parametric and minimax optimal (up to a poly-logarithmic
factor) as a function of the dimension, sample size, and noise variance. In
that sense, our approach should be viewed as a direct and implementable method
of enforcing regularization to alleviate the curse of dimensionality in
problems of the convex regression type. As a by-product of our analysis, we
also obtain guarantees on a classical algorithm for the phase retrieval problem
under considerably weaker assumptions on the design distribution than was
previously known. Numerical experiments illustrate the sharpness of our bounds
in the various problem parameters.
