Efficient Learning of Generalized Linear and Single Index Models with
  Isotonic Regression
  Generalized Linear Models (GLMs) and Single Index Models (SIMs) provide
powerful generalizations of linear regression, where the target variable is
assumed to be a (possibly unknown) 1-dimensional function of a linear
predictor. In general, these problems entail non-convex estimation procedures,
and, in practice, iterative local search heuristics are often used. Kalai and
Sastry (2009) recently provided the first provably efficient method for
learning SIMs and GLMs, under the assumptions that the data are in fact
generated under a GLM and under certain monotonicity and Lipschitz constraints.
However, to obtain provable performance, the method requires a fresh sample
every iteration. In this paper, we provide algorithms for learning GLMs and
SIMs, which are both computationally and statistically efficient. We also
provide an empirical study, demonstrating their feasibility in practice.
