Sample Complexity of Learning Mixtures of Sparse Linear Regressions
  In the problem of learning mixtures of linear regressions, the goal is to
learn a collection of signal vectors from a sequence of (possibly noisy) linear
measurements, where each measurement is evaluated on an unknown signal drawn
uniformly from this collection. This setting is quite expressive and has been
studied both in terms of practical applications and for the sake of
establishing theoretical guarantees. In this paper, we consider the case where
the signal vectors are sparse; this generalizes the popular compressed sensing
paradigm. We improve upon the state-of-the-art results as follows: In the noisy
case, we resolve an open question of Yin et al. (IEEE Transactions on
Information Theory, 2019) by showing how to handle collections of more than two
vectors and present the first robust reconstruction algorithm, i.e., if the
signals are not perfectly sparse, we still learn a good sparse approximation of
the signals. In the noiseless case, as well as in the noisy case, we show how
to circumvent the need for a restrictive assumption required in the previous
work. Our techniques are quite different from those in the previous work: for
the noiseless case, we rely on a property of sparse polynomials and for the
noisy case, we provide new connections to learning Gaussian mixtures and use
ideas from the theory of error-correcting codes.
