On the Role of Channel Capacity in Learning Gaussian Mixture Models
  This paper studies the sample complexity of learning the $k$ unknown centers
of a balanced Gaussian mixture model (GMM) in $\mathbb{R}^d$ with spherical
covariance matrix $\sigma^2\mathbf{I}$. In particular, we are interested in the
following question: what is the maximal noise level $\sigma^2$, for which the
sample complexity is essentially the same as when estimating the centers from
labeled measurements? To that end, we restrict attention to a Bayesian
formulation of the problem, where the centers are uniformly distributed on the
sphere $\sqrt{d}\mathcal{S}^{d-1}$. Our main results characterize the exact
noise threshold $\sigma^2$ below which the GMM learning problem, in the large
system limit $d,k\to\infty$, is as easy as learning from labeled observations,
and above which it is substantially harder. The threshold occurs at $\frac{\log
k}{d} = \frac12\log\left( 1+\frac{1}{\sigma^2} \right)$, which is the capacity
of the additive white Gaussian noise (AWGN) channel. Thinking of the set of $k$
centers as a code, this noise threshold can be interpreted as the largest noise
level for which the error probability of the code over the AWGN channel is
small. Previous works on the GMM learning problem have identified the minimum
distance between the centers as a key parameter in determining the statistical
difficulty of learning the corresponding GMM. While our results are only proved
for GMMs whose centers are uniformly distributed over the sphere, they hint
that perhaps it is the decoding error probability associated with the center
constellation as a channel code that determines the statistical difficulty of
learning the corresponding GMM, rather than just the minimum distance.
