A Sober Look at Spectral Learning
  Spectral learning recently generated lots of excitement in machine learning,
largely because it is the first known method to produce consistent estimates
(under suitable conditions) for several latent variable models. In contrast,
maximum likelihood estimates may get trapped in local optima due to the
non-convex nature of the likelihood function of latent variable models. In this
paper, we do an empirical evaluation of spectral learning (SL) and expectation
maximization (EM), which reveals an important gap between the theory and the
practice. First, SL often leads to negative probabilities. Second, EM often
yields better estimates than spectral learning and it does not seem to get
stuck in local optima. We discuss how the rank of the model parameters and the
amount of training data can yield negative probabilities. We also question the
common belief that maximum likelihood estimators are necessarily inconsistent.
