Spectral M-estimation with Applications to Hidden Markov Models
  Method of moment estimators exhibit appealing statistical properties, such as
asymptotic unbiasedness, for nonconvex problems. However, they typically
require a large number of samples and are extremely sensitive to model
misspecification. In this paper, we apply the framework of M-estimation to
develop both a generalized method of moments procedure and a principled method
for regularization. Our proposed M-estimator obtains optimal sample efficiency
rates (in the class of moment-based estimators) and the same well-known rates
on prediction accuracy as other spectral estimators. It also makes it
straightforward to incorporate regularization into the sample moment
conditions. We demonstrate empirically the gains in sample efficiency from our
approach on hidden Markov models.
