Learning Hidden Markov Models from Pairwise Co-occurrences with
  Application to Topic Modeling
  We present a new algorithm for identifying the transition and emission
probabilities of a hidden Markov model (HMM) from the emitted data.
Expectation-maximization becomes computationally prohibitive for long
observation records, which are often required for identification. The new
algorithm is particularly suitable for cases where the available sample size is
large enough to accurately estimate second-order output probabilities, but not
higher-order ones. We show that if one is only able to obtain a reliable
estimate of the pairwise co-occurrence probabilities of the emissions, it is
still possible to uniquely identify the HMM if the emission probability is
\emph{sufficiently scattered}. We apply our method to hidden topic Markov
modeling, and demonstrate that we can learn topics with higher quality if
documents are modeled as observations of HMMs sharing the same emission (topic)
probability, compared to the simple but widely used bag-of-words model.
