Unsupervised Sequence Classification using Sequential Output Statistics
  We consider learning a sequence classifier without labeled data by using
sequential output statistics. The problem is highly valuable since obtaining
labels in training data is often costly, while the sequential output statistics
(e.g., language models) could be obtained independently of input data and thus
with low or no cost. To address the problem, we propose an unsupervised
learning cost function and study its properties. We show that, compared to
earlier works, it is less inclined to be stuck in trivial solutions and avoids
the need for a strong generative model. Although it is harder to optimize in
its functional form, a stochastic primal-dual gradient method is developed to
effectively solve the problem. Experiment results on real-world datasets
demonstrate that the new unsupervised learning method gives drastically lower
errors than other baseline methods. Specifically, it reaches test errors about
twice of those obtained by fully supervised learning.
