Applying Deep Belief Networks to Word Sense Disambiguation
  In this paper, we applied a novel learning algorithm, namely, Deep Belief
Networks (DBN) to word sense disambiguation (WSD). DBN is a probabilistic
generative model composed of multiple layers of hidden units. DBN uses
Restricted Boltzmann Machine (RBM) to greedily train layer by layer as a
pretraining. Then, a separate fine tuning step is employed to improve the
discriminative power. We compared DBN with various state-of-the-art supervised
learning algorithms in WSD such as Support Vector Machine (SVM), Maximum
Entropy model (MaxEnt), Naive Bayes classifier (NB) and Kernel Principal
Component Analysis (KPCA). We used all words in the given paragraph,
surrounding context words and part-of-speech of surrounding words as our
knowledge sources. We conducted our experiment on the SENSEVAL-2 data set. We
observed that DBN outperformed all other learning algorithms.
