A Comparison of Latent Semantic Analysis and Correspondence Analysis of
  Document-Term Matrices
  Latent semantic analysis (LSA) and correspondence analysis (CA) are two
techniques that use a singular value decomposition (SVD) for dimensionality
reduction. LSA has been extensively used to obtain low-dimensional and dense
vectors that capture relationships among documents and terms. In this article,
we present a theoretical analysis and comparison of the two techniques in the
context of document-term matrices. We show that CA has some attractive
properties as compared to LSA, for instance that effects of margins arising
from differing document-lengths and term-frequencies are effectively
eliminated, so that the CA solution is optimally suited to focus on
relationships among documents and terms. A unifying framework is proposed that
includes both CA and LSA as special cases. We empirically compare CA to various
LSA based methods on two tasks, a document classification task in English and
an authorship attribution task on historical Dutch texts, and find that CA
performs significantly better. We also apply CA to a long-standing question
regarding the authorship of the Dutch national anthem Wilhelmus and provide
further support that it can be attributed to the author Datheen, amongst
several contenders.
