An eigenanalysis of data centering in machine learning
  Many pattern recognition methods rely on statistical information from
centered data, with the eigenanalysis of an empirical central moment, such as
the covariance matrix in principal component analysis (PCA), as well as partial
least squares regression, canonical-correlation analysis and Fisher
discriminant analysis. Recently, many researchers advocate working on
non-centered data. This is the case for instance with the singular value
decomposition approach, with the (kernel) entropy component analysis, with the
information-theoretic learning framework, and even with nonnegative matrix
factorization. Moreover, one can also consider a non-centered PCA by using the
second-order non-central moment.
  The main purpose of this paper is to bridge the gap between these two
viewpoints in designing machine learning methods. To provide a study at the
cornerstone of kernel-based machines, we conduct an eigenanalysis of the inner
product matrices from centered and non-centered data. We derive several results
connecting their eigenvalues and their eigenvectors. Furthermore, we explore
the outer product matrices, by providing several results connecting the largest
eigenvectors of the covariance matrix and its non-centered counterpart. These
results lay the groundwork to several extensions beyond conventional centering,
with the weighted mean shift, the rank-one update, and the multidimensional
scaling. Experiments conducted on simulated and real data illustrate the
relevance of this work.
