Improving the Accuracy of Principal Component Analysis by the Maximum
  Entropy Method
  Classical Principal Component Analysis (PCA) approximates data in terms of
projections on a small number of orthogonal vectors. There are simple
procedures to efficiently compute various functions of the data from the PCA
approximation. The most important function is arguably the Euclidean distance
between data items, This can be used, for example, to solve the approximate
nearest neighbor problem. We use random variables to model the inherent
uncertainty in such approximations, and apply the Maximum Entropy Method to
infer the underlying probability distribution. We propose using the expected
values of distances between these random variables as improved estimates of the
distance. We show by analysis and experimentally that in most cases results
obtained by our method are more accurate than what is obtained by the classical
approach. This improves the accuracy of a classical technique that have been
used with little change for over 100 years.
