FAST-PCA: A Fast and Exact Algorithm for Distributed Principal Component
  Analysis
  Principal Component Analysis (PCA) is a fundamental data preprocessing tool
in the world of machine learning. While PCA is often thought of as a
dimensionality reduction method, the purpose of PCA is actually two-fold:
dimension reduction and uncorrelated feature learning. Furthermore, the
enormity of the dimensions and sample size in the modern day datasets have
rendered the centralized PCA solutions unusable. In that vein, this paper
reconsiders the problem of PCA when data samples are distributed across nodes
in an arbitrarily connected network. While a few solutions for distributed PCA
exist, those either overlook the uncorrelated feature learning aspect of the
PCA, tend to have high communication overhead that makes them inefficient
and/or lack `exact' or `global' convergence guarantees. To overcome these
aforementioned issues, this paper proposes a distributed PCA algorithm termed
FAST-PCA (Fast and exAct diSTributed PCA). The proposed algorithm is efficient
in terms of communication and is proven to converge linearly and exactly to the
principal components, leading to dimension reduction as well as uncorrelated
features. The claims are further supported by experimental results.
