A Linearly Convergent Algorithm for Distributed Principal Component
  Analysis
  Principal Component Analysis (PCA) is the workhorse tool for dimensionality
reduction in this era of big data. While often overlooked, the purpose of PCA
is not only to reduce data dimensionality, but also to yield features that are
uncorrelated. Furthermore, the ever-increasing volume of data in the modern
world often requires storage of data samples across multiple machines, which
precludes the use of centralized PCA algorithms. This paper focuses on the dual
objective of PCA, namely, dimensionality reduction and decorrelation of
features, but in a distributed setting. This requires estimating the
eigenvectors of the data covariance matrix, as opposed to only estimating the
subspace spanned by the eigenvectors, when data is distributed across a network
of machines. Although a few distributed solutions to the PCA problem have been
proposed recently, convergence guarantees and/or communications overhead of
these solutions remain a concern. With an eye towards communications
efficiency, this paper introduces a feedforward neural network-based one
time-scale distributed PCA algorithm termed Distributed Sanger's Algorithm
(DSA) that estimates the eigenvectors of the data covariance matrix when data
is distributed across an undirected and arbitrarily connected network of
machines. Furthermore, the proposed algorithm is shown to converge linearly to
a neighborhood of the true solution. Numerical results are also provided to
demonstrate the efficacy of the proposed solution.
