EigenGame: PCA as a Nash Equilibrium
  We present a novel view on principal component analysis (PCA) as a
competitive game in which each approximate eigenvector is controlled by a
player whose goal is to maximize their own utility function. We analyze the
properties of this PCA game and the behavior of its gradient based updates. The
resulting algorithm -- which combines elements from Oja's rule with a
generalized Gram-Schmidt orthogonalization -- is naturally decentralized and
hence parallelizable through message passing. We demonstrate the scalability of
the algorithm with experiments on large image datasets and neural network
activations. We discuss how this new view of PCA as a differentiable game can
lead to further algorithmic developments and insights.
