An Inverse Power Method for Nonlinear Eigenproblems with Applications in
  1-Spectral Clustering and Sparse PCA
  Many problems in machine learning and statistics can be formulated as
(generalized) eigenproblems. In terms of the associated optimization problem,
computing linear eigenvectors amounts to finding critical points of a quadratic
function subject to quadratic constraints. In this paper we show that a certain
class of constrained optimization problems with nonquadratic objective and
constraints can be understood as nonlinear eigenproblems. We derive a
generalization of the inverse power method which is guaranteed to converge to a
nonlinear eigenvector. We apply the inverse power method to 1-spectral
clustering and sparse PCA which can naturally be formulated as nonlinear
eigenproblems. In both applications we achieve state-of-the-art results in
terms of solution quality and runtime. Moving beyond the standard eigenproblem
should be useful also in many other applications and our inverse power method
can be easily adapted to new problems.
