Improved SVD-based Initialization for Nonnegative Matrix Factorization
  using Low-Rank Correction
  Due to the iterative nature of most nonnegative matrix factorization
(\textsc{NMF}) algorithms, initialization is a key aspect as it significantly
influences both the convergence and the final solution obtained. Many
initialization schemes have been proposed for NMF, among which one of the most
popular class of methods are based on the singular value decomposition (SVD).
However, these SVD-based initializations do not satisfy a rather natural
condition, namely that the error should decrease as the rank of factorization
increases. In this paper, we propose a novel SVD-based \textsc{NMF}
initialization to specifically address this shortcoming by taking into account
the SVD factors that were discarded to obtain a nonnegative initialization.
This method, referred to as nonnegative SVD with low-rank correction
(NNSVD-LRC), allows us to significantly reduce the initial error at a
negligible additional computational cost using the low-rank structure of the
discarded SVD factors. NNSVD-LRC has two other advantages compared to previous
SVD-based initializations: (1) it provably generates sparse initial factors,
and (2) it is faster as it only requires to compute a truncated SVD of rank
$\lceil r/2 + 1 \rceil$ where $r$ is the factorization rank of the sought NMF
decomposition (as opposed to a rank-$r$ truncated SVD for other methods). We
show on several standard dense and sparse data sets that our new method
competes favorably with state-of-the-art SVD-based initializations for NMF.
