Memory-Efficient Convex Optimization for Self-Dictionary Separable
  Nonnegative Matrix Factorization: A Frank-Wolfe Approach
  Nonnegative matrix factorization (NMF) often relies on the separability
condition for tractable algorithm design. Separability-based NMF is mainly
handled by two types of approaches, namely, greedy pursuit and convex
programming. A notable convex NMF formulation is the so-called self-dictionary
multiple measurement vectors (SD-MMV), which can work without knowing the
matrix rank a priori, and is arguably more resilient to error propagation
relative to greedy pursuit. However, convex SD-MMV renders a large memory cost
that scales quadratically with the problem size. This memory challenge has been
around for a decade, and a major obstacle for applying convex SD-MMV to big
data analytics. This work proposes a memory-efficient algorithm for convex
SD-MMV. Our algorithm capitalizes on the special update rules of a classic
algorithm from the 1950s, namely, the Frank-Wolfe (FW) algorithm. It is shown
that, under reasonable conditions, the FW algorithm solves the noisy SD-MMV
problem with a memory cost that grows linearly with the amount of data. To
handle noisier scenarios, a smoothed group sparsity regularizer is proposed to
improve robustness while maintaining the low memory footprint with guarantees.
The proposed approach presents the first linear memory complexity algorithmic
framework for convex SD-MMV based NMF. The method is tested over a couple of
unsupervised learning tasks, i.e., text mining and community detection, to
showcase its effectiveness and memory efficiency.
