Distributionally Robust and Multi-Objective Nonnegative Matrix
  Factorization
  Nonnegative matrix factorization (NMF) is a linear dimensionality reduction
technique for analyzing nonnegative data. A key aspect of NMF is the choice of
the objective function that depends on the noise model (or statistics of the
noise) assumed on the data. In many applications, the noise model is unknown
and difficult to estimate. In this paper, we define a multi-objective NMF
(MO-NMF) problem, where several objectives are combined within the same NMF
model. We propose to use Lagrange duality to judiciously optimize for a set of
weights to be used within the framework of the weighted-sum approach, that is,
we minimize a single objective function which is a weighted sum of the all
objective functions. We design a simple algorithm based on multiplicative
updates to minimize this weighted sum. We show how this can be used to find
distributionally robust NMF (DR-NMF) solutions, that is, solutions that
minimize the largest error among all objectives, using a dual approach solved
via a heuristic inspired from the Frank-Wolfe algorithm. We illustrate the
effectiveness of this approach on synthetic, document and audio data sets. The
results show that DR-NMF is robust to our incognizance of the noise model of
the NMF problem.
