Variational Approximation Error in Bayesian Non-negative Matrix
  Factorization
  Non-negative matrix factorization (NMF) is a knowledge discovery method that
is used in many fields. Variational inference and Gibbs sampling methods for it
are also wellknown. However, the variational approximation error has not been
clarified yet, because NMF is not statistically regular and the prior
distribution used in variational Bayesian NMF (VBNMF) has zero or divergence
points. In this paper, using algebraic geometrical methods, we theoretically
analyze the difference in negative log evidence (a.k.a. free energy) between
VBNMF and Bayesian NMF, i.e., the Kullback-Leibler divergence between the
variational posterior and the true posterior. We derive an upper bound for the
learning coefficient (a.k.a. the real log canonical threshold) in Bayesian NMF.
By using the upper bound, we find a lower bound for the approximation error,
asymptotically. The result quantitatively shows how well the VBNMF algorithm
can approximate Bayesian NMF; the lower bound depends on the hyperparameters
and the true nonnegative rank. A numerical experiment demonstrates the
theoretical result.
