Training Deep Normalizing Flow Models in Highly Incomplete Data
  Scenarios with Prior Regularization
  Deep generative frameworks including GANs and normalizing flow models have
proven successful at filling in missing values in partially observed data
samples by effectively learning -- either explicitly or implicitly -- complex,
high-dimensional statistical distributions. In tasks where the data available
for learning is only partially observed, however, their performance decays
monotonically as a function of the data missingness rate. In high missing data
rate regimes (e.g., 60% and above), it has been observed that state-of-the-art
models tend to break down and produce unrealistic and/or semantically
inaccurate data. We propose a novel framework to facilitate the learning of
data distributions in high paucity scenarios that is inspired by traditional
formulations of solutions to ill-posed problems. The proposed framework
naturally stems from posing the process of learning from incomplete data as a
joint optimization task of the parameters of the model being learned and the
missing data values. The method involves enforcing a prior regularization term
that seamlessly integrates with objectives used to train explicit and tractable
deep generative frameworks such as deep normalizing flow models. We demonstrate
via extensive experimental validation that the proposed framework outperforms
competing techniques, particularly as the rate of data paucity approaches
unity.
