Deep Regression Bayesian Network and Its Applications
  Deep directed generative models have attracted much attention recently due to
their generative modeling nature and powerful data representation ability. In
this paper, we review different structures of deep directed generative models
and the learning and inference algorithms associated with the structures. We
focus on a specific structure that consists of layers of Bayesian Networks due
to the property of capturing inherent and rich dependencies among latent
variables. The major difficulty of learning and inference with deep directed
models with many latent variables is the intractable inference due to the
dependencies among the latent variables and the exponential number of latent
variable configurations. Current solutions use variational methods often
through an auxiliary network to approximate the posterior probability
inference. In contrast, inference can also be performed directly without using
any auxiliary network to maximally preserve the dependencies among the latent
variables. Specifically, by exploiting the sparse representation with the
latent space, max-max instead of max-sum operation can be used to overcome the
exponential number of latent configurations. Furthermore, the max-max operation
and augmented coordinate ascent are applied to both supervised and unsupervised
learning as well as to various inference. Quantitative evaluations on benchmark
datasets of different models are given for both data representation and feature
learning tasks.
