Convolutional dictionary learning based auto-encoders for natural
  exponential-family distributions
  We introduce a class of auto-encoder neural networks tailored to data from
the natural exponential family (e.g., count data). The architectures are
inspired by the problem of learning the filters in a convolutional generative
model with sparsity constraints, often referred to as convolutional dictionary
learning (CDL). Our work is the first to combine ideas from convolutional
generative models and deep learning for data that are naturally modeled with a
non-Gaussian distribution (e.g., binomial and Poisson). This perspective
provides us with a scalable and flexible framework that can be re-purposed for
a wide range of tasks and assumptions on the generative model. Specifically,
the iterative optimization procedure for solving CDL, an unsupervised task, is
mapped to an unfolded and constrained neural network, with iterative
adjustments to the inputs to account for the generative distribution. We also
show that the framework can easily be extended for discriminative training,
appropriate for a supervised task. We demonstrate 1) that fitting the
generative model to learn, in an unsupervised fashion, the latent stimulus that
underlies neural spiking data leads to better goodness-of-fit compared to other
baselines, 2) competitive performance compared to state-of-the-art algorithms
for supervised Poisson image denoising, with significantly fewer parameters,
and 3) gradient dynamics of shallow binomial auto-encoder.
