Towards improving discriminative reconstruction via simultaneous dense
  and sparse coding
  Discriminative features extracted from the sparse coding model have been
shown to perform well for classification and reconstruction. Recent deep
learning architectures have further improved reconstruction in inverse problems
by considering new dense priors learned from data. We propose a novel dense and
sparse coding model that integrates both representation capability and
discriminative features. The model considers the problem of recovering a dense
vector $\mathbf{x}$ and a sparse vector $\mathbf{u}$ given measurements of the
form $\mathbf{y} = \mathbf{A}\mathbf{x}+\mathbf{B}\mathbf{u}$. Our first
analysis proposes a natural geometric condition based on the minimal angle
between spanning subspaces corresponding to the measurement matrices
$\mathbf{A}$ and $\mathbf{B}$ to establish the uniqueness of solutions to the
linear system. The second analysis shows that, under mild assumptions, a convex
program recovers the dense and sparse components. We validate the effectiveness
of the proposed model on simulated data and propose a dense and sparse
autoencoder (DenSaE) tailored to learning the dictionaries from the dense and
sparse model. We demonstrate that a) DenSaE denoises natural images better than
architectures derived from the sparse coding model ($\mathbf{B}\mathbf{u}$), b)
in the presence of noise, training the biases in the latter amounts to
implicitly learning the $\mathbf{A}\mathbf{x} + \mathbf{B}\mathbf{u}$ model, c)
$\mathbf{A}$ and $\mathbf{B}$ capture low- and high-frequency contents,
respectively, and d) compared to the sparse coding model, DenSaE offers a
balance between discriminative power and representation.
