Scalable Convolutional Dictionary Learning with Constrained Recurrent
  Sparse Auto-encoders
  Given a convolutional dictionary underlying a set of observed signals, can a
carefully designed auto-encoder recover the dictionary in the presence of
noise? We introduce an auto-encoder architecture, termed constrained recurrent
sparse auto-encoder (CRsAE), that answers this question in the affirmative.
Given an input signal and an approximate dictionary, the encoder finds a sparse
approximation using FISTA. The decoder reconstructs the signal by applying the
dictionary to the output of the encoder. The encoder and decoder in CRsAE
parallel the sparse-coding and dictionary update steps in optimization-based
alternating-minimization schemes for dictionary learning. As such, the
parameters of the encoder and decoder are not independent, a constraint which
we enforce for the first time. We derive the back-propagation algorithm for
CRsAE. CRsAE is a framework for blind source separation that, only knowing the
number of sources (dictionary elements), and assuming sparsely-many can
overlap, is able to separate them. We demonstrate its utility in the context of
spike sorting, a source separation problem in computational neuroscience. We
demonstrate the ability of CRsAE to recover the underlying dictionary and
characterize its sensitivity as a function of SNR.
