Efficient Convolutional Auto-Encoding via Random Convexification and
  Frequency-Domain Minimization
  The omnipresence of deep learning architectures such as deep convolutional
neural networks (CNN)s is fueled by the synergistic combination of
ever-increasing labeled datasets and specialized hardware. Despite the
indisputable success, the reliance on huge amounts of labeled data and
specialized hardware can be a limiting factor when approaching new
applications. To help alleviating these limitations, we propose an efficient
learning strategy for layer-wise unsupervised training of deep CNNs on
conventional hardware in acceptable time. Our proposed strategy consists of
randomly convexifying the reconstruction contractive auto-encoding (RCAE)
learning objective and solving the resulting large-scale convex minimization
problem in the frequency domain via coordinate descent (CD). The main
advantages of our proposed learning strategy are: (1) single tunable
optimization parameter; (2) fast and guaranteed convergence; (3) possibilities
for full parallelization. Numerical experiments show that our proposed learning
strategy scales (in the worst case) linearly with image size, number of filters
and filter size.
