Soft Autoencoder and Its Wavelet Adaptation Interpretation
  Recently, deep learning becomes the main focus of machine learning research
and has greatly impacted many important fields. However, deep learning is
criticized for lack of interpretability. As a successful unsupervised model in
deep learning, the autoencoder embraces a wide spectrum of applications, yet it
suffers from the model opaqueness as well. In this paper, we propose a new type
of convolutional autoencoders, termed as Soft Autoencoder (Soft-AE), in which
the activation functions of encoding layers are implemented with adaptable
soft-thresholding units while decoding layers are realized with linear units.
Consequently, Soft-AE can be naturally interpreted as a learned cascaded
wavelet shrinkage system. Our denoising experiments demonstrate that Soft-AE
not only is interpretable but also offers a competitive performance relative to
its counterparts. Furthermore, we propose a generalized linear unit (GenLU) to
make an autoencoder more adaptive in nonlinearly filtering images and data,
such as denoising and deblurring.
