Majorization Minimization Technique for Optimally Solving Deep
  Dictionary Learning
  The concept of deep dictionary learning has been recently proposed. Unlike
shallow dictionary learning which learns single level of dictionary to
represent the data, it uses multiple layers of dictionaries. So far, the
problem could only be solved in a greedy fashion; this was achieved by learning
a single layer of dictionary in each stage where the coefficients from the
previous layer acted as inputs to the subsequent layer (only the first layer
used the training samples as inputs). This was not optimal; there was feedback
from shallower to deeper layers but not the other way. This work proposes an
optimal solution to deep dictionary learning whereby all the layers of
dictionaries are solved simultaneously. We employ the Majorization Minimization
approach. Experiments have been carried out on benchmark datasets; it shows
that optimal learning indeed improves over greedy piecemeal learning.
Comparison with other unsupervised deep learning tools (stacked denoising
autoencoder, deep belief network, contractive autoencoder and K-sparse
autoencoder) show that our method supersedes their performance both in accuracy
and speed.
