First and Second Order Methods for Online Convolutional Dictionary
  Learning
  Convolutional sparse representations are a form of sparse representation with
a structured, translation invariant dictionary. Most convolutional dictionary
learning algorithms to date operate in batch mode, requiring simultaneous
access to all training images during the learning process, which results in
very high memory usage and severely limits the training data that can be used.
Very recently, however, a number of authors have considered the design of
online convolutional dictionary learning algorithms that offer far better
scaling of memory and computational cost with training set size than batch
methods. This paper extends our prior work, improving a number of aspects of
our previous algorithm; proposing an entirely new one, with better performance,
and that supports the inclusion of a spatial mask for learning from incomplete
data; and providing a rigorous theoretical analysis of these methods.
