Convolutional Analysis Operator Learning: Acceleration and Convergence
  Convolutional operator learning is gaining attention in many signal
processing and computer vision applications. Learning kernels has mostly relied
on so-called patch-domain approaches that extract and store many overlapping
patches across training signals. Due to memory demands, patch-domain methods
have limitations when learning kernels from large datasets -- particularly with
multi-layered structures, e.g., convolutional neural networks -- or when
applying the learned kernels to high-dimensional signal recovery problems. The
so-called convolution approach does not store many overlapping patches, and
thus overcomes the memory problems particularly with careful algorithmic
designs; it has been studied within the "synthesis" signal model, e.g.,
convolutional dictionary learning. This paper proposes a new convolutional
analysis operator learning (CAOL) framework that learns an analysis sparsifying
regularizer with the convolution perspective, and develops a new convergent
Block Proximal Extrapolated Gradient method using a Majorizer (BPEG-M) to solve
the corresponding block multi-nonconvex problems. To learn diverse filters
within the CAOL framework, this paper introduces an orthogonality constraint
that enforces a tight-frame filter condition, and a regularizer that promotes
diversity between filters. Numerical experiments show that, with sharp
majorizers, BPEG-M significantly accelerates the CAOL convergence rate compared
to the state-of-the-art block proximal gradient (BPG) method. Numerical
experiments for sparse-view computational tomography show that a convolutional
sparsifying regularizer learned via CAOL significantly improves reconstruction
quality compared to a conventional edge-preserving regularizer. Using more and
wider kernels in a learned regularizer better preserves edges in reconstructed
images.
