Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural
  Networks
  Augmenting transformation knowledge onto a convolutional neural network's
weights has often yielded significant improvements in performance. For
rotational transformation augmentation, an important element to recent
approaches has been the use of a steerable basis i.e. the circular harmonics.
Here, we propose a scale-steerable filter basis for the locally scale-invariant
CNN, denoted as log-radial harmonics. By replacing the kernels in the locally
scale-invariant CNN \cite{lsi_cnn} with scale-steered kernels, significant
improvements in performance can be observed on the MNIST-Scale and FMNIST-Scale
datasets. Training with a scale-steerable basis results in filters which show
meaningful structure, and feature maps demonstrate which demonstrate visibly
higher spatial-structure preservation of input. Furthermore, the proposed
scale-steerable CNN shows on-par generalization to global affine transformation
estimation methods such as Spatial Transformers, in response to test-time data
distortions.
