ARMA Nets: Expanding Receptive Field for Dense Prediction
  Global information is essential for dense prediction problems, whose goal is
to compute a discrete or continuous label for each pixel in the images.
Traditional convolutional layers in neural networks, initially designed for
image classification, are restrictive in these problems since the filter size
limits their receptive fields. In this work, we propose to replace any
traditional convolutional layer with an autoregressive moving-average (ARMA)
layer, a novel module with an adjustable receptive field controlled by the
learnable autoregressive coefficients. Compared with traditional convolutional
layers, our ARMA layer enables explicit interconnections of the output neurons
and learns its receptive field by adapting the autoregressive coefficients of
the interconnections. ARMA layer is adjustable to different types of tasks: for
tasks where global information is crucial, it is capable of learning relatively
large autoregressive coefficients to allow for an output neuron's receptive
field covering the entire input; for tasks where only local information is
required, it can learn small or near zero autoregressive coefficients and
automatically reduces to a traditional convolutional layer. We show both
theoretically and empirically that the effective receptive field of networks
with ARMA layers (named as ARMA networks) expands with larger autoregressive
coefficients. We also provably solve the instability problem of learning and
prediction in the ARMA layer through a re-parameterization mechanism.
Additionally, we demonstrate that ARMA networks substantially improve their
baselines on challenging dense prediction tasks including video prediction and
semantic segmentation.
