Learning Affinity via Spatial Propagation Networks
  In this paper, we propose spatial propagation networks for learning the
affinity matrix for vision tasks. We show that by constructing a row/column
linear propagation model, the spatially varying transformation matrix exactly
constitutes an affinity matrix that models dense, global pairwise relationships
of an image. Specifically, we develop a three-way connection for the linear
propagation model, which (a) formulates a sparse transformation matrix, where
all elements can be the output from a deep CNN, but (b) results in a dense
affinity matrix that effectively models any task-specific pairwise similarity
matrix. Instead of designing the similarity kernels according to image features
of two points, we can directly output all the similarities in a purely
data-driven manner. The spatial propagation network is a generic framework that
can be applied to many affinity-related tasks, including but not limited to
image matting, segmentation and colorization, to name a few. Essentially, the
model can learn semantically-aware affinity values for high-level vision tasks
due to the powerful learning capability of the deep neural network classifier.
We validate the framework on the task of refinement for image segmentation
boundaries. Experiments on the HELEN face parsing and PASCAL VOC-2012 semantic
segmentation tasks show that the spatial propagation network provides a
general, effective and efficient solution for generating high-quality
segmentation results.
