Affinity Graph Supervision for Visual Recognition
  Affinity graphs are widely used in deep architectures, including graph
convolutional neural networks and attention networks. Thus far, the literature
has focused on abstracting features from such graphs, while the learning of the
affinities themselves has been overlooked. Here we propose a principled method
to directly supervise the learning of weights in affinity graphs, to exploit
meaningful connections between entities in the data source. Applied to a visual
attention network, our affinity supervision improves relationship recovery
between objects, even without the use of manually annotated relationship
labels. We further show that affinity learning between objects boosts scene
categorization performance and that the supervision of affinity can also be
applied to graphs built from mini-batches, for neural network training. In an
image classification task we demonstrate consistent improvement over the
baseline, with diverse network architectures and datasets.
