Neural Graph Machines: Learning Neural Networks Using Graphs
  Label propagation is a powerful and flexible semi-supervised learning
technique on graphs. Neural networks, on the other hand, have proven track
records in many supervised learning tasks. In this work, we propose a training
framework with a graph-regularised objective, namely "Neural Graph Machines",
that can combine the power of neural networks and label propagation. This work
generalises previous literature on graph-augmented training of neural networks,
enabling it to be applied to multiple neural architectures (Feed-forward NNs,
CNNs and LSTM RNNs) and a wide range of graphs. The new objective allows the
neural networks to harness both labeled and unlabeled data by: (a) allowing the
network to train using labeled data as in the supervised setting, (b) biasing
the network to learn similar hidden representations for neighboring nodes on a
graph, in the same vein as label propagation. Such architectures with the
proposed objective can be trained efficiently using stochastic gradient descent
and scaled to large graphs, with a runtime that is linear in the number of
edges. The proposed joint training approach convincingly outperforms many
existing methods on a wide range of tasks (multi-label classification on social
graphs, news categorization, document classification and semantic intent
classification), with multiple forms of graph inputs (including graphs with and
without node-level features) and using different types of neural networks.
