Node-Level Differentially Private Graph Neural Networks
  Graph Neural Networks (GNNs) are a popular technique for modelling
graph-structured data that compute node-level representations via aggregation
of information from the local neighborhood of each node. However, this
aggregation implies increased risk of revealing sensitive information, as a
node can participate in the inference for multiple nodes. This implies that
standard privacy preserving machine learning techniques, such as differentially
private stochastic gradient descent (DP-SGD) - which are designed for
situations where each data point participates in the inference for one point
only - either do not apply, or lead to inaccurate solutions. In this work, we
formally define the problem of learning 1-layer GNNs with node-level privacy,
and provide an algorithmic solution with a strong differential privacy
guarantee. Even though each node can be involved in the inference for multiple
nodes, by employing a careful sensitivity analysis anda non-trivial extension
of the privacy-by-amplification technique, our method is able to provide
accurate solutions with solid privacy parameters. Empirical evaluation on
standard benchmarks demonstrates that our method is indeed able to learn
accurate privacy preserving GNNs, while still outperforming standard
non-private methods that completely ignore graph information.
