Wide and Deep Graph Neural Networks with Distributed Online Learning
  Graph neural networks (GNNs) learn representations from network data with
naturally distributed architectures, rendering them well-suited candidates for
decentralized learning. Oftentimes, this decentralized graph support changes
with time due to link failures or topology variations. These changes create a
mismatch between the graphs on which GNNs were trained and the ones on which
they are tested. Online learning can be used to retrain GNNs at testing time,
overcoming this issue. However, most online algorithms are centralized and work
on convex problems (which GNNs rarely lead to). This paper proposes the Wide
and Deep GNN (WD-GNN), a novel architecture that can be easily updated with
distributed online learning mechanisms. The WD-GNN comprises two components:
the wide part is a bank of linear graph filters and the deep part is a GNN. At
training time, the joint architecture learns a nonlinear representation from
data. At testing time, the deep part (nonlinear) is left unchanged, while the
wide part is retrained online, leading to a convex problem. We derive
convergence guarantees for this online retraining procedure and further propose
a decentralized alternative. Experiments on the robot swarm control for
flocking corroborate theory and show potential of the proposed architecture for
distributed online learning.
