Probabilistic Dual Network Architecture Search on Graphs
  We present the first differentiable Network Architecture Search (NAS) for
Graph Neural Networks (GNNs). GNNs show promising performance on a wide range
of tasks, but require a large amount of architecture engineering. First, graphs
are inherently a non-Euclidean and sophisticated data structure, leading to
poor adaptivity of GNN architectures across different datasets. Second, a
typical graph block contains numerous different components, such as aggregation
and attention, generating a large combinatorial search space. To counter these
problems, we propose a Probabilistic Dual Network Architecture Search (PDNAS)
framework for GNNs. PDNAS not only optimises the operations within a single
graph block (micro-architecture), but also considers how these blocks should be
connected to each other (macro-architecture). The dual architecture (micro- and
marco-architectures) optimisation allows PDNAS to find deeper GNNs on diverse
datasets with better performance compared to other graph NAS methods. Moreover,
we use a fully gradient-based search approach to update architectural
parameters, making it the first differentiable graph NAS method. PDNAS
outperforms existing hand-designed GNNs and NAS results, for example, on the
PPI dataset, PDNAS beats its best competitors by 1.67 and 0.17 in F1 scores.
