A Novel Genetic Algorithm with Hierarchical Evaluation Strategy for
  Hyperparameter Optimisation of Graph Neural Networks
  Graph representation of structured data can facilitate the extraction of
stereoscopic features, and it has demonstrated excellent ability when working
with deep learning systems, the so-called Graph Neural Networks (GNNs).
Choosing a promising architecture for constructing GNNs can be transferred to a
hyperparameter optimisation problem, a very challenging task due to the size of
the underlying search space and high computational cost for evaluating
candidate GNNs. To address this issue, this research presents a novel genetic
algorithm with a hierarchical evaluation strategy (HESGA), which combines the
full evaluation of GNNs with a fast evaluation approach. By using full
evaluation, a GNN is represented by a set of hyperparameter values and trained
on a specified dataset, and root mean square error (RMSE) will be used to
measure the quality of the GNN represented by the set of hyperparameter values
(for regression problems). While in the proposed fast evaluation process, the
training will be interrupted at an early stage, the difference of RMSE values
between the starting and interrupted epochs will be used as a fast score, which
implies the potential of the GNN being considered. To coordinate both types of
evaluations, the proposed hierarchical strategy uses the fast evaluation in a
lower level for recommending candidates to a higher level, where the full
evaluation will act as a final assessor to maintain a group of elite
individuals. To validate the effectiveness of HESGA, we apply it to optimise
two types of deep graph neural networks. The experimental results on three
benchmark datasets demonstrate its advantages compared to Bayesian
hyperparameter optimization.
