On Evaluation Metrics for Graph Generative Models
  In image generation, generative models can be evaluated naturally by visually
inspecting model outputs. However, this is not always the case for graph
generative models (GGMs), making their evaluation challenging. Currently, the
standard process for evaluating GGMs suffers from three critical limitations:
i) it does not produce a single score which makes model selection challenging,
ii) in many cases it fails to consider underlying edge and node features, and
iii) it is prohibitively slow to perform. In this work, we mitigate these
issues by searching for scalar, domain-agnostic, and scalable metrics for
evaluating and ranking GGMs. To this end, we study existing GGM metrics and
neural-network-based metrics emerging from generative models of images that use
embeddings extracted from a task-specific network. Motivated by the power of
certain Graph Neural Networks (GNNs) to extract meaningful graph
representations without any training, we introduce several metrics based on the
features extracted by an untrained random GNN. We design experiments to
thoroughly test metrics on their ability to measure the diversity and fidelity
of generated graphs, as well as their sample and computational efficiency.
Depending on the quantity of samples, we recommend one of two random-GNN-based
metrics that we show to be more expressive than pre-existing metrics. While we
focus on applying these metrics to GGM evaluation, in practice this enables the
ability to easily compute the dissimilarity between any two sets of graphs
regardless of domain. Our code is released at:
https://github.com/uoguelph-mlrg/GGM-metrics.
