Order Matters: Probabilistic Modeling of Node Sequence for Graph
  Generation
  A graph generative model defines a distribution over graphs. One type of
generative model is constructed by autoregressive neural networks, which
sequentially add nodes and edges to generate a graph. However, the likelihood
of a graph under the autoregressive model is intractable, as there are numerous
sequences leading to the given graph; this makes maximum likelihood estimation
challenging. Instead, in this work we derive the exact joint probability over
the graph and the node ordering of the sequential process. From the joint, we
approximately marginalize out the node orderings and compute a lower bound on
the log-likelihood using variational inference. We train graph generative
models by maximizing this bound, without using the ad-hoc node orderings of
previous methods. Our experiments show that the log-likelihood bound is
significantly tighter than the bound of previous schemes. Moreover, the models
fitted with the proposed algorithm can generate high-quality graphs that match
the structures of target graphs not seen during training. We have made our code
publicly available at
\hyperref[https://github.com/tufts-ml/graph-generation-vi]{https://github.com/tufts-ml/graph-generation-vi}.
