PACE: A Parallelizable Computation Encoder for Directed Acyclic Graphs
  Optimization of directed acyclic graph (DAG) structures has many
applications, such as neural architecture search (NAS) and probabilistic
graphical model learning. Encoding DAGs into real vectors is a dominant
component in most neural-network-based DAG optimization frameworks. Currently,
most DAG encoders use an asynchronous message passing scheme which sequentially
processes nodes according to the dependency between nodes in a DAG. That is, a
node must not be processed until all its predecessors are processed. As a
result, they are inherently not parallelizable. In this work, we propose a
Parallelizable Attention-based Computation structure Encoder (PACE) that
processes nodes simultaneously and encodes DAGs in parallel. We demonstrate the
superiority of PACE through encoder-dependent optimization subroutines that
search the optimal DAG structure based on the learned DAG embeddings.
Experiments show that PACE not only improves the effectiveness over previous
sequential DAG encoders with a significantly boosted training and inference
speed, but also generates smooth latent (DAG encoding) spaces that are
beneficial to downstream optimization subroutines. Our source code is available
at \url{https://github.com/zehaodong/PACE}
