BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery
  A structural equation model (SEM) is an effective framework to reason over
causal relationships represented via a directed acyclic graph (DAG). Recent
advances have enabled effective maximum-likelihood point estimation of DAGs
from observational data. However, a point estimate may not accurately capture
the uncertainty in inferring the underlying graph in practical scenarios,
wherein the true DAG is non-identifiable and/or the observed dataset is
limited. We propose Bayesian Causal Discovery Nets (BCD Nets), a variational
inference framework for estimating a distribution over DAGs characterizing a
linear-Gaussian SEM. Developing a full Bayesian posterior over DAGs is
challenging due to the the discrete and combinatorial nature of graphs. We
analyse key design choices for scalable VI over DAGs, such as 1) the
parametrization of DAGs via an expressive variational family, 2) a continuous
relaxation that enables low-variance stochastic optimization, and 3) suitable
priors over the latent variables. We provide a series of experiments on real
and synthetic data showing that BCD Nets outperform maximum-likelihood methods
on standard causal discovery metrics such as structural Hamming distance in low
data regimes.
