Faithful Inversion of Generative Models for Effective Amortized
  Inference
  Inference amortization methods share information across multiple
posterior-inference problems, allowing each to be carried out more efficiently.
Generally, they require the inversion of the dependency structure in the
generative model, as the modeller must learn a mapping from observations to
distributions approximating the posterior. Previous approaches have involved
inverting the dependency structure in a heuristic way that fails to capture
these dependencies correctly, thereby limiting the achievable accuracy of the
resulting approximations. We introduce an algorithm for faithfully, and
minimally, inverting the graphical model structure of any generative model.
Such inverses have two crucial properties: (a) they do not encode any
independence assertions that are absent from the model and; (b) they are local
maxima for the number of true independencies encoded. We prove the correctness
of our approach and empirically show that the resulting minimally faithful
inverses lead to better inference amortization than existing heuristic
approaches.
