SAG-VAE: End-to-end Joint Inference of Data Representations and Feature
  Relations
  Variational Autoencoders (VAEs) are powerful in data representation
inference, but it cannot learn relations between features with its vanilla form
and common variations. The ability to capture relations within data can provide
the much needed inductive bias necessary for building more robust Machine
Learning algorithms with more interpretable results. In this paper, inspired by
recent advances in relational learning using Graph Neural Networks, we propose
the Self-Attention Graph Variational AutoEncoder (SAG-VAE) network which can
simultaneously learn feature relations and data representations in an
end-to-end manner. SAG-VAE is trained by jointly inferring the posterior
distribution of two types of latent variables, which denote the data
representation and a shared graph structure, respectively. Furthermore, we
introduce a novel self-attention graph network that improves the generative
capabilities of SAG-VAE by parameterizing the generative distribution allowing
SAG-VAE to generate new data via graph convolution, while still trainable via
backpropagation. A learnable relational graph representation enhances SAG-VAE's
robustness to perturbation and noise, while also providing deeper intuition
into model performance. Experiments based on graphs show that SAG-VAE is
capable of approximately retrieving edges and links between nodes based
entirely on feature observations. Finally, results on image data illustrate
that SAG-VAE is fairly robust against perturbations in image reconstruction and
sampling.
