Technical report: Training Mixture Density Networks with full covariance
  matrices
  Mixture Density Networks are a tried and tested tool for modelling
conditional probability distributions. As such, they constitute a great
baseline for novel approaches to this problem. In the standard formulation, an
MDN takes some input and outputs parameters for a Gaussian mixture model with
restrictions on the mixture components' covariance. Since covariance between
random variables is a central issue in the conditional modeling problems we
were investigating, I derived and implemented an MDN formulation with
unrestricted covariances. It is likely that this has been done before, but I
could not find any resources online. For this reason, I have documented my
approach in the form of this technical report, in hopes that it may be useful
to others facing a similar situation.
