Mutual information for fitting deep nonlinear models
  Deep nonlinear models pose a challenge for fitting parameters due to lack of
knowledge of the hidden layer and the potentially non-affine relation of the
initial and observed layers. In the present work we investigate the use of
information theoretic measures such as mutual information and Kullback-Leibler
(KL) divergence as objective functions for fitting such models without
knowledge of the hidden layer. We investigate one model as a proof of concept
and one application of cogntive performance. We further investigate the use of
optimizers with these methods. Mutual information is largely successful as an
objective, depending on the parameters. KL divergence is found to be similarly
succesful, given some knowledge of the statistics of the hidden layer.
