TATi-Thermodynamic Analytics ToolkIt: TensorFlow-based software for
  posterior sampling in machine learning applications
  With the advent of GPU-assisted hardware and maturing high-efficiency
software platforms such as TensorFlow and PyTorch, Bayesian posterior sampling
for neural networks becomes plausible. In this article we discuss Bayesian
parametrization in machine learning based on Markov Chain Monte Carlo methods,
specifically discretized stochastic differential equations such as Langevin
dynamics and extended system methods in which an ensemble of walkers is
employed to enhance sampling. We provide a glimpse of the potential of the
sampling-intensive approach by studying (and visualizing) the loss landscape of
a neural network applied to the MNIST data set. Moreover, we investigate how
the sampling efficiency itself can be significantly enhanced through an
ensemble quasi-Newton preconditioning method. This article accompanies the
release of a new TensorFlow software package, the Thermodynamic Analytics
ToolkIt, which is used in the computational experiments.
