Differential Bayesian Neural Nets
  Neural Ordinary Differential Equations (N-ODEs) are a powerful building block
for learning systems, which extend residual networks to a continuous-time
dynamical system. We propose a Bayesian version of N-ODEs that enables
well-calibrated quantification of prediction uncertainty, while maintaining the
expressive power of their deterministic counterpart. We assign Bayesian Neural
Nets (BNNs) to both the drift and the diffusion terms of a Stochastic
Differential Equation (SDE) that models the flow of the activation map in time.
We infer the posterior on the BNN weights using a straightforward adaptation of
Stochastic Gradient Langevin Dynamics (SGLD). We illustrate significantly
improved stability on two synthetic time series prediction tasks and report
better model fit on UCI regression benchmarks with our method when compared to
its non-Bayesian counterpart.
