Variational State-Space Models for Localisation and Dense 3D Mapping in
  6 DoF
  We solve the problem of 6-DoF localisation and 3D dense reconstruction in
spatial environments as approximate Bayesian inference in a deep state-space
model. Our approach leverages both learning and domain knowledge from
multiple-view geometry and rigid-body dynamics. This results in an expressive
predictive model of the world, often missing in current state-of-the-art visual
SLAM solutions. The combination of variational inference, neural networks and a
differentiable raycaster ensures that our model is amenable to end-to-end
gradient-based optimisation. We evaluate our approach on realistic unmanned
aerial vehicle flight data, nearing the performance of state-of-the-art
visual-inertial odometry systems. We demonstrate the applicability of the model
to generative prediction and planning.
