GELATO: Geometrically Enriched Latent Model for Offline Reinforcement
  Learning
  Offline reinforcement learning approaches can generally be divided to
proximal and uncertainty-aware methods. In this work, we demonstrate the
benefit of combining the two in a latent variational model. We impose a latent
representation of states and actions and leverage its intrinsic Riemannian
geometry to measure distance of latent samples to the data. Our proposed
metrics measure both the quality of out of distribution samples as well as the
discrepancy of examples in the data. We integrate our metrics in a model-based
offline optimization framework, in which proximity and uncertainty can be
carefully controlled. We illustrate the geodesics on a simple grid-like
environment, depicting its natural inherent topology. Finally, we analyze our
approach and improve upon contemporary offline RL benchmarks.
