Learning Discrete State Abstractions With Deep Variational Inference
  Abstraction is crucial for effective sequential decision making in domains
with large state spaces. In this work, we propose an information bottleneck
method for learning approximate bisimulations, a type of state abstraction. We
use a deep neural encoder to map states onto continuous embeddings. We map
these embeddings onto a discrete representation using an action-conditioned
hidden Markov model, which is trained end-to-end with the neural network. Our
method is suited for environments with high-dimensional states and learns from
a stream of experience collected by an agent acting in a Markov decision
process. Through this learned discrete abstract model, we can efficiently plan
for unseen goals in a multi-goal Reinforcement Learning setting. We test our
method in simplified robotic manipulation domains with image states. We also
compare it against previous model-based approaches to finding bisimulations in
discrete grid-world-like environments. Source code is available at
https://github.com/ondrejba/discrete_abstractions.
