Unsupervised Learning of Temporal Abstractions with Slot-based
  Transformers
  The discovery of reusable sub-routines simplifies decision-making and
planning in complex reinforcement learning problems. Previous approaches
propose to learn such temporal abstractions in a purely unsupervised fashion
through observing state-action trajectories gathered from executing a policy.
However, a current limitation is that they process each trajectory in an
entirely sequential manner, which prevents them from revising earlier decisions
about sub-routine boundary points in light of new incoming information. In this
work we propose SloTTAr, a fully parallel approach that integrates sequence
processing Transformers with a Slot Attention module and adaptive computation
for learning about the number of such sub-routines in an unsupervised fashion.
We demonstrate how SloTTAr is capable of outperforming strong baselines in
terms of boundary point discovery, even for sequences containing variable
amounts of sub-routines, while being up to 7x faster to train on existing
benchmarks.
