Learning Robust State Abstractions for Hidden-Parameter Block MDPs
  Many control tasks exhibit similar dynamics that can be modeled as having
common latent structure. Hidden-Parameter Markov Decision Processes (HiP-MDPs)
explicitly model this structure to improve sample efficiency in multi-task
settings. However, this setting makes strong assumptions on the observability
of the state that limit its application in real-world scenarios with rich
observation spaces. In this work, we leverage ideas of common structure from
the HiP-MDP setting, and extend it to enable robust state abstractions inspired
by Block MDPs. We derive instantiations of this new framework for both
multi-task reinforcement learning (MTRL) and meta-reinforcement learning
(Meta-RL) settings. Further, we provide transfer and generalization bounds
based on task and state similarity, along with sample complexity bounds that
depend on the aggregate number of samples across tasks, rather than the number
of tasks, a significant improvement over prior work that use the same
environment assumptions. To further demonstrate the efficacy of the proposed
method, we empirically compare and show improvement over multi-task and
meta-reinforcement learning baselines.
