ED2: An Environment Dynamics Decomposition Framework for World Model
  Construction
  Model-based reinforcement learning methods achieve significant sample
efficiency in many tasks, but their performance is often limited by the
existence of the model error. To reduce the model error, previous works use a
single well-designed network to fit the entire environment dynamics, which
treats the environment dynamics as a black box. However, these methods lack to
consider the environmental decomposed property that the dynamics may contain
multiple sub-dynamics, which can be modeled separately, allowing us to
construct the world model more accurately. In this paper, we propose the
Environment Dynamics Decomposition (ED2), a novel world model construction
framework that models the environment in a decomposing manner. ED2 contains two
key components: sub-dynamics discovery (SD2) and dynamics decomposition
prediction (D2P). SD2 discovers the sub-dynamics in an environment and then D2P
constructs the decomposed world model following the sub-dynamics. ED2 can be
easily combined with existing MBRL algorithms and empirical results show that
ED2 significantly reduces the model error and boosts the performance of the
state-of-the-art MBRL algorithms on various tasks.
