A Decentralized Policy Gradient Approach to Multi-task Reinforcement
  Learning
  We develop a mathematical framework for solving multi-task reinforcement
learning (MTRL) problems based on a type of policy gradient method. The goal in
MTRL is to learn a common policy that operates effectively in different
environments; these environments have similar (or overlapping) state spaces,
but have different rewards and dynamics. We highlight two fundamental
challenges in MTRL that are not present in its single task counterpart, and
illustrate them with simple examples. We then develop a decentralized
entropy-regularized policy gradient method for solving the MTRL problem, and
study its finite-time convergence rate. We demonstrate the effectiveness of the
proposed method using a series of numerical experiments. These experiments
range from small-scale "GridWorld" problems that readily demonstrate the
trade-offs involved in multi-task learning to large-scale problems, where
common policies are learned to navigate an airborne drone in multiple
(simulated) environments.
