CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and
  Transfer Learning
  Despite recent successes of reinforcement learning (RL), it remains a
challenge for agents to transfer learned skills to related environments. To
facilitate research addressing this problem, we propose CausalWorld, a
benchmark for causal structure and transfer learning in a robotic manipulation
environment. The environment is a simulation of an open-source robotic
platform, hence offering the possibility of sim-to-real transfer. Tasks consist
of constructing 3D shapes from a given set of blocks - inspired by how children
learn to build complex structures. The key strength of CausalWorld is that it
provides a combinatorial family of such tasks with common causal structure and
underlying factors (including, e.g., robot and object masses, colors, sizes).
The user (or the agent) may intervene on all causal variables, which allows for
fine-grained control over how similar different tasks (or task distributions)
are. One can thus easily define training and evaluation distributions of a
desired difficulty level, targeting a specific form of generalization (e.g.,
only changes in appearance or object mass). Further, this common
parametrization facilitates defining curricula by interpolating between an
initial and a target task. While users may define their own task distributions,
we present eight meaningful distributions as concrete benchmarks, ranging from
simple to very challenging, all of which require long-horizon planning as well
as precise low-level motor control. Finally, we provide baseline results for a
subset of these tasks on distinct training curricula and corresponding
evaluation protocols, verifying the feasibility of the tasks in this benchmark.
