Scenic4RL: Programmatic Modeling and Generation of Reinforcement
  Learning Environments
  The capability of reinforcement learning (RL) agent directly depends on the
diversity of learning scenarios the environment generates and how closely it
captures real-world situations. However, existing environments/simulators lack
the support to systematically model distributions over initial states and
transition dynamics. Furthermore, in complex domains such as soccer, the space
of possible scenarios is infinite, which makes it impossible for one research
group to provide a comprehensive set of scenarios to train, test, and benchmark
RL algorithms. To address this issue, for the first time, we adopt an existing
formal scenario specification language, SCENIC, to intuitively model and
generate interactive scenarios. We interfaced SCENIC to Google Research Soccer
environment to create a platform called SCENIC4RL. Using this platform, we
provide a dataset consisting of 36 scenario programs encoded in SCENIC and
demonstration data generated from a subset of them. We share our experimental
results to show the effectiveness of our dataset and the platform to train,
test, and benchmark RL algorithms. More importantly, we open-source our
platform to enable RL community to collectively contribute to constructing a
comprehensive set of scenarios.
