A survey of benchmarking frameworks for reinforcement learning
  Reinforcement learning has recently experienced increased prominence in the
machine learning community. There are many approaches to solving reinforcement
learning problems with new techniques developed constantly. When solving
problems using reinforcement learning, there are various difficult challenges
to overcome. To ensure progress in the field, benchmarks are important for
testing new algorithms and comparing with other approaches. The reproducibility
of results for fair comparison is therefore vital in ensuring that improvements
are accurately judged. This paper provides an overview of different
contributions to reinforcement learning benchmarking and discusses how they can
assist researchers to address the challenges facing reinforcement learning. The
contributions discussed are the most used and recent in the literature. The
paper discusses the contributions in terms of implementation, tasks and
provided algorithm implementations with benchmarks. The survey aims to bring
attention to the wide range of reinforcement learning benchmarking tasks
available and to encourage research to take place in a standardised manner.
Additionally, this survey acts as an overview for researchers not familiar with
the different tasks that can be used to develop and test new reinforcement
learning algorithms.
