Extending the OpenAI Gym for robotics: a toolkit for reinforcement   learning using ROS and Gazebo
  This paper presents an extension of the OpenAI Gym for robotics using the Robot Operating System (ROS) and the Gazebo simulator. The content discusses the software architecture proposed and the results obtained by using two Reinforcement Learning techniques: Q-Learning and Sarsa. Ultimately, the output of this work presents a benchmarking system for robotics that allows different techniques and algorithms to be compared using the same virtual conditions. 
Iker Zamora, Nestor Gonzalez Lopez, Victor Mayoral Vilches and   Alejandro Hernandez Cordero
cs.RO