3D Simulation for Robot Arm Control with Deep Q-Learning
  Recent trends in robot arm control have seen a shift towards end-to-end
solutions, using deep reinforcement learning to learn a controller directly
from raw sensor data, rather than relying on a hand-crafted, modular pipeline.
However, the high dimensionality of the state space often means that it is
impractical to generate sufficient training data with real-world experiments.
As an alternative solution, we propose to learn a robot controller in
simulation, with the potential of then transferring this to a real robot.
Building upon the recent success of deep Q-networks, we present an approach
which uses 3D simulations to train a 7-DOF robotic arm in a control task
without any prior knowledge. The controller accepts images of the environment
as its only input, and outputs motor actions for the task of locating and
grasping a cube, over a range of initial configurations. To encourage efficient
learning, a structured reward function is designed with intermediate rewards.
We also present preliminary results in direct transfer of policies over to a
real robot, without any further training.
