Robotic Lever Manipulation using Hindsight Experience Replay and Shapley
  Additive Explanations
  This paper deals with robotic lever control using Explainable Deep
Reinforcement Learning. First, we train a policy by using the Deep
Deterministic Policy Gradient algorithm and the Hindsight Experience Replay
technique, where the goal is to control a robotic manipulator to manipulate a
lever. This enables us both to use continuous states and actions and to learn
with sparse rewards. Being able to learn from sparse rewards is especially
desirable for Deep Reinforcement Learning because designing a reward function
for complex tasks such as this is challenging. We first train in the PyBullet
simulator, which accelerates the training procedure, but is not accurate on
this task compared to the real-world environment. After completing the training
in PyBullet, we further train in the Gazebo simulator, which runs more slowly
than PyBullet, but is more accurate on this task. We then transfer the policy
to the real-world environment, where it achieves comparable performance to the
simulated environments for most episodes. To explain the decisions of the
policy we use the SHAP method to create an explanation model based on the
episodes done in the real-world environment. This gives us some results that
agree with intuition, and some that do not. We also question whether the
independence assumption made when approximating the SHAP values influences the
accuracy of these values for a system such as this, where there are some
correlations between the states.
