Mutual Information State Intrinsic Control
  Reinforcement learning has been shown to be highly successful at many
challenging tasks. However, success heavily relies on well-shaped rewards.
Intrinsically motivated RL attempts to remove this constraint by defining an
intrinsic reward function. Motivated by the self-consciousness concept in
psychology, we make a natural assumption that the agent knows what constitutes
itself, and propose a new intrinsic objective that encourages the agent to have
maximum control on the environment. We mathematically formalize this reward as
the mutual information between the agent state and the surrounding state under
the current agent policy. With this new intrinsic motivation, we are able to
outperform previous methods, including being able to complete the
pick-and-place task for the first time without using any task reward. A video
showing experimental results is available at https://youtu.be/AUCwc9RThpk.
