Free Will Belief as a consequence of Model-based Reinforcement Learning
  The debate on whether or not humans have free will has been raging for
centuries. Although there are good arguments based on our current understanding
of the laws of nature for the view that it is not possible for humans to have
free will, most people believe they do. This discrepancy begs for an
explanation. If we accept that we do not have free will, we are faced with two
problems: (1) while freedom is a very commonly used concept that everyone
intuitively understands, what are we actually referring to when we say that an
action or choice is "free" or not? And, (2) why is the belief in free will so
common? Where does this belief come from, and what is its purpose, if any? In
this paper, we examine these questions from the perspective of reinforcement
learning (RL). RL is a framework originally developed for training artificial
intelligence agents. However, it can also be used as a computational model of
human decision making and learning, and by doing so, we propose that the first
problem can be answered by observing that people's common sense understanding
of freedom is closely related to the information entropy of an RL agent's
normalized action values, while the second can be explained by the necessity
for agents to model themselves as if they could have taken decisions other than
those they actually took, when dealing with the temporal credit assignment
problem. Put simply, we suggest that by applying the RL framework as a model
for human learning it becomes evident that in order for us to learn efficiently
and be intelligent we need to view ourselves as if we have free will.
