Balancing New Against Old Information: The Role of Surprise in Learning
  Surprise describes a range of phenomena from unexpected events to behavioral
responses. We propose a measure of surprise and use it for surprise-driven
learning. Our surprise measure takes into account data likelihood as well as
the degree of commitment to a belief via the entropy of the belief
distribution. We find that surprise-minimizing learning dynamically adjusts the
balance between new and old information without the need of knowledge about the
temporal statistics of the environment. We apply our framework to a dynamic
decision-making task and a maze exploration task. Our surprise minimizing
framework is suitable for learning in complex environments, even if the
environment undergoes gradual or sudden changes and could eventually provide a
framework to study the behavior of humans and animals encountering surprising
events.
