Learning in embodied action-perception loops through exploration
  Although exploratory behaviors are ubiquitous in the animal kingdom, their
computational underpinnings are still largely unknown. Behavioral Psychology
has identified learning as a primary drive underlying many exploratory
behaviors. Exploration is seen as a means for an animal to gather sensory data
useful for reducing its ignorance about the environment. While related problems
have been addressed in Data Mining and Reinforcement Learning, the
computational modeling of learning-driven exploration by embodied agents is
largely unrepresented.
  Here, we propose a computational theory for learning-driven exploration based
on the concept of missing information that allows an agent to identify
informative actions using Bayesian inference. We demonstrate that when
embodiment constraints are high, agents must actively coordinate their actions
to learn efficiently. Compared to earlier approaches, our exploration policy
yields more efficient learning across a range of worlds with diverse
structures. The improved learning in turn affords greater success in general
tasks including navigation and reward gathering. We conclude by discussing how
the proposed theory relates to previous information-theoretic objectives of
behavior, such as predictive information and the free energy principle, and how
it might contribute to a general theory of exploratory behavior.
