What can I do here? A Theory of Affordances in Reinforcement Learning
  Reinforcement learning algorithms usually assume that all actions are always
available to an agent. However, both people and animals understand the general
link between the features of their environment and the actions that are
feasible. Gibson (1977) coined the term "affordances" to describe the fact that
certain states enable an agent to do certain actions, in the context of
embodied agents. In this paper, we develop a theory of affordances for agents
who learn and plan in Markov Decision Processes. Affordances play a dual role
in this case. On one hand, they allow faster planning, by reducing the number
of actions available in any given situation. On the other hand, they facilitate
more efficient and precise learning of transition models from data, especially
when such models require function approximation. We establish these properties
through theoretical results as well as illustrative examples. We also propose
an approach to learn affordances and use it to estimate transition models that
are simpler and generalize better.
