Modeling the effects of environmental and perceptual uncertainty using
  deterministic reinforcement learning dynamics with partial observability
  Assessing the systemic effects of uncertainty that arises from agents'
partial observation of the true states of the world is critical for
understanding a wide range of scenarios. Yet, previous modeling work on agent
learning and decision-making either lacks a systematic way to describe this
source of uncertainty or puts the focus on obtaining optimal policies using
complex models of the world that would impose an unrealistically high cognitive
demand on real agents. In this work we aim to efficiently describe the emergent
behavior of biologically plausible and parsimonious learning agents faced with
partially observable worlds. Therefore we derive and present deterministic
reinforcement learning dynamics where the agents observe the true state of the
environment only partially. We showcase the broad applicability of our dynamics
across different classes of partially observable agent-environment systems. We
find that partial observability creates unintuitive benefits in a number of
specific contexts, pointing the way to further research on a general
understanding of such effects. For instance, partially observant agents can
learn better outcomes faster, in a more stable way and even overcome social
dilemmas. Furthermore, our method allows the application of dynamical systems
theory to partially observable multiagent leaning. In this regard we find the
emergence of catastrophic limit cycles, a critical slowing down of the learning
processes between reward regimes and the separation of the learning dynamics
into fast and slow directions, all caused by partial observability. Therefore,
the presented dynamics have the potential to become a formal, yet practical,
lightweight and robust tool for researchers in biology, social science and
machine learning to systematically investigate the effects of interacting
partially observant agents.
