Information-Theoretic Methods for Planning and Learning in Partially
  Observable Markov Decision Processes
  Bounded agents are limited by intrinsic constraints on their ability to
process information that is available in their sensors and memory and choose
actions and memory updates. In this dissertation, we model these constraints as
information-rate constraints on communication channels connecting these various
internal components of the agent. We make four major contributions detailed
below and many smaller contributions detailed in each section. First, we
formulate the problem of optimizing the agent under both extrinsic and
intrinsic constraints and develop the main tools for solving it. Second, we
identify another reason for the challenging convergence properties of the
optimization algorithm, which is the bifurcation structure of the update
operator near phase transitions. Third, we study the special case of
linear-Gaussian dynamics and quadratic cost (LQG), where the optimal solution
has a particularly simple and solvable form. Fourth, we explore the learning
task, where the model of the world dynamics is unknown and sample-based updates
are used instead.
