Active Inference and Epistemic Value in Graphical Models
  The Free Energy Principle (FEP) postulates that biological agents perceive
and interact with their environment in order to minimize a Variational Free
Energy (VFE) with respect to a generative model of their environment. The
inference of a policy (future control sequence) according to the FEP is known
as Active Inference (AIF). The AIF literature describes multiple VFE objectives
for policy planning that lead to epistemic (information-seeking) behavior.
However, most objectives have limited modeling flexibility. This paper
approaches epistemic behavior from a constrained Bethe Free Energy (CBFE)
perspective. Crucially, variational optimization of the CBFE can be expressed
in terms of message passing on free-form generative models. The key intuition
behind the CBFE is that we impose a point-mass constraint on predicted
outcomes, which explicitly encodes the assumption that the agent will make
observations in the future. We interpret the CBFE objective in terms of its
constituent behavioral drives. We then illustrate resulting behavior of the
CBFE by planning and interacting with a simulated T-maze environment.
Simulations for the T-maze task illustrate how the CBFE agent exhibits an
epistemic drive, and actively plans ahead to account for the impact of
predicted outcomes. Compared to an EFE agent, the CBFE agent incurs expected
reward in significantly more environmental scenarios. We conclude that CBFE
optimization by message passing suggests a general mechanism for
epistemic-aware AIF in free-form generative models.
