Active Inference for Robotic Manipulation
Robotic manipulation stands as a largely unsolved problem despite significant
advances in robotics and machine learning in the last decades. One of the
central challenges of manipulation is partial observability, as the agent
usually does not know all physical properties of the environment and the
objects it is manipulating in advance. A recently emerging theory that deals
with partial observability in an explicit manner is Active Inference. It does
so by driving the agent to act in a way that is not only goal-directed but also
informative about the environment. In this work, we apply Active Inference to a
hard-to-explore simulated robotic manipulation tasks, in which the agent has to
balance a ball into a target zone. Since the reward of this task is sparse, in
order to explore this environment, the agent has to learn to balance the ball
without any extrinsic feedback, purely driven by its own curiosity. We show
that the information-seeking behavior induced by Active Inference allows the
agent to explore these challenging, sparse environments systematically.
Finally, we conclude that using an information-seeking objective is beneficial
in sparse environments and allows the agent to solve tasks in which methods
that do not exhibit directed exploration fail.