Joint Perception and Control as Inference with an Object-based
  Implementation
  Existing model-based reinforcement learning methods often study perception
modeling and decision making separately. We introduce joint Perception and
Control as Inference (PCI), a general framework to combine perception and
control for partially observable environments through Bayesian inference. Based
on the fact that object-level inductive biases are critical in human perceptual
learning and reasoning, we propose Object-based Perception Control (OPC), an
instantiation of PCI which manages to facilitate control using automatic
discovered object-based representations. We develop an unsupervised end-to-end
solution and analyze the convergence of the perception model update.
Experiments in a high-dimensional pixel environment demonstrate the learning
effectiveness of our object-based perception control approach. Specifically, we
show that OPC achieves good perceptual grouping quality and outperforms several
strong baselines in accumulated rewards.
