Active Predictive Coding Networks: A Neural Solution to the Problem of
  Learning Reference Frames and Part-Whole Hierarchies
  We introduce Active Predictive Coding Networks (APCNs), a new class of neural
networks that solve a major problem posed by Hinton and others in the fields of
artificial intelligence and brain modeling: how can neural networks learn
intrinsic reference frames for objects and parse visual scenes into part-whole
hierarchies by dynamically allocating nodes in a parse tree? APCNs address this
problem by using a novel combination of ideas: (1) hypernetworks are used for
dynamically generating recurrent neural networks that predict parts and their
locations within intrinsic reference frames conditioned on higher object-level
embedding vectors, and (2) reinforcement learning is used in conjunction with
backpropagation for end-to-end learning of model parameters. The APCN
architecture lends itself naturally to multi-level hierarchical learning and is
closely related to predictive coding models of cortical function. Using the
MNIST, Fashion-MNIST and Omniglot datasets, we demonstrate that APCNs can (a)
learn to parse images into part-whole hierarchies, (b) learn compositional
representations, and (c) transfer their knowledge to unseen classes of objects.
With their ability to dynamically generate parse trees with part locations for
objects, APCNs offer a new framework for explainable AI that leverages advances
in deep learning while retaining interpretability and compositionality.
