A Memory-Augmented Neural Network Model of Abstract Rule Learning
  Human intelligence is characterized by a remarkable ability to infer abstract
rules from experience and apply these rules to novel domains. As such,
designing neural network algorithms with this capacity is an important step
toward the development of deep learning systems with more human-like
intelligence. However, doing so is a major outstanding challenge, one that some
argue will require neural networks to use explicit symbol-processing
mechanisms. In this work, we focus on neural networks' capacity for arbitrary
role-filler binding, the ability to associate abstract "roles" to
context-specific "fillers," which many have argued is an important mechanism
underlying the ability to learn and apply rules abstractly. Using a simplified
version of Raven's Progressive Matrices, a hallmark test of human intelligence,
we introduce a sequential formulation of a visual problem-solving task that
requires this form of binding. Further, we introduce the Emergent Symbol
Binding Network (ESBN), a recurrent neural network model that learns to use an
external memory as a binding mechanism. This mechanism enables symbol-like
variable representations to emerge through the ESBN's training process without
the need for explicit symbol-processing machinery. We empirically demonstrate
that the ESBN successfully learns the underlying abstract rule structure of our
task and perfectly generalizes this rule structure to novel fillers.
