Emergent Symbols through Binding in External Memory
  A key aspect of human intelligence is the ability to infer abstract rules
directly from high-dimensional sensory data, and to do so given only a limited
amount of training experience. Deep neural network algorithms have proven to be
a powerful tool for learning directly from high-dimensional data, but currently
lack this capacity for data-efficient induction of abstract rules, leading some
to argue that symbol-processing mechanisms will be necessary to account for
this capacity. In this work, we take a step toward bridging this gap by
introducing the Emergent Symbol Binding Network (ESBN), a recurrent network
augmented with an external memory that enables a form of variable-binding and
indirection. This binding mechanism allows symbol-like representations to
emerge through the learning process without the need to explicitly incorporate
symbol-processing machinery, enabling the ESBN to learn rules in a manner that
is abstracted away from the particular entities to which those rules apply.
Across a series of tasks, we show that this architecture displays nearly
perfect generalization of learned rules to novel entities given only a limited
number of training examples, and outperforms a number of other competitive
neural network architectures.
