Learning to Perform Role-Filler Binding with Schematic Knowledge
  Through specific experiences, humans learn relationships underlying the
structure of events in the world. Schema theory suggests that we organize this
information in mental frameworks called "schemata," which represent our
knowledge of the structure of the world. Generalizing knowledge of structural
relationships to new situations requires role-filler binding, the ability to
associate specific "fillers" with abstract "roles." For instance, when we hear
the sentence "Alice ordered a tea from Bob," the role-filler bindings
"Alice:customer," "tea:drink," and "Bob:barista" allow us to understand and
make inferences about the sentence. We can perform these bindings for arbitrary
fillers -- we understand this sentence even if we have never heard the names
"Alice," "tea," or "Bob" before. In this work, we define a model as capable of
performing role-filler binding if it can recall arbitrary fillers corresponding
to a specified role, even when these pairings violate correlations seen during
training. Previous work found that models can learn this ability when
explicitly told what the roles and fillers are, or when given fillers seen
during training. We show that networks with external memory can learn these
relationships with fillers not seen during training and without explicitly
labeled role-filler bindings, and show that analyses inspired by neural
decoding can provide a means of understanding what the networks have learned.
