Compositional generalization through meta sequence-to-sequence learning
  People can learn a new concept and use it compositionally, understanding how
to "blicket twice" after learning how to "blicket." In contrast, powerful
sequence-to-sequence (seq2seq) neural networks fail such tests of
compositionality, especially when composing new concepts together with existing
concepts. In this paper, I show how memory-augmented neural networks can be
trained to generalize compositionally through meta seq2seq learning. In this
approach, models train on a series of seq2seq problems to acquire the
compositional skills needed to solve new seq2seq problems. Meta se2seq learning
solves several of the SCAN tests for compositional learning and can learn to
apply implicit rules to variables.
