Learning to Generalize Compositionally by Transferring Across Semantic
  Parsing Tasks
  Neural network models often generalize poorly to mismatched domains or
distributions. In NLP, this issue arises in particular when models are expected
to generalize compositionally, that is, to novel combinations of familiar words
and constructions. We investigate learning representations that facilitate
transfer learning from one compositional task to another: the representation
and the task-specific layers of the models are strategically trained
differently on a pre-finetuning task such that they generalize well on
mismatched splits that require compositionality. We apply this method to
semantic parsing, using three very different datasets, COGS, GeoQuery and SCAN,
used alternately as the pre-finetuning and target task. Our method
significantly improves compositional generalization over baselines on the test
set of the target task, which is held out during fine-tuning. Ablation studies
characterize the utility of the major steps in the proposed algorithm and
support our hypothesis.
