Example-based Hypernetworks for Out-of-Distribution Generalization
  While Natural Language Processing (NLP) algorithms keep reaching
unprecedented milestones, out-of-distribution generalization is still
challenging. In this paper we address the problem of multi-source adaptation to
unknown domains: Given labeled data from multiple source domains, we aim to
generalize to data drawn from target domains that are unknown to the algorithm
at training time. We present an algorithmic framework based on example-based
Hypernetwork adaptation: Given an input example, a T5 encoder-decoder first
generates a unique signature which embeds this example in the semantic space of
the source domains, and this signature is then fed into a Hypernetwork which
generates the weights of the task classifier. In an advanced version of our
model, the learned signature also serves for improving the representation of
the input example. In experiments with two tasks, sentiment classification and
natural language inference, across 29 adaptation settings, our algorithms
substantially outperform existing algorithms for this adaptation setup. To the
best of our knowledge, this is the first time Hypernetworks are applied to
domain adaptation or in example-based manner in NLP.
