Attentional meta-learners are polythetic classifiers
  Polythetic classifications, based on shared patterns of features that need
neither be universal nor constant among members of a class, are common in the
natural world and greatly outnumber monothetic classifications over a set of
features. We show that threshold meta-learners require an embedding dimension
that is exponential in the number of features to emulate these functions. In
contrast, attentional classifiers are polythetic by default and able to solve
these problems with a linear embedding dimension. However, we find that in the
presence of task-irrelevant features, inherent to meta-learning problems,
attentional models are susceptible to misclassification. To address this
challenge, we further propose a self-attention feature-selection mechanism that
adaptively dilutes non-discriminative features. We demonstrate the
effectiveness of our approach in meta-learning Boolean functions, and synthetic
and real-world few-shot learning tasks.
