A Simple Neural Attentive Meta-Learner
  Deep neural networks excel in regimes with large amounts of data, but tend to
struggle when data is scarce or when they need to adapt quickly to changes in
the task. In response, recent work in meta-learning proposes training a
meta-learner on a distribution of similar tasks, in the hopes of generalization
to novel but related tasks by learning a high-level strategy that captures the
essence of the problem it is asked to solve. However, many recent meta-learning
approaches are extensively hand-designed, either using architectures
specialized to a particular application, or hard-coding algorithmic components
that constrain how the meta-learner solves the task. We propose a class of
simple and generic meta-learner architectures that use a novel combination of
temporal convolutions and soft attention; the former to aggregate information
from past experience and the latter to pinpoint specific pieces of information.
In the most extensive set of meta-learning experiments to date, we evaluate the
resulting Simple Neural AttentIve Learner (or SNAIL) on several
heavily-benchmarked tasks. On all tasks, in both supervised and reinforcement
learning, SNAIL attains state-of-the-art performance by significant margins.
