Practical Conditional Neural Processes Via Tractable Dependent
  Predictions
  Conditional Neural Processes (CNPs; Garnelo et al., 2018a) are meta-learning
models which leverage the flexibility of deep learning to produce
well-calibrated predictions and naturally handle off-the-grid and missing data.
CNPs scale to large datasets and train with ease. Due to these features, CNPs
appear well-suited to tasks from environmental sciences or healthcare.
Unfortunately, CNPs do not produce correlated predictions, making them
fundamentally inappropriate for many estimation and decision making tasks.
Predicting heat waves or floods, for example, requires modelling dependencies
in temperature or precipitation over time and space. Existing approaches which
model output dependencies, such as Neural Processes (NPs; Garnelo et al.,
2018b) or the FullConvGNP (Bruinsma et al., 2021), are either complicated to
train or prohibitively expensive. What is needed is an approach which provides
dependent predictions, but is simple to train and computationally tractable. In
this work, we present a new class of Neural Process models that make correlated
predictions and support exact maximum likelihood training that is simple and
scalable. We extend the proposed models by using invertible output
transformations, to capture non-Gaussian output distributions. Our models can
be used in downstream estimation tasks which require dependent function
samples. By accounting for output dependencies, our models show improved
predictive performance on a range of experiments with synthetic and real data.
