Contrastive Conditional Neural Processes
  Conditional Neural Processes~(CNPs) bridge neural networks with probabilistic
inference to approximate functions of Stochastic Processes under meta-learning
settings. Given a batch of non-{\it i.i.d} function instantiations, CNPs are
jointly optimized for in-instantiation observation prediction and
cross-instantiation meta-representation adaptation within a generative
reconstruction pipeline. There can be a challenge in tying together such two
targets when the distribution of function observations scales to
high-dimensional and noisy spaces. Instead, noise contrastive estimation might
be able to provide more robust representations by learning distributional
matching objectives to combat such inherent limitation of generative models. In
light of this, we propose to equip CNPs by 1) aligning prediction with encoded
ground-truth observation, and 2) decoupling meta-representation adaptation from
generative reconstruction. Specifically, two auxiliary contrastive branches are
set up hierarchically, namely in-instantiation temporal contrastive
learning~({\tt TCL}) and cross-instantiation function contrastive
learning~({\tt FCL}), to facilitate local predictive alignment and global
function consistency, respectively. We empirically show that {\tt TCL} captures
high-level abstraction of observations, whereas {\tt FCL} helps identify
underlying functions, which in turn provides more efficient representations.
Our model outperforms other CNPs variants when evaluating function distribution
reconstruction and parameter identification across 1D, 2D and high-dimensional
time-series.
