Recurrent Attentive Neural Process for Sequential Data
  Neural processes (NPs) learn stochastic processes and predict the
distribution of target output adaptively conditioned on a context set of
observed input-output pairs. Furthermore, Attentive Neural Process (ANP)
improved the prediction accuracy of NPs by incorporating attention mechanism
among contexts and targets. In a number of real-world applications such as
robotics, finance, speech, and biology, it is critical to learn the temporal
order and recurrent structure from sequential data. However, the capability of
NPs capturing these properties is limited due to its permutation invariance
instinct. In this paper, we proposed the Recurrent Attentive Neural Process
(RANP), or alternatively, Attentive Neural Process-RecurrentNeural
Network(ANP-RNN), in which the ANP is incorporated into a recurrent neural
network. The proposed model encapsulates both the inductive biases of recurrent
neural networks and also the strength of NPs for modelling uncertainty. We
demonstrate that RANP can effectively model sequential data and outperforms NPs
and LSTMs remarkably in a 1D regression toy example as well as
autonomous-driving applications.
