Can neural networks predict dynamics they have never seen?
  Neural networks have proven to be remarkably successful for a wide range of
complicated tasks, from image recognition and object detection to speech
recognition and machine translation. One of their successes is the skill in
prediction of future dynamics given a suitable training set of data. Previous
studies have shown how Echo State Networks (ESNs), a subset of Recurrent Neural
Networks, can successfully predict even chaotic systems for times longer than
the Lyapunov time. This study shows that, remarkably, ESNs can successfully
predict dynamical behavior that is qualitatively different from any behavior
contained in the training set. Evidence is provided for a fluid dynamics
problem where the flow can transition between laminar (ordered) and turbulent
(disordered) regimes. Despite being trained on the turbulent regime only, ESNs
are found to predict laminar behavior. Moreover, the statistics of
turbulent-to-laminar and laminar-to-turbulent transitions are also predicted
successfully, and the utility of ESNs in acting as an early-warning system for
transition is discussed. These results are expected to be widely applicable to
data-driven modelling of temporal behaviour in a range of physical, climate,
biological, ecological and finance models characterized by the presence of
tipping points and sudden transitions between several competing states.
