Leveraging the structure of dynamical systems for data-driven modeling
  The reliable prediction of the temporal behavior of complex systems is
required in numerous scientific fields. This strong interest is however
hindered by modeling issues: often, the governing equations describing the
physics of the system under consideration are not accessible or, when known,
their solution might require a computational time incompatible with the
prediction time constraints. Nowadays, approximating complex systems at hand in
a generic functional format and informing it ex--nihilo from available
observations has become a common practice, as illustrated by the enormous
amount of scientific work appeared in the last years. Numerous successful
examples based on deep neural networks are already available, although
generalizability of the models and margins of guarantee are often overlooked.
Here, we consider Long-Short Term Memory neural networks and thoroughly
investigate the impact of the training set and its structure on the quality of
the long-term prediction. Leveraging insights from ergodic theory, we perform a
thorough computational analysis to assess the amount of data sufficient for a
priori guaranteeing a faithful model of the physical system. We show how an
informed design of the training set, based on invariants of the system and the
structure of the underlying attractor, significantly improves the resulting
models, opening up avenues for research within the context of active learning.
Further, the non-trivial effects of the memory initializations when relying on
memory-capable models will be illustrated. Our findings provide evidence-based
good-practice on the amount and the choice of data required for an effective
data-driven modeling of any complex dynamical system.
