Reconstructing nonlinear dynamical systems from multi-modal time series
  Empirically observed time series in physics, biology, or medicine, are
commonly generated by some underlying dynamical system (DS) which is the target
of scientific interest. There is an increasing interest to harvest machine
learning methods to reconstruct this latent DS in a data-driven, unsupervised
way. In many areas of science it is common to sample time series observations
from many data modalities simultaneously, e.g. electrophysiological and
behavioral time series in a typical neuroscience experiment. However, current
machine learning tools for reconstructing DSs usually focus on just one data
modality. Here we propose a general framework for multi-modal data integration
for the purpose of nonlinear DS reconstruction and the analysis of cross-modal
relations. This framework is based on dynamically interpretable recurrent
neural networks as general approximators of nonlinear DSs, coupled to sets of
modality-specific decoder models from the class of generalized linear models.
Both an expectation-maximization and a variational inference algorithm for
model training are advanced and compared. We show on nonlinear DS benchmarks
that our algorithms can efficiently compensate for too noisy or missing
information in one data channel by exploiting other channels, and demonstrate
on experimental neuroscience data how the algorithm learns to link different
data domains to the underlying dynamics.
