Demystifying Deep Learning in Predictive Spatio-Temporal Analytics: An
  Information-Theoretic Framework
  Deep learning has achieved incredible success over the past years, especially
in various challenging predictive spatio-temporal analytics (PSTA) tasks, such
as disease prediction, climate forecast, and traffic prediction, where
intrinsic dependency relationships among data exist and generally manifest at
multiple spatio-temporal scales. However, given a specific PSTA task and the
corresponding dataset, how to appropriately determine the desired configuration
of a deep learning model, theoretically analyze the model's learning behavior,
and quantitatively characterize the model's learning capacity remains a
mystery. In order to demystify the power of deep learning for PSTA, in this
paper, we provide a comprehensive framework for deep learning model design and
information-theoretic analysis. First, we develop and demonstrate a novel
interactively- and integratively-connected deep recurrent neural network
(I$^2$DRNN) model. I$^2$DRNN consists of three modules: an Input module that
integrates data from heterogeneous sources; a Hidden module that captures the
information at different scales while allowing the information to flow
interactively between layers; and an Output module that models the integrative
effects of information from various hidden layers to generate the output
predictions. Second, to theoretically prove that our designed model can learn
multi-scale spatio-temporal dependency in PSTA tasks, we provide an
information-theoretic analysis to examine the information-based learning
capacity (i-CAP) of the proposed model. Third, to validate the I$^2$DRNN model
and confirm its i-CAP, we systematically conduct a series of experiments
involving both synthetic datasets and real-world PSTA tasks. The experimental
results show that the I$^2$DRNN model outperforms both classical and
state-of-the-art models, and is able to capture meaningful multi-scale
spatio-temporal dependency.
