A Statistical Investigation of Long Memory in Language and Music
  Representation and learning of long-range dependencies is a central challenge
confronted in modern applications of machine learning to sequence data. Yet
despite the prominence of this issue, the basic problem of measuring long-range
dependence, either in a given data source or as represented in a trained deep
model, remains largely limited to heuristic tools. We contribute a statistical
framework for investigating long-range dependence in current applications of
deep sequence modeling, drawing on the well-developed theory of long memory
stochastic processes. This framework yields testable implications concerning
the relationship between long memory in real-world data and its learned
representation in a deep learning architecture, which are explored through a
semiparametric framework adapted to the high-dimensional setting.
