Multi-Modal Prototype Learning for Interpretable Multivariable Time
  Series Classification
  Multivariable time series classification problems are increasing in
prevalence and complexity in a variety of domains, such as biology and finance.
While deep learning methods are an effective tool for these problems, they
often lack interpretability. In this work, we propose a novel modular prototype
learning framework for multivariable time series classification. In the first
stage of our framework, encoders extract features from each variable
independently. Prototype layers identify single-variable prototypes in the
resulting feature spaces. The next stage of our framework represents the
multivariable time series sample points in terms of their similarity to these
single-variable prototypes. This results in an inherently interpretable
representation of multivariable patterns, on which prototype learning is
applied to extract representative examples i.e. multivariable prototypes. Our
framework is thus able to explicitly identify both informative patterns in the
individual variables, as well as the relationships between the variables. We
validate our framework on a simulated dataset with embedded patterns, as well
as a real human activity recognition problem. Our framework attains comparable
or superior classification performance to existing time series classification
methods on these tasks. On the simulated dataset, we find that our model
returns interpretations consistent with the embedded patterns. Moreover, the
interpretations learned on the activity recognition dataset align with domain
knowledge.
