A deep convolutional neural network that is invariant to time rescaling
  Human learners can readily understand speech, or a melody, when it is
presented slower or faster than usual. Although deep convolutional neural
networks (CNNs) are extremely powerful in extracting information from time
series, they require explicit training to generalize to different time scales.
This paper presents a deep CNN that incorporates a temporal representation
inspired by recent findings from neuroscience. In the mammalian brain, time is
represented by populations of neurons with temporal receptive fields.
Critically, the peaks of the receptive fields form a geometric series, such
that the population codes a set of temporal basis functions over log time.
Because memory for the recent past is a function of log time, rescaling the
input results in translation of the memory. The Scale-Invariant Temporal
History Convolution network (SITHCon) builds a convolutional layer over this
logarithmically-distributed temporal memory. A max-pool operation results in a
network that is invariant to rescalings of time modulo edge effects. We compare
performance of SITHCon to a Temporal Convolution Network (TCN). Although both
networks can learn classification and regression problems on both univariate
and multivariate time series f(t), only SITHCon generalizes to rescalings
f(at). This property, inspired by findings from contemporary neuroscience and
consistent with findings from cognitive psychology, may enable networks that
learn with fewer training examples, fewer weights and that generalize more
robustly to out of sample data.
