Time-Dependent Representation for Neural Event Sequence Prediction
  Existing sequence prediction methods are mostly concerned with
time-independent sequences, in which the actual time span between events is
irrelevant and the distance between events is simply the difference between
their order positions in the sequence. While this time-independent view of
sequences is applicable for data such as natural languages, e.g., dealing with
words in a sentence, it is inappropriate and inefficient for many real world
events that are observed and collected at unequally spaced points of time as
they naturally arise, e.g., when a person goes to a grocery store or makes a
phone call. The time span between events can carry important information about
the sequence dependence of human behaviors. In this work, we propose a set of
methods for using time in sequence prediction. Because neural sequence models
such as RNN are more amenable for handling token-like input, we propose two
methods for time-dependent event representation, based on the intuition on how
time is tokenized in everyday life and previous work on embedding
contextualization. We also introduce two methods for using next event duration
as regularization for training a sequence prediction model. We discuss these
methods based on recurrent neural nets. We evaluate these methods as well as
baseline models on five datasets that resemble a variety of sequence prediction
tasks. The experiments revealed that the proposed methods offer accuracy gain
over baseline models in a range of settings.
