Compositional Structure Learning for Sequential Video Data
  Conventional sequential learning methods such as Recurrent Neural Networks
(RNNs) focus on interactions between consecutive inputs, i.e. first-order
Markovian dependency. However, most of sequential data, as seen with videos,
have complex temporal dependencies that imply variable-length semantic flows
and their compositions, and those are hard to be captured by conventional
methods. Here, we propose Temporal Dependency Networks (TDNs) for learning
video data by discovering these complex structures of the videos. The TDNs
represent video as a graph whose nodes and edges correspond to frames of the
video and their dependencies respectively. Via a parameterized kernel with
graph-cut and graph convolutions, the TDNs find compositional temporal
dependencies of the data in multilevel graph forms. We evaluate the proposed
method on the large-scale video dataset Youtube-8M. The experimental results
show that our model efficiently learns the complex semantic structure of video
data.
