Unsupervised Learning from Video with Deep Neural Embeddings
  Because of the rich dynamical structure of videos and their ubiquity in
everyday life, it is a natural idea that video data could serve as a powerful
unsupervised learning signal for training visual representations in deep neural
networks. However, instantiating this idea, especially at large scale, has
remained a significant artificial intelligence challenge. Here we present the
Video Instance Embedding (VIE) framework, which extends powerful recent
unsupervised loss functions for learning deep nonlinear embeddings to
multi-stream temporal processing architectures on large-scale video datasets.
We show that VIE-trained networks substantially advance the state of the art in
unsupervised learning from video datastreams, both for action recognition in
the Kinetics dataset, and object recognition in the ImageNet dataset. We show
that a hybrid model with both static and dynamic processing pathways is optimal
for both transfer tasks, and provide analyses indicating how the pathways
differ. Taken in context, our results suggest that deep neural embeddings are a
promising approach to unsupervised visual learning across a wide variety of
domains.
