On-Demand Video Dispatch Networks: A Scalable End-to-End Learning
  Approach
  We design a dispatch system to improve the peak service quality of video on
demand (VOD). Our system predicts the hot videos during the peak hours of the
next day based on the historical requests, and dispatches to the content
delivery networks (CDNs) at the previous off-peak time. In order to scale to
billions of videos, we build the system with two neural networks, one for video
clustering and the other for dispatch policy developing. The clustering network
employs autoencoder layers and reduces the video number to a fixed value. The
policy network employs fully connected layers and ranks the clustered videos
with dispatch probabilities. The two networks are coupled with weight-sharing
temporal layers, which analyze the video request sequences with convolutional
and recurrent modules. Therefore, the clustering and dispatch tasks are trained
in an end-to-end mechanism. The real-world results show that our approach
achieves an average prediction accuracy of 17%, compared with 3% from the
present baseline method, for the same amount of dispatches.
