Deep Learning At Scale and At Ease
  Recently, deep learning techniques have enjoyed success in various multimedia
applications, such as image classification and multi-modal data analysis. Large
deep learning models are developed for learning rich representations of complex
data. There are two challenges to overcome before deep learning can be widely
adopted in multimedia and other applications. One is usability, namely the
implementation of different models and training algorithms must be done by
non-experts without much effort especially when the model is large and complex.
The other is scalability, that is the deep learning system must be able to
provision for a huge demand of computing resources for training large models
with massive datasets. To address these two challenges, in this paper, we
design a distributed deep learning platform called SINGA which has an intuitive
programming model based on the common layer abstraction of deep learning
models. Good scalability is achieved through flexible distributed training
architecture and specific optimization techniques. SINGA runs on GPUs as well
as on CPUs, and we show that it outperforms many other state-of-the-art deep
learning systems. Our experience with developing and training deep learning
models for real-life multimedia applications in SINGA shows that the platform
is both usable and scalable.
