SparkNet: Training Deep Networks in Spark
  Training deep networks is a time-consuming process, with networks for object
recognition often requiring multiple days to train. For this reason, leveraging
the resources of a cluster to speed up training is an important area of work.
However, widely-popular batch-processing computational frameworks like
MapReduce and Spark were not designed to support the asynchronous and
communication-intensive workloads of existing distributed deep learning
systems. We introduce SparkNet, a framework for training deep networks in
Spark. Our implementation includes a convenient interface for reading data from
Spark RDDs, a Scala interface to the Caffe deep learning framework, and a
lightweight multi-dimensional tensor library. Using a simple parallelization
scheme for stochastic gradient descent, SparkNet scales well with the cluster
size and tolerates very high-latency communication. Furthermore, it is easy to
deploy and use with no parameter tuning, and it is compatible with existing
Caffe models. We quantify the dependence of the speedup obtained by SparkNet on
the number of machines, the communication frequency, and the cluster's
communication overhead, and we benchmark our system's performance on the
ImageNet dataset.
