HierTrain: Fast Hierarchical Edge AI Learning with Hybrid Parallelism in
  Mobile-Edge-Cloud Computing
  Nowadays, deep neural networks (DNNs) are the core enablers for many emerging
edge AI applications. Conventional approaches to training DNNs are generally
implemented at central servers or cloud centers for centralized learning, which
is typically time-consuming and resource-demanding due to the transmission of a
large amount of data samples from the device to the remote cloud. To overcome
these disadvantages, we consider accelerating the learning process of DNNs on
the Mobile-Edge-Cloud Computing (MECC) paradigm. In this paper, we propose
HierTrain, a hierarchical edge AI learning framework, which efficiently deploys
the DNN training task over the hierarchical MECC architecture. We develop a
novel \textit{hybrid parallelism} method, which is the key to HierTrain, to
adaptively assign the DNN model layers and the data samples across the three
levels of edge device, edge server and cloud center. We then formulate the
problem of scheduling the DNN training tasks at both layer-granularity and
sample-granularity. Solving this optimization problem enables us to achieve the
minimum training time. We further implement a hardware prototype consisting of
an edge device, an edge server and a cloud server, and conduct extensive
experiments on it. Experimental results demonstrate that HierTrain can achieve
up to 6.9x speedup compared to the cloud-based hierarchical training approach.
