Offloading Algorithms for Maximizing Inference Accuracy on Edge Device
  Under a Time Constraint
  With the emergence of edge computing, the problem of offloading jobs between
an Edge Device (ED) and an Edge Server (ES) received significant attention in
the past. Motivated by the fact that an increasing number of applications are
using Machine Learning (ML) inference, we study the problem of offloading
inference jobs by considering the following novel aspects: 1) in contrast to a
typical computational job, the processing time of an inference job depends on
the size of the ML model, and 2) recently proposed Deep Neural Networks (DNNs)
for resource-constrained devices provide the choice of scaling the model size.
We formulate an assignment problem with the aim of maximizing the total
inference accuracy of n data samples available at the ED, subject to a time
constraint T on the makespan. We propose an approximation algorithm AMR2, and
prove that it results in a makespan at most 2T, and achieves a total accuracy
that is lower by a small constant from optimal total accuracy. As proof of
concept, we implemented AMR2 on a Raspberry Pi, equipped with MobileNet, and is
connected to a server equipped with ResNet, and studied the total accuracy and
makespan performance of AMR2 for image classification application.
