ALERT: Accurate Learning for Energy and Timeliness
  An increasing number of software applications incorporate runtime Deep Neural
Networks (DNNs) to process sensor data and return inference results to humans.
Effective deployment of DNNs in these interactive scenarios requires meeting
latency and accuracy constraints while minimizing energy, a problem exacerbated
by common system dynamics. Prior approaches handle dynamics through either (1)
system-oblivious DNN adaptation, which adjusts DNN latency/accuracy tradeoffs,
or (2) application-oblivious system adaptation, which adjusts resources to
change latency/energy tradeoffs. In contrast, this paper improves on the
state-of-the-art by coordinating application- and system-level adaptation.
ALERT, our runtime scheduler, uses a probabilistic model to detect
environmental volatility and then simultaneously select both a DNN and a system
resource configuration to meet latency, accuracy, and energy constraints. We
evaluate ALERT on CPU and GPU platforms for image and speech tasks in dynamic
environments. ALERT's holistic approach achieves more than 13% energy
reduction, and 27% error reduction over prior approaches that adapt solely at
the application or system level. Furthermore, ALERT incurs only 3% more energy
consumption and 2% higher DNN-inference error than an oracle scheme with
perfect application and system knowledge.
