Exploring System Performance of Continual Learning for Mobile and
  Embedded Sensing Applications
  Continual learning approaches help deep neural network models adapt and learn
incrementally by trying to solve catastrophic forgetting. However, whether
these existing approaches, applied traditionally to image-based tasks, work
with the same efficacy to the sequential time series data generated by mobile
or embedded sensing systems remains an unanswered question.
  To address this void, we conduct the first comprehensive empirical study that
quantifies the performance of three predominant continual learning schemes
(i.e., regularization, replay, and replay with examples) on six datasets from
three mobile and embedded sensing applications in a range of scenarios having
different learning complexities. More specifically, we implement an end-to-end
continual learning framework on edge devices. Then we investigate the
generalizability, trade-offs between performance, storage, computational costs,
and memory footprint of different continual learning methods.
  Our findings suggest that replay with exemplars-based schemes such as iCaRL
has the best performance trade-offs, even in complex scenarios, at the expense
of some storage space (few MBs) for training examples (1% to 5%). We also
demonstrate for the first time that it is feasible and practical to run
continual learning on-device with a limited memory budget. In particular, the
latency on two types of mobile and embedded devices suggests that both
incremental learning time (few seconds - 4 minutes) and training time (1 - 75
minutes) across datasets are acceptable, as training could happen on the device
when the embedded device is charging thereby ensuring complete data privacy.
Finally, we present some guidelines for practitioners who want to apply a
continual learning paradigm for mobile sensing tasks.
