Tiny Machine Learning for Concept Drift
  Tiny Machine Learning (TML) is a new research area whose goal is to design
machine and deep learning techniques able to operate in Embedded Systems and
IoT units, hence satisfying the severe technological constraints on memory,
computation, and energy characterizing these pervasive devices. Interestingly,
the related literature mainly focused on reducing the computational and memory
demand of the inference phase of machine and deep learning models. At the same
time, the training is typically assumed to be carried out in Cloud or edge
computing systems (due to the larger memory and computational requirements).
This assumption results in TML solutions that might become obsolete when the
process generating the data is affected by concept drift (e.g., due to
periodicity or seasonality effect, faults or malfunctioning affecting sensors
or actuators, or changes in the users' behavior), a common situation in
real-world application scenarios. For the first time in the literature, this
paper introduces a Tiny Machine Learning for Concept Drift (TML-CD) solution
based on deep learning feature extractors and a k-nearest neighbors classifier
integrating a hybrid adaptation module able to deal with concept drift
affecting the data-generating process. This adaptation module continuously
updates (in a passive way) the knowledge base of TML-CD and, at the same time,
employs a Change Detection Test to inspect for changes (in an active way) to
quickly adapt to concept drift by removing the obsolete knowledge. Experimental
results on both image and audio benchmarks show the effectiveness of the
proposed solution, whilst the porting of TML-CD on three off-the-shelf
micro-controller units shows the feasibility of what is proposed in real-world
pervasive systems.
