Sense and Learn: Self-Supervision for Omnipresent Sensors
  Learning general-purpose representations from multisensor data produced by
the omnipresent sensing systems (or IoT in general) has numerous applications
in diverse use cases. Existing purely supervised end-to-end deep learning
techniques depend on the availability of a massive amount of well-curated data,
acquiring which is notoriously difficult but required to achieve a sufficient
level of generalization on a task of interest. In this work, we leverage the
self-supervised learning paradigm towards realizing the vision of continual
learning from unlabeled inputs. We present a generalized framework named Sense
and Learn for representation or feature learning from raw sensory data. It
consists of several auxiliary tasks that can learn high-level and broadly
useful features entirely from unannotated data without any human involvement in
the tedious labeling process. We demonstrate the efficacy of our approach on
several publicly available datasets from different domains and in various
settings, including linear separability, semi-supervised or few shot learning,
and transfer learning. Our methodology achieves results that are competitive
with the supervised approaches and close the gap through fine-tuning a network
while learning the downstream tasks in most cases. In particular, we show that
the self-supervised network can be utilized as initialization to significantly
boost the performance in a low-data regime with as few as 5 labeled instances
per class, which is of high practical importance to real-world problems.
Likewise, the learned representations with self-supervision are found to be
highly transferable between related datasets, even when few labeled instances
are available from the target domains. The self-learning nature of our
methodology opens up exciting possibilities for on-device continual learning.
