Self-Supervised Transformers for Activity Classification using Ambient
  Sensors
  Providing care for ageing populations is an onerous task, and as life
expectancy estimates continue to rise, the number of people that require senior
care is growing rapidly. This paper proposes a methodology based on Transformer
Neural Networks to classify the activities of a resident within an ambient
sensor based environment. We also propose a methodology to pre-train
Transformers in a self-supervised manner, as a hybrid autoencoder-classifier
model instead of using contrastive loss. The social impact of the research is
considered with wider benefits of the approach and next steps for identifying
transitions in human behaviour. In recent years there has been an increasing
drive for integrating sensor based technologies within care facilities for data
collection. This allows for employing machine learning for many aspects
including activity recognition and anomaly detection. Due to the sensitivity of
healthcare environments, some methods of data collection used in current
research are considered to be intrusive within the senior care industry,
including cameras for image based activity recognition, and wearables for
activity tracking, but recent studies have shown that using these methods
commonly result in poor data quality due to the lack of resident interest in
participating in data gathering. This has led to a focus on ambient sensors,
such as binary PIR motion, connected domestic appliances, and electricity and
water metering. By having consistency in ambient data collection, the quality
of data is considerably more reliable, presenting the opportunity to perform
classification with enhanced accuracy. Therefore, in this research we looked to
find an optimal way of using deep learning to classify human activity with
ambient sensor data.
