Predictive Maintenance for General Aviation Using Convolutional
  Transformers
  Predictive maintenance systems have the potential to significantly reduce
costs for maintaining aircraft fleets as well as provide improved safety by
detecting maintenance issues before they come severe. However, the development
of such systems has been limited due to a lack of publicly labeled multivariate
time series (MTS) sensor data. MTS classification has advanced greatly over the
past decade, but there is a lack of sufficiently challenging benchmarks for new
methods. This work introduces the NGAFID Maintenance Classification (NGAFID-MC)
dataset as a novel benchmark in terms of difficulty, number of samples, and
sequence length. NGAFID-MC consists of over 7,500 labeled flights, representing
over 11,500 hours of per second flight data recorder readings of 23 sensor
parameters. Using this benchmark, we demonstrate that Recurrent Neural Network
(RNN) methods are not well suited for capturing temporally distant
relationships and propose a new architecture called Convolutional Multiheaded
Self Attention (Conv-MHSA) that achieves greater classification performance at
greater computational efficiency. We also demonstrate that image inspired
augmentations of cutout, mixup, and cutmix, can be used to reduce overfitting
and improve generalization in MTS classification. Our best trained models have
been incorporated back into the NGAFID to allow users to potentially detect
flights that require maintenance as well as provide feedback to further expand
and refine the NGAFID-MC dataset.
