Soft Sensing Transformer: Hundreds of Sensors are Worth a Single Word
  With the rapid development of AI technology in recent years, there have been
many studies with deep learning models in soft sensing area. However, the
models have become more complex, yet, the data sets remain limited: researchers
are fitting million-parameter models with hundreds of data samples, which is
insufficient to exercise the effectiveness of their models and thus often fail
to perform when implemented in industrial applications. To solve this
long-lasting problem, we are providing large scale, high dimensional time
series manufacturing sensor data from Seagate Technology to the public. We
demonstrate the challenges and effectiveness of modeling industrial big data by
a Soft Sensing Transformer model on these data sets. Transformer is used
because, it has outperformed state-of-the-art techniques in Natural Language
Processing, and since then has also performed well in the direct application to
computer vision without introduction of image-specific inductive biases. We
observe the similarity of a sentence structure to the sensor readings and
process the multi-variable sensor readings in a time series in a similar manner
of sentences in natural language. The high-dimensional time-series data is
formatted into the same shape of embedded sentences and fed into the
transformer model. The results show that transformer model outperforms the
benchmark models in soft sensing field based on auto-encoder and long
short-term memory (LSTM) models. To the best of our knowledge, we are the first
team in academia or industry to benchmark the performance of original
transformer model with large-scale numerical soft sensing data.
