Generative Pre-Trained Transformer for Cardiac Abnormality Detection
  ECG heartbeat classification plays a vital role in diagnosis of cardiac
arrhythmia. The goal of the Physionet/CinC 2021 challenge was to accurately
classify clinical diagnosis based on 12, 6, 4, 3 or 2-lead ECG recordings in
order to aid doctors in the diagnoses of different heart conditions.
Transformers have had great success in the field of natural language processing
in the past years. Our team, CinCSEM, proposes to draw the parallel between
text and periodic time series signals by viewing the repeated period as words
and the whole signal as a sequence of such words. In this way, the attention
mechanisms of the transformers can be applied to periodic time series signals.
In our implementation, we follow the Transformer Encoder architecture, which
combines several encoder layers followed by a dense layer with linear or
sigmoid activation for generative pre-training or classification, respectively.
The use case presented here is multi-label classification of heartbeat
abnormalities of ECG recordings shared by the challenge. Our best entry, not
exceeding the challenge's hardware limitations, achieved a score of 0.12, 0.07,
0.10, 0.10 and 0.07 on 12-lead, 6-lead, 4-lead, 3-lead and 2-lead test set
respectively. Unfortunately, our team was unable to be ranked because of a
missing pre-print.
