KDCTime: Knowledge Distillation with Calibration on InceptionTime for
  Time-series Classification
  Time-series classification approaches based on deep neural networks are easy
to be overfitting on UCR datasets, which is caused by the few-shot problem of
those datasets. Therefore, in order to alleviate the overfitting phenomenon for
further improving the accuracy, we first propose Label Smoothing for
InceptionTime (LSTime), which adopts the information of soft labels compared to
just hard labels. Next, instead of manually adjusting soft labels by LSTime,
Knowledge Distillation for InceptionTime (KDTime) is proposed in order to
automatically generate soft labels by the teacher model. At last, in order to
rectify the incorrect predicted soft labels from the teacher model, Knowledge
Distillation with Calibration for InceptionTime (KDCTime) is proposed, where it
contains two optional calibrating strategies, i.e. KDC by Translating (KDCT)
and KDC by Reordering (KDCR). The experimental results show that the accuracy
of KDCTime is promising, while its inference time is two orders of magnitude
faster than ROCKET with an acceptable training time overhead.
