Dynamic Time Warping as a New Evaluation for Dst Forecast with Machine
  Learning
  Models based on neural networks and machine learning are seeing a rise in
popularity in space physics. In particular, the forecasting of geomagnetic
indices with neural network models is becoming a popular field of study. These
models are evaluated with metrics such as the root-mean-square error (RMSE) and
Pearson correlation coefficient. However, these classical metrics sometimes
fail to capture crucial behavior. To show where the classical metrics are
lacking, we trained a neural network, using a long short-term memory network,
to make a forecast of the disturbance storm time index at origin time $t$ with
a forecasting horizon of 1 up to 6 hours, trained on OMNIWeb data. Inspection
of the model's results with the correlation coefficient and RMSE indicated a
performance comparable to the latest publications. However, visual inspection
showed that the predictions made by the neural network were behaving similarly
to the persistence model. In this work, a new method is proposed to measure
whether two time series are shifted in time with respect to each other, such as
the persistence model output versus the observation. The new measure, based on
Dynamical Time Warping, is capable of identifying results made by the
persistence model and shows promising results in confirming the visual
observations of the neural network's output. Finally, different methodologies
for training the neural network are explored in order to remove the persistence
behavior from the results.
