How to Train Your Flare Prediction Model: Revisiting Robust Sampling of
  Rare Events
  We present a case study of solar flare forecasting by means of metadata
feature time series, by treating it as a prominent class-imbalance and
temporally coherent problem. Taking full advantage of pre-flare time series in
solar active regions is made possible via the Space Weather Analytics for Solar
Flares (SWAN-SF) benchmark dataset; a partitioned collection of multivariate
time series of active region properties comprising 4075 regions and spanning
over 9 years of the Solar Dynamics Observatory (SDO) period of operations. We
showcase the general concept of temporal coherence triggered by the demand of
continuity in time series forecasting and show that lack of proper
understanding of this effect may spuriously enhance models' performance. We
further address another well-known challenge in rare event prediction, namely,
the class-imbalance issue. The SWAN-SF is an appropriate dataset for this, with
a 60:1 imbalance ratio for GOES M- and X-class flares and a 800:1 for X-class
flares against flare-quiet instances. We revisit the main remedies for these
challenges and present several experiments to illustrate the exact impact that
each of these remedies may have on performance. Moreover, we acknowledge that
some basic data manipulation tasks such as data normalization and cross
validation may also impact the performance -- we discuss these problems as
well. In this framework we also review the primary advantages and disadvantages
of using true skill statistic and Heidke skill score, as two widely used
performance verification metrics for the flare forecasting task. In conclusion,
we show and advocate for the benefits of time series vs. point-in-time
forecasting, provided that the above challenges are measurably and
quantitatively addressed.
