A Framework for Designing and Evaluating Solar Flare Forecasting Systems
  Disturbances in space weather can negatively affect several fields, including
aviation and aerospace, satellites, oil and gas industries, and electrical
systems, leading to economic and commercial losses. Solar flares are the most
significant events that can affect the Earth's atmosphere, thus leading
researchers to drive efforts on their forecasting. The related literature is
comprehensive and holds several systems proposed for flare forecasting.
However, most techniques are tailor-made and designed for specific purposes,
not allowing researchers to customize them in case of changes in data input or
in the prediction algorithm. This paper proposes a framework to design, train,
and evaluate flare prediction systems which present promising results. Our
proposed framework involves model and feature selection, randomized
hyper-parameters optimization, data resampling, and evaluation under
operational settings. Compared to baseline predictions, our framework generated
some proof-of-concept models with positive recalls between 0.70 and 0.75 for
forecasting $\geq M$ class flares up to 96 hours ahead while keeping the area
under the ROC curve score at high levels.
