HardVis: Visual Analytics to Handle Instance Hardness Using
  Undersampling and Oversampling Techniques
  Despite the tremendous advances in machine learning (ML), training with
imbalanced data still poses challenges in many real-world applications. Among a
series of diverse techniques to solve this problem, sampling algorithms are
regarded as an efficient solution. However, the problem is more fundamental,
with many works emphasizing the importance of instance hardness. This issue
refers to the significance of managing unsafe or potentially noisy instances
that are more likely to be misclassified and serve as the root cause of poor
classification performance. This paper introduces HardVis, a visual analytics
system designed to handle instance hardness mainly in imbalanced classification
scenarios. Our proposed system assists users in visually comparing different
distributions of data types, selecting types of instances based on local
characteristics that will later be affected by the active sampling method, and
validating which suggestions from undersampling or oversampling techniques are
beneficial for the ML model. Additionally, rather than uniformly
undersampling/oversampling a specific class, we allow users to find and sample
easy and difficult to classify training instances from all classes. Users can
explore subsets of data from different perspectives to decide all those
parameters, while HardVis keeps track of their steps and evaluates the model's
predictive performance in a test set separately. The end result is a
well-balanced data set that boosts the predictive power of the ML model. The
efficacy and effectiveness of HardVis are demonstrated with a hypothetical
usage scenario and a use case. Finally, we also look at how useful our system
is based on feedback we received from ML experts.
