Calibrating for Class Weights by Modeling Machine Learning
  A much studied issue is the extent to which the confidence scores provided by
machine learning algorithms are calibrated to ground truth probabilities. Our
starting point is that calibration is seemingly incompatible with class
weighting, a technique often employed when one class is less common (class
imbalance) or with the hope of achieving some external objective
(cost-sensitive learning). We provide a model-based explanation for this
incompatibility and use our anthropomorphic model to generate a simple method
of recovering likelihoods from an algorithm that is miscalibrated due to class
weighting. We validate this approach in the binary pneumonia detection task of
Rajpurkar, Irvin, Zhu, et al. (2017).
