Why Does My Model Fail? Contrastive Local Explanations for Retail
  Forecasting
  In various business settings, there is an interest in using more complex
machine learning techniques for sales forecasting. It is difficult to convince
analysts, along with their superiors, to adopt these techniques since the
models are considered to be "black boxes," even if they perform better than
current models in use. We examine the impact of contrastive explanations about
large errors on users' attitudes towards a "black-box'" model. We propose an
algorithm, Monte Carlo Bounds for Reasonable Predictions. Given a large error,
MC-BRP determines (1) feature values that would result in a reasonable
prediction, and (2) general trends between each feature and the target, both
based on Monte Carlo simulations. We evaluate on a real dataset with real users
by conducting a user study with 75 participants to determine if explanations
generated by MC-BRP help users understand why a prediction results in a large
error, and if this promotes trust in an automatically-learned model. Our study
shows that users are able to answer objective questions about the model's
predictions with overall 81.1% accuracy when provided with these contrastive
explanations. We show that users who saw MC-BRP explanations understand why the
model makes large errors in predictions significantly more than users in the
control group. We also conduct an in-depth analysis on the difference in
attitudes between Practitioners and Researchers, and confirm that our results
hold when conditioning on the users' background.
