"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials
  for Humans
  To support human decision making with machine learning models, we often need
to elucidate patterns embedded in the models that are unsalient, unknown, or
counterintuitive to humans. While existing approaches focus on explaining
machine predictions with real-time assistance, we explore model-driven
tutorials to help humans understand these patterns in a training phase. We
consider both tutorials with guidelines from scientific papers, analogous to
current practices of science communication, and automatically selected examples
from training data with explanations. We use deceptive review detection as a
testbed and conduct large-scale, randomized human-subject experiments to
examine the effectiveness of such tutorials. We find that tutorials indeed
improve human performance, with and without real-time assistance. In
particular, although deep learning provides superior predictive performance
than simple models, tutorials and explanations from simple models are more
useful to humans. Our work suggests future directions for human-centered
tutorials and explanations towards a synergy between humans and AI.
