Toward a Taxonomy of Trust for Probabilistic Machine Learning
  Probabilistic machine learning increasingly informs critical decisions in
medicine, economics, politics, and beyond. We need evidence to support that the
resulting decisions are well-founded. To aid development of trust in these
decisions, we develop a taxonomy delineating where trust in an analysis can
break down: (1) in the translation of real-world goals to goals on a particular
set of available training data, (2) in the translation of abstract goals on the
training data to a concrete mathematical problem, (3) in the use of an
algorithm to solve the stated mathematical problem, and (4) in the use of a
particular code implementation of the chosen algorithm. We detail how trust can
fail at each step and illustrate our taxonomy with two case studies: an
analysis of the efficacy of microcredit and The Economist's predictions of the
2020 US presidential election. Finally, we describe a wide variety of methods
that can be used to increase trust at each step of our taxonomy. The use of our
taxonomy highlights steps where existing research work on trust tends to
concentrate and also steps where establishing trust is particularly
challenging.
