Misplaced Trust: Measuring the Interference of Machine Learning in Human
  Decision-Making
  ML decision-aid systems are increasingly common on the web, but their
successful integration relies on people trusting them appropriately: they
should use the system to fill in gaps in their ability, but recognize signals
that the system might be incorrect. We measured how people's trust in ML
recommendations differs by expertise and with more system information through a
task-based study of 175 adults. We used two tasks that are difficult for
humans: comparing large crowd sizes and identifying similar-looking animals.
Our results provide three key insights: (1) People trust incorrect ML
recommendations for tasks that they perform correctly the majority of the time,
even if they have high prior knowledge about ML or are given information
indicating the system is not confident in its prediction; (2) Four different
types of system information all increased people's trust in recommendations;
and (3) Math and logic skills may be as important as ML for decision-makers
working with ML recommendations.
