A Brief Guide to Designing and Evaluating Human-Centered Interactive
  Machine Learning
  Interactive machine learning (IML) is a field of research that explores how
to leverage both human and computational abilities in decision making systems.
IML represents a collaboration between multiple complementary human and machine
intelligent systems working as a team, each with their own unique abilities and
limitations. This teamwork might mean that both systems take actions at the
same time, or in sequence. Two major open research questions in the field of
IML are: "How should we design systems that can learn to make better decisions
over time with human interaction?" and "How should we evaluate the design and
deployment of such systems?" A lack of appropriate consideration for the humans
involved can lead to problematic system behaviour, and issues of fairness,
accountability, and transparency. Thus, our goal with this work is to present a
human-centred guide to designing and evaluating IML systems while mitigating
risks. This guide is intended to be used by machine learning practitioners who
are responsible for the health, safety, and well-being of interacting humans.
An obligation of responsibility for public interaction means acting with
integrity, honesty, fairness, and abiding by applicable legal statutes. With
these values and principles in mind, we as a machine learning research
community can better achieve goals of augmenting human skills and abilities.
This practical guide therefore aims to support many of the responsible
decisions necessary throughout the iterative design, development, and
dissemination of IML systems.
