On Interactive Machine Learning and the Potential of Cognitive Feedback
  In order to increase productivity, capability, and data exploitation,
numerous defense applications are experiencing an integration of
state-of-the-art machine learning and AI into their architectures. Especially
for defense applications, having a human analyst in the loop is of high
interest due to quality control, accountability, and complex subject matter
expertise not readily automated or replicated by AI. However, many applications
are suffering from a very slow transition. This may be in large part due to
lack of trust, usability, and productivity, especially when adapting to
unforeseen classes and changes in mission context. Interactive machine learning
is a newly emerging field in which machine learning implementations are
trained, optimized, evaluated, and exploited through an intuitive
human-computer interface. In this paper, we introduce interactive machine
learning and explain its advantages and limitations within the context of
defense applications. Furthermore, we address several of the shortcomings of
interactive machine learning by discussing how cognitive feedback may inform
features, data, and results in the state of the art. We define the three
techniques by which cognitive feedback may be employed: self reporting,
implicit cognitive feedback, and modeled cognitive feedback. The advantages and
disadvantages of each technique are discussed.
