Beneficial and Harmful Explanatory Machine Learning
  Given the recent successes of Deep Learning in AI there has been increased
interest in the role and need for explanations in machine learned theories. A
distinct notion in this context is that of Michie's definition of Ultra-Strong
Machine Learning (USML). USML is demonstrated by a measurable increase in human
performance of a task following provision to the human of a symbolic machine
learned theory for task performance. A recent paper demonstrates the beneficial
effect of a machine learned logic theory for a classification task, yet no
existing work to our knowledge has examined the potential harmfulness of
machine's involvement for human comprehension during learning. This paper
investigates the explanatory effects of a machine learned theory in the context
of simple two person games and proposes a framework for identifying the
harmfulness of machine explanations based on the Cognitive Science literature.
The approach involves a cognitive window consisting of two quantifiable bounds
and it is supported by empirical evidence collected from human trials. Our
quantitative and qualitative results indicate that human learning aided by a
symbolic machine learned theory which satisfies a cognitive window has achieved
significantly higher performance than human self learning. Results also
demonstrate that human learning aided by a symbolic machine learned theory that
fails to satisfy this window leads to significantly worse performance than
unaided human learning.
