Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
Understanding how ML models work is a prerequisite for responsibly designing,
deploying, and using ML-based systems. With interpretability approaches, ML can
now offer explanations for its outputs to aid human understanding. Though these
approaches rely on guidelines for how humans explain things to each other, they
ultimately solve for improving the artifact -- an explanation. In this paper,
we propose an alternate framework for interpretability grounded in Weick's
sensemaking theory, which focuses on who the explanation is intended for.
Recent work has advocated for the importance of understanding stakeholders'
needs -- we build on this by providing concrete properties (e.g., identity,
social context, environmental cues, etc.) that shape human understanding. We
use an application of sensemaking in organizations as a template for discussing
design guidelines for Sensible AI, AI that factors in the nuances of human
cognition when trying to explain itself.