Neo: Generalizing Confusion Matrix Visualization to Hierarchical and
  Multi-Output Labels
  The confusion matrix, a ubiquitous visualization for helping people evaluate
machine learning models, is a tabular layout that compares predicted class
labels against actual class labels over all data instances. We conduct
formative research with machine learning practitioners at Apple and find that
conventional confusion matrices do not support more complex data-structures
found in modern-day applications, such as hierarchical and multi-output labels.
To express such variations of confusion matrices, we design an algebra that
models confusion matrices as probability distributions. Based on this algebra,
we develop Neo, a visual analytics system that enables practitioners to
flexibly author and interact with hierarchical and multi-output confusion
matrices, visualize derived metrics, renormalize confusions, and share matrix
specifications. Finally, we demonstrate Neo's utility with three model
evaluation scenarios that help people better understand model performance and
reveal hidden confusions.
