Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels
Rahul Nair
Published at
CHI
| New Orleans, LA
2022
- Best Paper Award
Abstract
The confusion matrix, a ubiquitous visualization for helping people evaluate
machine learning models, is a tabular layout that compares predicted class
labels against actual class labels over all data instances. We conduct formative
research with machine learning practitioners at Apple and find that conventional
confusion matrices do not support more complex data-structures found in
modern-day applications, such as hierarchical and multi-output labels. To
express such variations of confusion matrices, we design an algebra that models
confusion matrices as probability distributions. Based on this algebra, we
develop Neo, a visual analytics system that enables practitioners to flexibly
author and interact with hierarchical and multi-output confusion matrices,
visualize derived metrics, renormalize confusions, and share matrix
specifications. Finally, we demonstrate Neo's utility with three model
evaluation scenarios that help people better understand model performance and
reveal hidden confusions.