Regularizing Black-box Models for Improved Interpretability

Picture of Gregory Plumb
Gregory Plumb
Picture of Mauran Al-Shedivat
Mauran Al-Shedivat
Picture of Eric Xing
Eric Xing
Picture of Ameet Talwalkar
Ameet Talwalkar
Published at NeurIPS in Vancouver 2020
Teaser image


Most of the work on interpretable machine learning has focused on designing either inherently interpretable models, which typically trade-off accuracy for interpretability, or post-hoc explanation systems, whose explanation quality can be unpredictable. Our method, ExpO, is a hybridization of these approaches that regularizes a model for explanation quality at training time. Importantly, these regularizers are differentiable, model agnostic, and require no domain knowledge to define. We demonstrate that post-hoc explanations for ExpO-regularized models have better explanation quality, as measured by the common fidelity and stability metrics. We verify that improving these metrics leads to significantly more useful explanations with a user study on a realistic task.