How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions
Hao-Fei Cheng
Logan Stapleton
Anna Kawakami
Yanghuidi Cheng
Diana Qing
Zhiwei Steven Wu
Haiyi Zhu
Published at
CHI
in New Orleans, LA
2022

Abstract
Machine learning tools have been deployed in various contexts to support human
decision-making, in the hope that human-algorithm collaboration can improve
decision quality. However, the question of whether such collaborations reduce or
exacerbate biases in decision-making remains underexplored. In this work, we
conducted a mixed-methods study, analyzing child welfare call screen workers'
decision-making over a span of four years, and interviewing them on how they
incorporate algorithmic predictions into their decision-making process. Our data
analysis shows that, compared to the algorithm alone, workers reduced the
disparity in screen-in rate between Black and white children from 20% to 9%. Our
qualitative data show that workers achieved this by making holistic risk
assessments and adjusting for the algorithm's limitations. Our analyses also
show more nuanced results about how human-algorithm collaboration affects
prediction accuracy, and how to measure these effects. These results shed light
on potential mechanisms for improving human-algorithm collaboration in high-risk
decision-making contexts.
Two co-first authors contributed equally to this work.