Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference
Chaoqun Wang
Jinmook Lee
Jeffrey P Bigham
Zhile Ren
Cecile Foret
Qi Shan
Xiaoyi Zhang
Published at
CHI
| Honolulu, HI
2024
- Best Paper Honorable Mention
Abstract
On-device machine learning (ML) moves computation from the cloud to personal
devices, protecting user privacy and enabling intelligent user experiences.
However, fitting models on devices with limited resources presents a major
technical challenge: practitioners need to optimize models and balance hardware
metrics such as model size, latency, and power. To help practitioners create
efficient ML models, we designed and developed Talaria: a model visualization
and optimization system. Talaria enables practitioners to compile models to
hardware, interactively visualize model statistics, and simulate optimizations
to test the impact on inference metrics. Since its internal deployment two years
ago, we have evaluated Talaria using three methodologies: (1) a log analysis
highlighting its growth of 800+ practitioners submitting 3,600+ models; (2) a
usability survey with 26 users assessing the utility of 20 Talaria features; and
(3) a qualitative interview with the 7 most active users about their experience
using Talaria.