Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: The Efficiency Misnomer
(Submitted on 25 Oct 2021 (v1), last revised 16 Mar 2022 (this version, v2))
Abstract: Model efficiency is a critical aspect of developing and deploying machine learning models. Inference time and latency directly affect the user experience, and some applications have hard requirements. In addition to inference costs, model training also have direct financial and environmental impacts. Although there are numerous well-established metrics (cost indicators) for measuring model efficiency, researchers and practitioners often assume that these metrics are correlated with each other and report only few of them. In this paper, we thoroughly discuss common cost indicators, their advantages and disadvantages, and how they can contradict each other. We demonstrate how incomplete reporting of cost indicators can lead to partial conclusions and a blurred or incomplete picture of the practical considerations of different models. We further present suggestions to improve reporting of efficiency metrics.
Submission history
From: Mostafa Dehghani [view email][v1] Mon, 25 Oct 2021 12:48:07 GMT (77kb,D)
[v2] Wed, 16 Mar 2022 13:27:44 GMT (163kb,D)
Link back to: arXiv, form interface, contact.