Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: On the benefits of maximum likelihood estimation for Regression and Forecasting
(Submitted on 18 Jun 2021 (v1), last revised 9 Oct 2021 (this version, v2))
Abstract: We advocate for a practical Maximum Likelihood Estimation (MLE) approach towards designing loss functions for regression and forecasting, as an alternative to the typical approach of direct empirical risk minimization on a specific target metric. The MLE approach is better suited to capture inductive biases such as prior domain knowledge in datasets, and can output post-hoc estimators at inference time that can optimize different types of target metrics. We present theoretical results to demonstrate that our approach is competitive with any estimator for the target metric under some general conditions. In two example practical settings, Poisson and Pareto regression, we show that our competitive results can be used to prove that the MLE approach has better excess risk bounds than directly minimizing the target metric. We also demonstrate empirically that our method instantiated with a well-designed general purpose mixture likelihood family can obtain superior performance for a variety of tasks across time-series forecasting and regression datasets with different data distributions.
Submission history
From: Rajat Sen [view email][v1] Fri, 18 Jun 2021 22:10:43 GMT (56kb)
[v2] Sat, 9 Oct 2021 18:06:56 GMT (160kb,D)
Link back to: arXiv, form interface, contact.