We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.AP

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Applications

Title: A scoring framework for tiered warnings and multicategorical forecasts based on fixed risk measures

Abstract: The use of tiered warnings and multicategorical forecasts are ubiquitous in meteorological operations. Here, a flexible family of scoring functions is presented for evaluating the performance of ordered multicategorical forecasts. Each score has a risk parameter $\alpha$, selected for the specific use case, so that it is consistent with a forecast directive based on the fixed threshold probability $1-\alpha$ (equivalently, a fixed $\alpha$-quantile mapping). Each score also has use-case specific weights so that forecasters who accurately discriminate between categorical thresholds are rewarded in proportion to the weight for that threshold. A variation is presented where the penalty assigned to near misses or close false alarms is discounted, which again is consistent with directives based on fixed risk measures. The scores presented provide an alternative to many performance measures currently in use, whose optimal threshold probabilities for forecasting an event typically vary with each forecast case, and in the case of equitable scores are based around sample base rates rather than risk measures suitable for users.
Comments: 24 pages, 5 figures, 2 tables. This revision contains a section on confidence interval estimation and substantial rearrangement of material. The core results are unchanged. Quarterly Journal of the Royal Meteorological Society (2022)
Subjects: Applications (stat.AP)
MSC classes: 62C05, 91B06
DOI: 10.1002/qj.4206
Cite as: arXiv:2108.12814 [stat.AP]
  (or arXiv:2108.12814v2 [stat.AP] for this version)

Submission history

From: Robert Taggart [view email]
[v1] Sun, 29 Aug 2021 10:30:53 GMT (177kb,D)
[v2] Fri, 29 Apr 2022 09:30:58 GMT (202kb)

Link back to: arXiv, form interface, contact.