We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Measuring the Reliability of Reinforcement Learning Algorithms

Abstract: Lack of reliability is a well-known issue for reinforcement learning (RL) algorithms. This problem has gained increasing attention in recent years, and efforts to improve it have grown substantially. To aid RL researchers and production users with the evaluation and improvement of reliability, we propose a set of metrics that quantitatively measure different aspects of reliability. In this work, we focus on variability and risk, both during training and after learning (on a fixed policy). We designed these metrics to be general-purpose, and we also designed complementary statistical tests to enable rigorous comparisons on these metrics. In this paper, we first describe the desired properties of the metrics and their design, the aspects of reliability that they measure, and their applicability to different scenarios. We then describe the statistical tests and make additional practical recommendations for reporting results. The metrics and accompanying statistical tools have been made available as an open-source library at this https URL We apply our metrics to a set of common RL algorithms and environments, compare them, and analyze the results.
Comments: Accepted for publication at ICLR 2020 (spotlight)
Subjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:1912.05663 [stat.ML]
  (or arXiv:1912.05663v2 [stat.ML] for this version)

Submission history

From: Stephanie Chan [view email]
[v1] Tue, 10 Dec 2019 17:50:33 GMT (6425kb,D)
[v2] Wed, 12 Feb 2020 19:23:15 GMT (9253kb,D)

Link back to: arXiv, form interface, contact.