We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Evaluating the Performance of Reinforcement Learning Algorithms

Abstract: Performance evaluations are critical for quantifying algorithmic advances in reinforcement learning. Recent reproducibility analyses have shown that reported performance results are often inconsistent and difficult to replicate. In this work, we argue that the inconsistency of performance stems from the use of flawed evaluation metrics. Taking a step towards ensuring that reported results are consistent, we propose a new comprehensive evaluation methodology for reinforcement learning algorithms that produces reliable measurements of performance both on a single environment and when aggregated across environments. We demonstrate this method by evaluating a broad class of reinforcement learning algorithms on standard benchmark tasks.
Comments: 30 pages, 9 figures, Thirty-seventh International Conference on Machine Learning (ICML 2020)
Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Cite as: arXiv:2006.16958 [cs.LG]
  (or arXiv:2006.16958v2 [cs.LG] for this version)

Submission history

From: Scott Jordan [view email]
[v1] Tue, 30 Jun 2020 16:52:23 GMT (3192kb,D)
[v2] Thu, 13 Aug 2020 16:05:28 GMT (3192kb,D)

Link back to: arXiv, form interface, contact.