We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Learning to Reach Goals via Iterated Supervised Learning

Abstract: Current reinforcement learning (RL) algorithms can be brittle and difficult to use, especially when learning goal-reaching behaviors from sparse rewards. Although supervised imitation learning provides a simple and stable alternative, it requires access to demonstrations from a human supervisor. In this paper, we study RL algorithms for learning goal reaching policies that leverage the stability of imitation learning without the need for explicit expert demonstrations. In lieu of expert demonstrations, supervision can be derived by leveraging the property that any trajectory is a successful demonstration for reaching the final state in that same trajectory. We propose a simple algorithm in which an agent continually relabels and imitates its own experience to progressively learn goal-reaching behaviors. Each iteration, the agent collects new trajectories using the latest policy, and maximizes the likelihood of the actions along these trajectories under the goal that was actually reached, so as to improve the policy. We formally link our supervised learning objective to the true RL objective, derive performance bounds, and demonstrate improved performance over current RL algorithms on goal-reaching in several benchmark tasks.
Comments: First two authors contributed equally. Code available at this https URL
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
Cite as: arXiv:1912.06088 [cs.LG]
  (or arXiv:1912.06088v3 [cs.LG] for this version)

Submission history

From: Abhishek Gupta [view email]
[v1] Thu, 12 Dec 2019 17:26:47 GMT (2110kb,D)
[v2] Fri, 13 Dec 2019 01:42:38 GMT (2104kb,D)
[v3] Wed, 10 Jun 2020 17:22:46 GMT (4353kb,D)

Link back to: arXiv, form interface, contact.