We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.AI

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Artificial Intelligence

Title: An Imitation from Observation Approach to Transfer Learning with Dynamics Mismatch

Abstract: We examine the problem of transferring a policy learned in a source environment to a target environment with different dynamics, particularly in the case where it is critical to reduce the amount of interaction with the target environment during learning. This problem is particularly important in sim-to-real transfer because simulators inevitably model real-world dynamics imperfectly. In this paper, we show that one existing solution to this transfer problem - grounded action transformation - is closely related to the problem of imitation from observation (IfO): learning behaviors that mimic the observations of behavior demonstrations. After establishing this relationship, we hypothesize that recent state-of-the-art approaches from the IfO literature can be effectively repurposed for grounded transfer learning.To validate our hypothesis we derive a new algorithm - generative adversarial reinforced action transformation (GARAT) - based on adversarial imitation from observation techniques. We run experiments in several domains with mismatched dynamics, and find that agents trained with GARAT achieve higher returns in the target environment compared to existing black-box transfer methods
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Journal reference: Neural Information Processing Systems (NeurIPS 2020)
Cite as: arXiv:2008.01594 [cs.AI]
  (or arXiv:2008.01594v3 [cs.AI] for this version)

Submission history

From: Haresh Karnan [view email]
[v1] Tue, 4 Aug 2020 14:36:02 GMT (2043kb,D)
[v2] Thu, 5 Nov 2020 17:47:30 GMT (1818kb,D)
[v3] Mon, 16 Nov 2020 22:58:45 GMT (1784kb,D)

Link back to: arXiv, form interface, contact.