We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration

Abstract: The generative adversarial imitation learning (GAIL) has provided an adversarial learning framework for imitating expert policy from demonstrations in high-dimensional continuous tasks. However, almost all GAIL and its extensions only design a kind of reward function of logarithmic form in the adversarial training strategy with the Jensen-Shannon (JS) divergence for all complex environments. The fixed logarithmic type of reward function may be difficult to solve all complex tasks, and the vanishing gradients problem caused by the JS divergence will harm the adversarial learning process. In this paper, we propose a new algorithm named Wasserstein Distance guided Adversarial Imitation Learning (WDAIL) for promoting the performance of imitation learning (IL). There are three improvements in our method: (a) introducing the Wasserstein distance to obtain more appropriate measure in the adversarial training process, (b) using proximal policy optimization (PPO) in the reinforcement learning stage which is much simpler to implement and makes the algorithm more efficient, and (c) exploring different reward function shapes to suit different tasks for improving the performance. The experiment results show that the learning procedure remains remarkably stable, and achieves significant performance in the complex continuous control tasks of MuJoCo.
Comments: M. Zhang and Y. Wang contribute equally to this work
Subjects: Machine Learning (cs.LG); Robotics (cs.RO); Machine Learning (stat.ML)
Journal reference: 2020 IEEE 9th Data Driven Control and Learning Systems Conference (DDCLS)
DOI: 10.1109/DDCLS49620.2020.9275169
Cite as: arXiv:2006.03503 [cs.LG]
  (or arXiv:2006.03503v2 [cs.LG] for this version)

Submission history

From: Ming Zhang [view email]
[v1] Fri, 5 Jun 2020 15:10:00 GMT (1239kb,D)
[v2] Tue, 8 Dec 2020 13:06:20 GMT (1239kb,D)

Link back to: arXiv, form interface, contact.