Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: See, Hear, Explore: Curiosity via Audio-Visual Association
(Submitted on 7 Jul 2020 (v1), last revised 18 Jan 2021 (this version, v2))
Abstract: Exploration is one of the core challenges in reinforcement learning. A common formulation of curiosity-driven exploration uses the difference between the real future and the future predicted by a learned model. However, predicting the future is an inherently difficult task which can be ill-posed in the face of stochasticity. In this paper, we introduce an alternative form of curiosity that rewards novel associations between different senses. Our approach exploits multiple modalities to provide a stronger signal for more efficient exploration. Our method is inspired by the fact that, for humans, both sight and sound play a critical role in exploration. We present results on several Atari environments and Habitat (a photorealistic navigation simulator), showing the benefits of using an audio-visual association model for intrinsically guiding learning agents in the absence of external rewards. For videos and code, see this https URL
Submission history
From: Victoria Dean [view email][v1] Tue, 7 Jul 2020 17:56:35 GMT (5736kb,D)
[v2] Mon, 18 Jan 2021 16:26:54 GMT (13716kb,D)
Link back to: arXiv, form interface, contact.