We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Robotics

Title: Prescient teleoperation of humanoid robots

Abstract: Humanoid robots could be versatile and intuitive human avatars that operate remotely in inaccessible places: the robot could reproduce in the remote location the movements of an operator equipped with a wearable motion capture device while sending visual feedback to the operator. While substantial progress has been made on transferring ("retargeting") human motions to humanoid robots, a major problem preventing the deployment of such systems in real applications is the presence of communication delays between the human input and the feedback from the robot: even a few hundred milliseconds of delay can irreversibly disturb the operator, let alone a few seconds. To overcome these delays, we introduce a system in which a humanoid robot executes commands before it actually receives them, so that the visual feedback appears to be synchronized to the operator, whereas the robot executed the commands in the past. To do so, the robot continuously predicts future commands by querying a machine learning model that is trained on past trajectories and conditioned on the last received commands. In our experiments, an operator was able to successfully control a humanoid robot (32 degrees of freedom) with stochastic delays up to 2 seconds in several whole-body manipulation tasks, including reaching different targets, picking up a bottle, and placing a box at distinct locations.
Comments: Video: this https URL
Subjects: Robotics (cs.RO); Machine Learning (cs.LG)
Cite as: arXiv:2107.01281 [cs.RO]
  (or arXiv:2107.01281v3 [cs.RO] for this version)

Submission history

From: Jean-Baptiste Mouret [view email]
[v1] Fri, 2 Jul 2021 21:10:35 GMT (36414kb,D)
[v2] Mon, 12 Jul 2021 15:26:57 GMT (21379kb,D)
[v3] Mon, 28 Mar 2022 16:14:51 GMT (16313kb,D)

Link back to: arXiv, form interface, contact.