We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CV

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars

Abstract: Real-time tracking of human body motion is crucial for interactive and immersive experiences in AR/VR. However, very limited sensor data about the body is available from standalone wearable devices such as HMDs (Head Mounted Devices) or AR glasses. In this work, we present a reinforcement learning framework that takes in sparse signals from an HMD and two controllers, and simulates plausible and physically valid full body motions. Using high quality full body motion as dense supervision during training, a simple policy network can learn to output appropriate torques for the character to balance, walk, and jog, while closely following the input signals. Our results demonstrate surprisingly similar leg motions to ground truth without any observations of the lower body, even when the input is only the 6D transformations of the HMD. We also show that a single policy can be robust to diverse locomotion styles, different body sizes, and novel environments.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
Journal reference: SIGGRAPH Asia 2022 Conference Papers, December 6 to 9, 2022, Daegu, Republic of Korea
DOI: 10.1145/3550469.3555411
Cite as: arXiv:2209.09391 [cs.CV]
  (or arXiv:2209.09391v1 [cs.CV] for this version)

Submission history

From: Alexander W. Winkler [view email]
[v1] Tue, 20 Sep 2022 00:25:54 GMT (4856kb,D)

Link back to: arXiv, form interface, contact.