References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Hierarchical Kinematic Probability Distributions for 3D Human Shape and Pose Estimation from Images in the Wild
(Submitted on 3 Oct 2021 (v1), last revised 23 Nov 2022 (this version, v2))
Abstract: This paper addresses the problem of 3D human body shape and pose estimation from an RGB image. This is often an ill-posed problem, since multiple plausible 3D bodies may match the visual evidence present in the input - particularly when the subject is occluded. Thus, it is desirable to estimate a distribution over 3D body shape and pose conditioned on the input image instead of a single 3D reconstruction. We train a deep neural network to estimate a hierarchical matrix-Fisher distribution over relative 3D joint rotation matrices (i.e. body pose), which exploits the human body's kinematic tree structure, as well as a Gaussian distribution over SMPL body shape parameters. To further ensure that the predicted shape and pose distributions match the visual evidence in the input image, we implement a differentiable rejection sampler to impose a reprojection loss between ground-truth 2D joint coordinates and samples from the predicted distributions, projected onto the image plane. We show that our method is competitive with the state-of-the-art in terms of 3D shape and pose metrics on the SSP-3D and 3DPW datasets, while also yielding a structured probability distribution over 3D body shape and pose, with which we can meaningfully quantify prediction uncertainty and sample multiple plausible 3D reconstructions to explain a given input image. Code is available at this https URL .
Submission history
From: Akash Sengupta [view email][v1] Sun, 3 Oct 2021 11:59:37 GMT (25931kb,D)
[v2] Wed, 23 Nov 2022 15:01:11 GMT (6791kb,D)
Link back to: arXiv, form interface, contact.