Generalizing 3D Human Shape and Pose Estimation for Diverse Non-Adult Populations

Abstract

The accurate three-dimensional (3D) shape and pose estimation of humans from a single image constitutes a fundamental and complex problem in Computer Vision, with critical applications spanning health, biomechanics, Virtual Reality (VR), and animation. Despite significant advancements for the adult population, the majority of current methods fail to generalize effectively to children and infants due to their unique anthropometric proportions and the scarcity of specialized datasets required for model training. This diploma thesis addresses this challenge by introducing a comprehensive framework designed to bridge this domain gap. We propose an optimization-based method that extends a top-performing model by incorporating the SMPL-A body model, enabling the concurrent and accurate modeling of adults, children, and infants. Leveraging this approach, we generated pseudo-ground-truth annotations for publicly available databases of child and infant images. Utilizing this new training data, we then developed and trained a specialized transformer-based Deep Learning model capable of real-time 3D human reconstruction. Furthermore, we introduce the BabyRobot dataset, which contains the 3D reconstructions produced by our method from videos of children interacting with robots with many actions, gestures and movements in the environment. Our methods contribute to the anonymization of sensitive data, like that of children and infants, since the 3D reconstructions provide information about the body and the motion of humans, but not their identity. Our results demonstrate a substantial improvement in the quality of shape and pose estimation for child and infant images, while simultaneously maintaining high performance across the adult population.