Perceiving Humans in 4D

Abstract: From the moment we open our eyes, we are surrounded by people. By observing the people around us, we learn how to interact with them and the world. To create intelligent agents with similar capabilities, it is crucial to endow them with a perceptual system that can interpret and understand human behavior from visual observations. These observations are streams of two-dimensional images; however, the actual underlying state of humans is 4D—they have 3D bodies that move over time. In this talk, I will present our recent work in the direction of perceiving humans in 4D from video. This includes estimating their articulated 3D body pose, recovering a consistent and coherent 4D reconstruction of their motion, as well as capturing their interactions with other people and objects in the 3D space. I will highlight the limitations of systems that only operate in the space of image pixels and showcase the benefits of reasoning in 4D.

Speaker Bio: Georgios Pavlakos is an Assistant Professor of Computer Science at UT Austin. His research interests include computer vision, machine learning, and robotics. Previously, he was a postdoctoral scholar at UC Berkeley, advised by Angjoo Kanazawa and Jitendra Malik. He completed his PhD in Computer Science at the University of Pennsylvania with his advisor, Kostas Daniilidis. He has spent time at Max Planck Institute with Michael Black and at Facebook Reality Labs. His PhD dissertation received the Morris and Dorothy Rubinoff Award for the Best Computer Science Dissertation at UPenn.