Center for Research in Comptuer Vision
Center for Research in Comptuer Vision

Seminar Announcement

Déjà Vécu: Learning from Legacy MoCap Data for Robust Human Action Recognition

Dr. Ajmal Mian of the University of Western Australia

Monday, December 11, 2017 · 4:00PM · HEC 101

Annotating human action videos is tedious and expensive. We bypass this step and capitalize on legacy motion capture (MoCap) data to synthesize video frames. The MoCap data is fitted with synthetic 3D humans with varying sizes, gender and clothing, placed in random backgrounds and rendered from 180 camera viewpoints under random lighting conditions. In essence, actions performed by real humans in the past are lived through in different bodies, clothes and locations giving us a large corpus of RGB and depth images where the exact human pose is known. Since, synthetic and real images come from different distributions, we perform unsupervised generative adversarial training to minimize the distribution gap. From the refined synthetic images, we learn Human Pose Models (CNNs) that map an input image to one of representative human poses learned by clustering the MoCap data. The trained CNNs generalize well to extract invariant features from real images. Fourier Temporal Pyramid over the CNN features is used to model videos and classification is performed with SVM. Experiments on three cross-view human action datasets show that our algorithm outperforms existing methods by significant margins for RGB only and RGB-D action recognition. Interestingly, our RGB only model outperforms existing RGB-D methods on the most challenging NTU dataset. Finally, I will show how our data synthesis paradigm generalizes to other applications such as human pose estimation, 3D face recognition and depth estimation from Light Field Images.

Ajmal Mian is an Associate Professor of Computer Science at The University of Western Australia. He has received several awards including the West Australian Early Career Scientist of the Year Award, the Vice-chancellors Mid-career Research Award, the Aspire Professional Development Award, the Outstanding Young Investigator Award, EH Thompson Award for best paper in photogrammetry and the IAPR Best Paper Award. He has received two prestigious fellowships and seven major grants from the Australian Research Council and the National Health and Medical Research Council with a total funding of over $3.0 Million. He has published 150 scientific papers and edited special issues in three journals. His major research interests are in computer vision, machine learning, 3D face analysis, human action recognition and hyperspectral image analysis. He has also published multidisciplinary research in marine science, neurodevelopmental disorders, agriculture, medicine and sleep science.