"Sequential Determinantal Point Process: Modeling the Diverse and Sequential Properties in Video Summarization"
Dr. Boqing Gong of University of Southern California
Tuesday, April 7, 2015 · 3:30PM · HEC 118
Determinantal point processes (DPPs) were first used to characterize the Pauli exclusion principle, which states that two identical particles cannot occupy the same quantum state simultaneously. The notion of exclusion has made DPP an appealing tool to model *diversity* in applications such as document summarization and image ranking. In this talk, I will give a gentle introduction to DPPs and present the sequential DPP (seqDPP), a probabilistic model we proposed for modeling video summarization as a diverse sequential subset selection process.
Video summarization is a challenging problem with great application potential. Whereas prior approaches, largely unsupervised in nature, focus on sampling useful frames and assembling them as summaries, we consider video summarization as a supervised subset selection problem. Our idea is to teach the system to learn from human-created summaries how to select informative and diverse subsets, so as to best meet the evaluation metrics derived from human-perceived quality. To this end, we propose the seqDPP model to capture the inherent sequential structures in video data. Our model overcomes the deficiency of the standard DPP, which treats video frames as randomly permutable items. Meanwhile, seqDPP retains the power of modeling diverse subsets, essential for summarization. Our extensive results of summarizing videos from 3 datasets demonstrate the superior performance of our method, compared to not only existing unsupervised methods but also naive applications of the standard! DPP model.
Boqing Gong is a Ph.D. candidate at the University of Southern California and has defended his thesis. His research lies in the intersection between machine learning and computer vision, and has been focusing on the topics of domain adaptation, object recognition, understanding human activities from videos, video summarization, and 3D object retrieval. His work is partially supported by the Viterbi School of Engineering Doctoral Fellowship. Boqing holds a M.Phil. degree from the Chinese University of Hong Kong and a B.E. degree from the University of Science and Technology of China.