Do shallow kernel methods match deep neural networks - and if not, what can the shallow ones learn from the deep ones?
Dr. Fei Sha of the University of California, Los Angeles
Thursday, March 31, 2016 · 2:00PM · HEC 101
Deep neural networks (DNNs) and other types of deep learning architectures have been hugely successful in a large number of applications. In contrast, kernel methods, which were exceedingly popular, have become lackluster. The crippling obstacle is the computational complexity of those methods. Nonetheless, there has been a recently resurgent interest in them. In particular, several research groups have studied how to scale kernel methods to cope with large-scale learning problems.
Despite those progresses, there has not been a systematic and head-on comparison between kernel methods and DNNs. Specifically, while recent approaches have shown exciting promises, we are still left with at least one itching question unanswered: can kernel methods, after being scaled up for large-scale datasets, truly match DNNs' performance?
In this talk, I will describe our efforts in (partially) answering that question. I will present extensive empirical studies of comparing kernel methods and DNNs for automatic speech recognition, a key field to which DNNs have been applied. Our investigative studies highlight the similarity and difference of those two paradigms. I will leave our main conclusion out as a teaser to this talk.
Dr. Fei Sha is an associate professor and Saemuli Fellow at UCLA's Computer Science Department. Previously, he was the Jack Munushian Early Career Chair and an associate professor at the University of Southern California, Dept. of Computer Science. His primary research interests are machine learning and its application to speech and language processing, computer vision, and robotics. He had won outstanding student paper awards at NIPS 2006 and ICML 2004. He was selected as a Sloan Research Fellow in 2013, won an Army Research Office Young Investigator Award in 2012, and was a member of DARPA 2010 Computer Science Study Panel. He has a Ph.D (2007) from Computer and Information Science from U. of Pennsylvania and B.Sc and M.Sc from Southeast University (Nanjing, China)