New Frontiers of Large Scale Multimedia Information Analysis and Retrieval

Despite the rapid progress in automatic perception (such as face and speech recognition), many fundamental challenges remain open in exploiting the full power of multimedia content in various applications, such as large-scale reverse search, detecting fake content, live video event monitoring, caption generation, and question-answering using multimedia corpora. In this talk, I will present the open issues facing researchers in these areas, discuss some prevailing statistical and deep learning models adopted in achieving state-of-the-art solutions, and share demos of some pilot systems. 

Shih-Fu Chang’s research is dedicated to multimedia information retrieval, computer vision, machine learning, and signal processing. In the early 90's, his group developed some of the earliest and best-known content-based image/video search systems, such as VisualSEEk and VideoQ, which helped set the foundation of this vibrant area. During last two decades, he has made contributions in shaping the field of multimedia retrieval by developing large multimedia ontologies, large libraries of visual concept classifies, and new methodologies for automatic multimedia ontology construction. In addition, he has developed a series of fundamental methods of semi-supervised graph-based learning to address the challenge of large-scale multimedia retrieval with noisy and sparse labels, and compact hashing techniques for rapid search over billion-scale image databases. With many paper awards, issued patents, and several spinoff companies, he has been awarded the IEEE Signal Processing Society Technical Achievement Award, ACM Multimedia SIG Technical Achievement Award, Honorary Doctorate from the University of Amsterdam, the IEEE Kiyo Tomiyasu Award, and IBM Faculty Award. He received the Great Teacher Award from the Society of Columbia Graduates. In his current capacity as Senior Executive Vice Dean in Columbia Engineering, he plays a key role in the School's strategic planning, research initiatives, international collaboration, and faculty development. He is a Fellow of the American Association for the Advancement of Science (AAAS), Fellow of IEEE, and listed among the most influential scholars in multimedia by AMiner in 2016.