How did "DRC-HUBO+" win the DARPA Robotics Challenge? 
– Robust Computer Vision Algorithms for "DRC-HUBO+"

For the intelligent robots to operate in very complex environments, it is essential to have a robust perception system using many different sensors, such as cameras and lidar depth sensors. There have been significant advances in the perception technology for intelligent robots in the last decade. We, however, encountered many problems in applying the state-of-the-art computer vision solutions to DRC-HUBO+ for the DRC challenge. It often failed to detect and grasp objects, such as a drill, a door handle, and a valve, which were important objects in the DRC missions.
    In this talk, we present the computer vision techniques to robustly detect those objects under challenging outdoor lighting conditions. Specifically, a simple camera distortion model shows much improvement in the camera-lidar calibration with the sub-pixel accuracy of re-projection error. A fusion algorithm for the camera and lidar sensors successfully aligns the color and depth images with the smallest error, as of today, in the Middlebury benchmarking data. The resulting fused images provide the accuracy required for the position-based DRC-HUBO+ to successfully carry out given missions in the DRC Finals, such as stair climbing, opening door, drilling a hole, and operating a valve. Our novel CNN network, called “AttentionNet”, is developed to accurately classify and localize the target objects in the given input image. We also present the DRC-HUBO+ robot system with a video clip of the DARPA Robotics Challenge (DRC) Finals. We also present the improvements and applications of the DRC-HUBO vision solutions for other real-world applications, such as intelligent vehicles and 3D displays.

In So Kweon received the B.S. and the M.S. degrees in Mechanical Design and Production Engineering from Seoul National University, Korea, in 1981 and 1983, respectively, and the Ph.D. degree in Robotics from the Robotics Institute at Carnegie Mellon University in 1990. He worked for Toshiba R&D Center, Japan, and joined KAIST in 1992. He is now a KEPCO Chair professor of School of Electrical Engineering and the director for the National Core Research Center – P3 DigiCar Center at KAIST. His research interests include computer vision and robotics. He has co-authored several books, including "Metric Invariants for Camera Calibration," and more than 500 technical papers. He served as a Founding Associate-Editor-in-Chief for “International Journal of Computer Vision and Applications”, and had been on the Editorial Board member for “International Journal of Computer Vision” for ten years since 2005. Professor Kweon is a member of many computer vision and robotics conference program committees and had been a general and program co-chairs for several conferences and workshops, including the 2012 ACCV. Most recently, he becomes a program chair of the 2019 ICCV. Professor Kweon received several awards from the international conferences, including “The Best Paper Award of the IEEE Transaction on CSVT 2014”, “The Best Student Paper Runnerup Award in the IEEE-CVPR 2009” and “The Student Paper Award in the ICCAS’2008”. He is a member of the KROS and IEEE.