Hunan University, China
Speech Title: Multimodal Intelligent Perception Technology and Applications of UAVs in Complex Power Scenarios
Abstract: To address challenges in drone inspection tasks for complex power scenarios, including infrared thermal fault detection, line vegetation classification, and tower tilt detection, this report proposes intelligent perception technologies based on multimodal information fusion. By integrating visible light images, infrared images, point cloud data, and multispectral data, it overcomes challenges such as environmental complexity, information incompleteness, and sensor perception limitations, significantly enhancing the perception and cognition capabilities of UAV systems in complex environments. The report focuses on the following key aspects: 1) Adaptive image registration and predictive information transfer techniques are proposed to address the spatial alignment of multimodal data, enabling precise localization of power equipment and accurate temperature interpretation; 2) A tree obstacle classification method based on point cloud and multispectral data fusion is designed, leveraging the complementarity of different modalities to accurately identify tree species in power corridors, thereby improving the accuracy and efficiency of inspection tasks; 3) Multimodal information-coordinated tower tilt detection and semantic segmentation technologies are developed, enhancing the intelligence level of power facility inspection in complex environments. Through multimodal data fusion and intelligent processing, this report demonstrates how multimodal sensing technologies can improve the efficiency, accuracy, and safety of drone inspections in complex power scenarios, effectively meeting the demands of national strategic needs.
Brief Bio: Hui Zhang, Ph.D., is a Professor (Second Class) and doctoral supervisor. He serves as Dean of the School of Artificial Intelligence and Robotics at Hunan University, Dean of the School of Future Technology at Hunan University, and Deputy Director of the National Engineering Research Center for Visual Sensing and Control. He was selected for the national high-level talent program, is a member of the Expert Group for the “14th Five-Year Plan” Key R&D Program on Intelligent Robotics under the Ministry of Science and Technology, an Executive Director of the Chinese Association of Automation, and Deputy Secretary-General of the Supervisory Board of the China Society of Image and Graphics. His research focuses on robotic visual inspection, deep-learning–based image recognition, and technologies and applications of intelligent manufacturing robots.
In recent years, he has led more than 20 projects, including a task under the Science and Technology Innovation 2030---“New Generation Artificial Intelligence” Major Project, two Key Projects of the National Natural Science Foundation of China, a key project under the JW1XX program, as well as subprojects of the National Key R&D Program and the National Science and Technology Support Program. He has published over 70 papers in domestic and international journals such as IEEE Transactions, holds 42 granted invention patents in China and 5 computer software copyrights. He received a Second Prize of the National Technological Invention Award in 2018; as first contributor, he led teams to win the First Prize of the Hunan Provincial Science and Technology Progress Award (2022), the Second Prize of the Hunan Provincial Science and Technology Progress Award (2019), and the First Prize of the Science and Technology Progress Award of the China General Chamber of Commerce (2019). As a principal contributor, he has received 15 provincial- and ministerial-level Science and Technology Progress Awards, the Special Prize of the 13th Hunan Provincial Teaching Achievement Awards (2022), and the Second Prize of the National Teaching Achievement Awards (Higher Education--Graduate Education, 2022).