Huai Yu

Huai Yu (余淮)

Postdoctoral Fellow of Robotics Institute, Carnegie Mellon University

About Me

I am a Postdoctor in the AirLab of Robotics Institute, Carnegie Mellon University (CMU) , working with Associate Research Professor Sebastian Scherer. Before that I was a PhD student of SPL under the supervision of Professor Wen Yang at Wuhan Univeristy. Now my research is on Computer Vision and Robotics. Particularly I am interested in 2D/3D visual correspondence for registration, calibration and camera localization.


Research Experience

2D-3D Registration and Localization - Robotics Institute, Canegie Mellon University, advisor: Prof. Sebastian Scherer (11/2018 - Present)

2D-3D registration for camera pose estimation using 2D-3D line correspondences

  • Efficient 2D/3D line segment feature extraction from imagery and point clouds.
  • Rotation estimation using vanishing points and camera pose estimation.
  • Real-time camera localization within prior LiDAR maps with 2D-3D line correspondences.

Change Detection - Signal Processing Lab, Wuhan University, advisor: Prof. Wen Yang (09/2017 - 10/2018)

Change detection using high resolution remote sensing imagery

  • An interactive object-based change detection using active learning with Gaussian processes.
  • The integration of attribute information and con-textural information using Markov Random Fields.

SLAM and robotic system Signal Processing Lab, Wuhan University, advisor: Prof. Wen Yang (01/2016 - 08/2017)

Vision based navigation and mapping system for ground robots and drones

  • Specific object detection, objected-based automatic navigation.
  • Participate in DJI Developer challenge 2016, ICRA 2017 Robot Challenge, ICRA 2018 AI Challenge.

Image Stitching and OBIA analysis Signal Processing Lab, Wuhan University, advisor: Prof. Wen Yang (10/2014 - 12/2015)

Feature based aerial image stitching and classification algorithm

  • A fast feature extraction and matching algorithm for UAV Images.
  • A Color-Texture-Structure descriptor using binary partition tree for semantic segmentation.

Tracking SmartCar Smart Car Group, Wuhan University, advisor: A.P. Jinsheng Xiao and Fei Yang (10/2013 - 08/2014)

Vision-based high speed lane-tracking

  • Lane detection, speed control and direction control algorithm.
  • Participate in the ninth "Freescale cup" smart car competition.

Selected Papers [Full Publications]

Conference Papers

Monocular Camera Localization in Prior LiDAR Maps with 2D-3D Line Correspondences

Huai Yu, Weikun Zhen, Wen Yang, Ji Zhang, Sebastian Scherer

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.

Paper Presentation

LiDAR Enhanced Structure-from-Motion

Weikun Zhen, Yaoyu Hu, Huai Yu, Sebastian Scherer

IEEE International Conference on Robotics and Automation (ICRA). 2020

Paper Demo

An Intelligent Unmanned Aircraft System for Wilderness Search and Rescue

Huai Yu, Shijie Lin, Jinwang Wang, Kaimin Fu, Wen Yang

International Micro Air Vehicles Conference and Flight Competition (IMAV), 2017


Journal Papers

ULSD: Unified Line Segment Detection across Pinhole, Fisheye, and Spherical Cameras

Hao Li*, Huai Yu*, Jinwang Wang, Wen Yang, Lei Yu, Sebastian Scherer (* cofirst author)

ISPRS Journal of Photogrammetry and Remote Sensing, 2021.

Paper Code Presentation

Line-based 2D-3D Registration and Camera Localization in Structured Environments

Paper Video

Huai Yu, Weikun Zhen, Wen Yang, Sebastian Scherer

IEEE Transactions on Instrument and Measurement, 2020.

Analysis of Large Scale UAV Images using A Multi-scale Hierarchical Representation


Huai Yu, Jinwang Wang, Yu Bai, Wen Yang, Gui-Song Xia

Geo-spatial Information Science, 2018.

Coarse-to-Fine Accurate Registration for Airborne SAR Images Using SAR-FAST and DSP-LATCH


Huai Yu, Wen Yang, Yan Liu

Progress In Electromagnetics Research, 2018.

Change Detection using High Resolution Remote Sensing Images based on Active Learning and Markov Random Fields


Huai Yu, Wen Yang, Guang Hua, Hui Ru, Pingping Huang

Remote Sensing, 2017.

A Color-Texture-Structure Descriptor for High-Resolution Satellite Image Classification


Huai Yu, Wen Yang, Gui-Song Xia, Gang Liu

Remote Sensing, 2016.

Selected Projects

ICRA2018 DJI RoboMaster AI Challenge (DOTA-AI), 2018
TensorFlow ROS Python C++ OpenCV SqueezeDet

In this project, we designed a multi-functional AI robot system, which can be used to evaluate the localization and navigation, object detection and tracking methods. We developed a robot localization algorithm based on mult-sensor fusion. It won 6th place in the DJI RoboMaster AI Challenge.


The Intelligent Unmanned Aircraft System for Wilderness Search and Rescue (WiSAR), 2017

In this project, we collected about 20,000 images to build a UAV perspective person dataset (UAV-PP) and adapted detection algorithms YOLOv2, SSD and Faster R-CNN for person detection and chose the SSD as the detection algorithm because it’s balance on accuracy and speed, the detection accuracy achieved 88.92%.

Demo Poster

ICRA2017 DJI RoboMaster Mobile Manipulation Challenge, 2017
C++ ROS OpenCV

In this project, we designed a robot to carry blocks from the starting point to the working area. It contain four main modules, i.e. mechanical module, global navigation module, local navigation module and the designed controlling logic. It won 6th place in the DJI RoboMaster Mobile Manipulation Challenge.


2016 DJI Developer Challenge
C++ ROS OpenCV

We developed a UAV system based on DJI M100 and a ground station to search objects (AprilTags) in wild with DJI Onboard and Mobile SDK. The robot can autonomous take-off and land on a moving platform. By using stereo vision and ROS navigation package, we also made it possible for autonomous search targets in the wild. It won 3th place in the 2016 DJI Developer Challenge.