Visual Simultaneous Localization and Mapping: From Geometry to Deep Learning

Visual Simultaneous Localization and Mapping (SLAM) is essential to achieve persistent autonomy for mobile robots in unknown environments, and is a key technique for enormous vision based applications, such as virtual and augmented reality. Researchers from the robotics and computer vision communities have endeavored and managed to design some efficient and versatile visual SLAM systems in the past several decades. However, how to achieve more accurate and robust localization and construct semantic map is still a challenging problem. The work in this thesis describes several novel algorithms to solve the above problem. Several novel methods that merge data driven approaches, such as deep learning, with visual SLAM techniques are proposed in order to gain better performance. Firstly, a novel model-based SLAM method based on points and plane-patches is proposed. Evaluational experiments on multiple benchmark dataset are performed to evaluate the proposed method. Secondly, a novel indoor relocalization system based on supervised deep learning is presented. Deep neural network is successfully applied for indoor pose regression. Thirdly, in order to solve the lack of labeled datasets. A novel unsupervised deep learning based visual SLAM system (called DeepSLAM) is proposed. The novel DeepSLAM includes Mapping-Net, Tracking-Net, Loop-Net, and a graph optimization unit. Experimental evaluations prove that the proposed DeepSLAM outperforms the state-of-the-art monocular SLAMs in terms of pose estimation accuracy, and is more robust than other SLAM systems in some challenging scenes. Finally, a novel spatio-temporal deep neural network for semantic segmentation is proposed with a SLAM system to achieve semantic 3D mapping. The proposed method is verified to be effective for 3D semantic mapping.