A 3D Tracking and Registration Method Based on Point Cloud and Visual Features for Augmented Reality Aided Assembly System

点云和视觉特征融合的增强现实装配系统三维跟踪注册方法

School of Mechanical Engineering, Northwestern Polytechnical University, Xi’an 710072, China

Received: 26 February 2018

Abstract

To improve the robustness and applicability of 3D tracking and registration for augmented reality(AR) aided mechanical assembly system, a 3D registration and tracking method based on the point cloud and visual features is proposed. Firstly, the reference model point cloud is used to definite absolute tracking coordinate system, thus the locating datum of the virtual assembly guidance information is determined. Then by adding visual features matching to the iterative closest points (ICP) registration process, the robustness of tracking and registration is improved. In order to obtain sufficient number of visual feature matching points in this process, a visual feature matching strategy based on orientation vector consistency is proposed. Finally, the loop closure detection and global pose optimization from key frames are added in the tracking registration process. The experimental result shows that the proposed method has good real-time performance and accuracy, and the running speed can reach 30 frames per second. Moreover, it also shows good robustness when the camera is moving fast and the depth information is inaccurate, and the comprehensive performance of the proposed method is better than the KinectFusion method.

摘要

为了提高三维跟踪注册方法面向机械产品增强现实装配引导的适用性和鲁棒性，提出了一种点云和视觉特征融合的三维跟踪注册方法。首先利用参考模型点云对三维跟踪注册绝对坐标系进行定义，从而确定虚拟装配引导信息的定位基准。然后在迭代最近点法点云数据配准基础上，结合深度传感器彩色图像信息，通过视觉特征匹配，提高深度传感器快速移动时的跟踪注册过程鲁棒性。为了在此过程获取足够数量的视觉特征匹配点对，提出了一种基于方向向量一致性的视觉特征匹配策略。最后在跟踪注册过程加入基于关键帧的回环检测和全局位姿优化。实验结果表明：新方法精确性、实时性好，能达到每秒30帧。而且在相机快速移动时仍能表现出较好的鲁棒性，其综合性能优于基于点云的Kinect Fusion方法。